A few weeks ago Jon Hiller showed some ideas for next-level robot programming at our weekly engineering meeting.  We got to talking about boxes-and-arrows programming or “visual” programming or whatever you might want to call it.

We’re pretty into visual programming.  Cubelets take visual programming to the extreme and make it tangible.  When you snap Cubelets together, you’re snapping little plastic blocks of programming to each other, and they immediately begin to pass data.  But we like visual programming in the traditional sense too — the MOSS Scratch extension has proven to be popular not only for our users and our education team, but also for our design team: engineers at Modular Robotics usually default to Scratch when mocking up new robots and behaviors because it’s fast and easy to use.

Now we’re looking at some ideas for more accurately representing the Sense-Think-Act flow of robotics programming, and we’re playing with boxes-and-arrows language ideas.  This isn’t new: LEGO Mindstorms has a boxes-and-arrows language that’s based on LabVIEW, but it’s based on the notion of a single, centralized brain (not the distributed model of Cubelets and MOSS), and, of course, there’s the common sentiment that LabVIEW just straight sucks and is something to be learned for a control systems class, complained about, and then promptly forgotten.

In my world, Max/MSP is the benchmark for boxes-and-arrows programming.  My friends at Sketching in Hardware use it to program physical devices and prototypes.  My electronic music friends use it for synthesis, sound design, and elaborate rule-based IDM, even embedding Max inside Ableton Live.  Max has been around since the mid-1980s!

Funny thing is, I had never played with Max.  I referenced it in my dissertation, I’ve seen a bunch of demos, and I get the concepts, but it’s $400 for a license.  Now, the people behind Max also built Pd, a free and open-source boxes-and-arrows language, and I tried to build a mini granular synthesizer in Pd a few years ago, but the documentation is sparse, the tools aren’t polished, and I ended up never completing the project.  Max, the polished commercial product, has a 30 day free trial, though, so I thought I’d give it a go.  Here are the results of my two hour experiment.

This little patch is a simplistic step sequencer, and it works!  The toggle switch on the upper left triggers both the BPM calculation on the right and the main metronome.  The metronome sends beats to a counter that breaks them up into groups of 8 (two measures of quarter notes, say) and the select object splits each of the 8 beats into a unique output.  Squiggly lines, that I like to think are virtual cables on my modular sequencer, connect the beats to MIDI codes for a hi-hat (55), bass drum (50) and snare (40), the makenote object turns those into a complete MIDI note with note-off, velocity, and duration, and the noteout object triggers the internal synth on my Macbook.  Beats!

I learned some things by playing.  Patches flow from top to bottom, and each input and output on a particular object does something different.  Lines can transmit various data types from the simple pulse called a bang to integers or audio signals.  Max, and the whole boxes-and arrows paradigm, seem to inherently enable and encourage thinking in terms of parallel processing and distributed nodes of computation.  Instead of a huge page of procedural C, we’ve got a visual map of the code structure in front of us and can dig in atomically to the modules.

This is a little scary.  Remember Flash?  I used to write a lot of Flash in the early 2000s and it was great for hacking little graphic web things together.  You’d embed code into the window, into objects, into special tweening scripts, anywhere you’d want it.  And that worked fine until it was time to debug, and 400 code snippets in 400 different places quickly became a nightmare.  I worry about the same thing happening in a giant Max patch, but at least the main flow chart provides a clear overview of the whole system.  Managing complexity rears its head again as a hard problem.

I thought I’d share this because I learned something about visual programming.  Because I finally played with a thing that I’ve been talking about for ten years.  Because I think it might be a great way to get kids thinking and playing with Finite State Machines!  And because my Max free trial only has 17 days left so I better play with my dumb sequencer while I can.