Make a Reusable Sprite Component in TouchDesigner

Make a Reusable Sprite Component in TouchDesigner

One of the great strengths of TouchDesigner is its custom component system, where you can make powerful, reusable items. This video goes through creating a 2D sprite with controls for changing its image and modifying its alpha live. It will be a 3D object, so we can move it around in XYZ, rotate it, use [...]

Using the EMIT Quartz Composer Library

Using the EMIT Quartz Composer Library

A screen capture of building a quick visualization using the custom Quartz Composer library EMIT from Rybotron. Music: Four Tet – Love Cry Download EMIT here

Playtime for Selkie and Momo

Playtime for Selkie and Momo

Selkie is the Aerialist who performs with The Glitch Mob. After the LA Show, we got together to play with some realtime projection + dancing stuff. I had only my phone (Samsung Droid Charge) to record with, hence the low-res image. The first video is an openFrameworks visual I’m working on, controlled here with the [...]

New Movie Loader in Quartz Composer for Snow Leopard

New Movie Loader in Quartz Composer for Snow Leopard

Exploring the awesome updates to the Movie Loader patch in the new Quartz Composer for Snow Leopard

Structure in VJ Performance

Structure in VJ Performance

An exploration of Structure – 4 Layers for Visual Performance.

Cut Up Video using Lucifer and Ableton Live

Cut Up Video using Lucifer and Ableton Live

Make realtime AV Cutups the easy way – with Lucifer and Ableton Live

Quartz Composer: 3D Cubes and Audio Input

Quartz Composer: 3D Cubes and Audio Input

A Beginner’s Tutorial on Lighting, Cubes and Audio Input to make an Interactive Toy in Quartz Composer.

AV Challenge 2: Part One, Making the Clips

AV Challenge 2: Part One, Making the Clips

Momo the Monster’s Method for AV Madness

Loop-R VJ Touchscreen System

Loop-R VJ Touchscreen System

Homemade Performance system from visual_basiq

Playsonic: AV Fun with a Game Controller

Playsonic: AV Fun with a Game Controller

From the 2007 Maker Faire – a project where Video Art meets Gaming and DJing

Retinal Rumble

Retinal Rumble

Show Me Yours and I’ll Show You Mine – live Internet Video Battle at the 2007 Maker Faire.

AV Sequencing with Live + VDMX + Monome

AV Sequencing with Live + VDMX + Monome

A solution for hands-on live audiovisual performance

VJ Kung Fu Promo

VJ Kung Fu Promo

See the new VJKUNGFU.TV Promo!

Crossfader Triggering with the NuVJ

Crossfader Triggering with the NuVJ

Using the NuVJ for AudioVisual performance – a remix and demo.

NuVJ Crash Course

NuVJ Crash Course

The basics on the NuVJ – what the buttons do, what it’s like to mix with.

A VJ’s Pet – Lemur

A VJ’s Pet – Lemur

Introduction to the Lemur Multi-Touch Interface device. How to connect it up, some things you can do with it.

Build a Projector Mount that will Go Anywhere

Build a Projector Mount that will Go Anywhere

You can’t count on venues to have a proper place to put your projector. Build one of these and you can go anywhere.

Staring Into The Snow Preview

Staring Into The Snow Preview

Preview of an ambient abstract piece about losing yourself in the holiday season.

Maker Faire VJ Clips

Maker Faire VJ Clips

Free VJ-Ready Clips from the Maker Faire 2006

Play a Movie in Quartz Composer

Play a Movie in Quartz Composer

QC can look daunting at first – here’s how to get started.

How To Install Quartz Composer

How To Install Quartz Composer

Free video processing app from Apple – you may already have it!

Visit the VJ Kung Fu Channel on Vimeo to download the original, high-quality Quicktime.

Playsonic: AV Fun with a Game Controller

mfaire_gampead_small.png

The Playstation 2 Game Controller is a wonderful interface. I debuted this installation at the 2007 Maker Faire, and found that kids and adults alike (but especially kids and 20-something gamers) were able to understand and control this interactive Audio Visual System very quickly.

This system builds upon the work I started here: A/V Synth Controlled by Game Pad, and the graphics were inspired by the Interface Design of Battlestar Galactica. Watch the video above and come back for Nitty-Gritty technical details.

First up is Junxion – software that converts USB to MIDI messages. First, a disclaimer – I bought this back when it was $29, and I think it’s a bit overpriced now (€25 for the basic version and €75 for the deluxe). You can try arj.MultiControl – the last version I used was buggy, but they may have fixed things up since then. In any case – here’s what my Junxion setup looks like:
junxion_gamepad.jpg

I have most of the buttons sending CC messages, with the exception of the four circular buttons on the right side of the controller (this would be Triangle, Circle, X, Square on a PS2), which send notes, since they drive the synthesizer. This midi gets sent to my Ableton Live setup:
bsg_live.jpg Track 1 are my two drum clips – I trigger the Play Clip with L1 and R1 and the Track Stop with either L2 or R2. There’s a ‘Stereo Beat Cutter’ Effect – I control the ‘Chance’ parameter of this with my Directional Pad, defaulting to ’0′ when I’m not pressing anything, and jumping to four possible ranges when I press Up, Down, Left or Right.

live-synth.jpg

Track 2 is my Synth. I pitch the notes down 2 octaves and run them through an Arpeggiator to the Joystick preset. The Analog Sticks on the gamepad control ’2nd Pitch’ and ‘Filter’ in the Joystick Synth.

Track 3 is my MIDI-OUT track. Here I take the Arpeggiated MIDI generated by Track 2, and send it out to my IAC Port (the built-in MIDI routing device in the Mac OS). This brings us to:

qtz_bsg_cu.jpg

midi_keys_12.jpgThe Quartz Composer patch to run the visuals. Basically, we have a lot of sprites controlled by LFOs, and the controller (for which you can see my handy Macro Patch on the left here) modifies the LFOs, or changes the Alpha of the sprites to make them appear and disappear. One of the tricks I use is this patch to the right, called ‘MIDI Keys 12′. This was a simple enough idea, pointed out to me by Electronic Music Producer Chiral when I was brainstorming for ways to improve a synthesizer-based patch. Inside is a giant mess, of which you can see a tidbit below.
qc_octave_c.jpg

Basically, I take every possible octave for each note, and run it through a Javascript patch which averages the data from the 10 octaves coming in, and outputs one signal. The end result is a signal that responds to C# no matter which octave you’re playing.

Let me know what you think of this project so far – this is only the first version, and I’ve much more work to do. Ideas I have for the future:

  • Left Analog stick should control the tunnel flight, rather than the Directional Pad
  • Something should happen when you press Start or Select.
  • Third Drum loop? (I could use L2 for a loop, and have R2 be the only ‘Stop’ button)
  • The analog sticks can be pressed in for an additional two button clicks – I should use these
  • Add multiple controllers – one for drums, one for lead, one for Vox, etc. that all control the same graphic system.
  • Your Idea Here.

Downloads

Disclaimer: This is not ‘ready for the public’ at all yet. I’m providing the source code for those who tinker with MIDI routing and interactive Audio/Video. One of my goals is to build a system that will be easy for everyone to download and use, but we’re nowhere near that point.

Ableton Live Project: playsonic_ableton.zip
Junxion Preset: logitech_gamepad.jxn
Quartz Composer Project: playsonicv1.qtz
All 3 of the above in 1 file: playsonicv1.zip

YouTube version: Playsonic:  AV Fun with a Game Controller

2 Responses to “Playsonic: AV Fun with a Game Controller”

  1. dr.xnlb Says:

    too fun! i may have to try this out… think it will run on a mac?

  2. links for 2007-11-26 « Jay’s blog Says:

    [...] VJ Kung Fu » Playsonic: AV Fun with a Game Controller [...]

Leave a Reply

You must be logged in to post a comment.