In Finland, the homeland of Nokia, mobile phones are referred to as Kannys, which literally means “an extension of the hand”. I would like to make this extension of the hand very visceral by combining Nokia’s N900 with retro virtual-reality peripheral, the Nintendo Power Glove. The resulting hardware hack would enable hand gesture based interaction with the N900, controlled through the glove interface.

In order to answer a telephone call, the user makes the ‘telephone gesture’ and their hand becomes the handset, bringing a whole new meaning to “talk to the hand”!

Also as part of ISEA 2009 I’m running a workshop on ‘Hacking Toys Into Tangible Interfaces‘ at Dundalk IT. We’ll be building unique game control gizmos by adding usb connections to cheap electronic toys. The workshop is an introduction to physical computing, covering some basic interaction design theory along with practical hardware hacking techniques. There’s more on the workshop and everything else that’s happening as part of the conference at isea2009.org.

Then later on that evening was This Happened #6, where the audience learned all about the creative processes behind four unique tech art projects. These were Touched Echo by Markus Kison, Audience by rAndom International and Chris O’Shea, Troika‘s ‘Cloud‘ and the responsive installation at Covent Garden by United Visual Artists. One thing that was striking about all these was the ammount of time put into planning and pre-visualisation, leaving little or no room for error when the final product was delivered. Also they were made in very tight timeframes, Cloud for instance was nine months from initial idea to the final installation.

In the lobby of the BFI there was a project I really liked called Doodle Earth, which basically encourages people to draw. An animated map dotted with buildings and moving vehicles is projected onto a blank wall and people are encouraged to fill in the blanks with coloured markers. I drew this dragon head thing burning the building in the projection (someone added the ‘Cheese’ speech bubble and ‘JAFC rules…’ messages later on). All the art meets technology stuff is great but this really suceeds at creating a fun, collaborative user experience in a fairly lo-tech way. It’s good to draw!

First speaker was Gavin Burke from Galway based Future Audio Workshop who spoke about the physics of sound, how synth technology can reproduce these and his companies virtual synth product ‘Circle‘. Two years in development, Circle functions as a standalone instrument, a vst or a pro tools plugin and combines synth modeling with an interface that doesn’t try to reproduce the look of a hardware synths control panel but instead uses the full visual and control affordances the computer offers.

Next up Peter Kirn from CreateDigitalMusic examined the common metaphors that are used in music software, like the piano roll, the linear timeline and tape splicing and the possibilities for breaking out beyond these conventions. He cited current examples like the Ableton Live interface, the Tenori-on and many of the new touchscreen music apps for the iPhone like Brian Eno’s ‘Bloom’ and RJDJ. The possibilities of using Processing to make experimental music interfaces was also explored, one example shown was a circular sequencer that took inspiration from Balinese Gong circles.

We then viewed Niamh Ahern’s ‘Totally Wired‘, a documentary centering around Analog Synth store Schneiders Büro in Berlin, the musicians who buy their equipment there, the hardware manufacturers (many of which are based around Germany and are small family run businesses) and provides a great look into what motivates people to pursue Analog Synth hardware in this age of digital everything. Afterwards there was a Q+A with Niamh and Andreas Schneider. Andreas also presented an improv session on a patch cable controlled synth, and was so in the zone that afterwards he started talking to the audience in German :)

The final session was delivered by Mark Jenkins and Dave Vorhaus of White Noise and the BBC Radiophonic Workshop. Mark has wrote a book detailing the evolution of synth technology from analog through to virtual analog and took us on an audio journey through this development and his use of all forms of synths in his compositions.

Dave spoke of his experience recording the first ever electronic album ‘Electric Storm’ in 1968 while working at the BBC. The process was painstakingly laborious, the synth was tuned to play a note, this was recorded to tape. Then the oscillator was retuned, and the next note recorded… and so on. Interestingly he said this pioneering ‘cut and paste’ production method was inspired by the tape editing techniques of his father who was a blacklisted film director. Dave also said how he really doesn’t like keyboards because that their scale imposes limitations on how the instrument can be played, limitations he has sought to overcome with his double bass like Kaleidophon controller.

There were a couple of common themes through all these presentations. First was that all agreed (apart from one dissenting voice in the ‘Totally Wired’ documentary) that synth software along with regular computer hardware have finally reached the level of exactly modeling the sound qualities of analog hardware.

The other theme was the importance of the musician to instrument connection through the interface development, whether its through the cutting edge control and visual feedback of Circle or the visceral musician to hardware relationship that ensures the continued sales of analog gear at Schneiders Büro.

Either way the new language of electronic music control and notation is still under development and is going to keep evolving in new directions, not just due to commercial concerns but also due to the new found ease of access to software and hardware creation tools for the traditional end-user group, now able to undertake authorship of their own instruments and interfaces to create a new generation of music performance technologies.

Below is a work in progress shot from my ‘Alternative Displays’ project, it’s a 4 x 4 grid of square electromagnets. These are controlled by an Arduino and a Djuicer (a custom circuit designed especially for the workshop by 1scale1). A framed sheet of perspex goes on top of these and it contains a layer of industrial printer toner. When the magnets are switched on the the metal fibres stand up and you see a ‘pixel’ in the dust. Thanks to Mattias, Tony and David at 1scale1 for all their help making this. Next step is to have custom animations appear in the dust and have it respond to some external inputs.

‘Tiny Tag’ by team ‘Play It!’ is a graffiti simulator inspired by Tag Tool. ‘Play It!’ converted the steering wheel toy into their tag controller, allowing you to select a background, then pick your tags and move, rotate and scale them as you please.

Here’s a look inside the controller:

‘Music Maker’ by ‘Spanner in the works’ hacked this sound board toy into a controller for an ambient music generator.

A closeup of the keyboard hack, safely encased in an Ikea lunchbox!

Here’s a screenshot of Music Maker in action, when you hit a key on the board the corresponding shape lights up on screen and you hear a note, it plays the theme from ‘Close Encounters’ quite nicely!

Pixcube by the ‘Banjoes’ is an image browser, the ‘My First Keyboard’ was heavily modified to become the pixcube controller, with a new paintjob and several new buttons.

Here is the Pixcube itself, each side’s colour corresponds to a button on the controller.

When you press a button on the controller, the cube unfolds and you can navigate through the pictures, this is the portfolio side.

Magical Star Guitar by the ‘Three Amigos’ uses a converted Barney Guitar to control an game for young children that teaches counting and shape recognition. The battery case was carved away to make room for the keyboard module.

Finally, ‘Pass the Bomb’ by ‘Kawaii Productions’ combines this kids globe and a wireless keyboard to create an elimination game for up to six players.

Evil frogs have taken over the world and it’s only with the help of creatures from around the world that you can defeat them! After selecting the number of players the timer starts and you have to press the button corresponding to the continent shown onscreen, then pass it to the next person. If the timer goes to zero before you hit the button your eliminated!

More info on reacTABLE here and here.

Next up was a workshop by Brown Bag where they described their experiences using the animation toolset ‘Toon Boom Studio‘ to create a new children’s tv series ‘Wobbly Land‘. It seems to have a steep learning curve, particularly if you are used to Flash (no tweening!) but they had great technical support from Toon Boom’s creators, including 2 weeks of training. TB also created custom effects plugins for Brown Bag, that helped give the show it’s unique (and very non-generic vector animation) look.

The last workshop I attended was ‘Machinima FilmMaking’ (the art of making movies inside video game engines) by Friedrich Kirschner, the developer of Movie Sandbox, a heavy modification of the game Unreal Tournament 2004 that turns it into a Machinima application. The game world becomes your stage and the characters your actors, who you can direct as you please, either ‘live’ or through pre-defined scripts.

Some cool input methods were shown, one involved controlling your ‘actor’ like a virtual puppet using the strings on a usb golf controller. Another made use of the Xbox 360′s Guitar Hero controller so your machinima actor can jam on some chords.

Also demoed was an innovative supporting technology called Milkscanner, a low budget but highly effective method of scanning 3d objects using a webcam, lego, a bowl and some milk(!). A (small plastic) elephant was gradually submerged in milk, each stage captured by the webcam and then the software assembled each ‘slice’ into a 3d wireframe suitable for import into Movie Sandbox. Great stuff, innovative and nutricious!

Anomie
An experimental art piece showing the journey of an amnesiac from a healthy state of mind to a state of uncertainty. Through a route of non-linear interactivity, the user helps guide the character to the reason of his ill health and the incident that caused it.

M.A.M.M.Y.
M.A.M.M.Y. (mass audience multimedia-media yeah) is a social networking site for the creative students of DKIT, where they can communicate, collaborate and display their work. It uses bluetooth to beam recently uploaded content to students mobile phones.

R.E.A.C.H.
The R.E.A.C.H. (Routine Education for Autistic Children at Home) team produced an e-learning CD-ROM for autistic children to show them how to get ready for school in the morning. Their research found that autistic children are visual thinkers so it uses a cartoon based approach to effectively communicate it’s message.

Delirium
Delerium generates a visual representation of the current mood of the user using music and colour. The users temperature is recorded and inputted into the computer, at which point images and a piece of music appears and floats around the walls of the room. Questions appear onscreen that the user answers, after which these answers are compared and contrasted with the temperature reading.

The Land Of Luna
An exploratory game aimed at 8 to 9 year old girls. The game is from a first person perspective. The user plays the character of Lana who is transported to the world inside her snow globe, where she must collect a series of objects in different levels to save her friend Noah. There is no time limit on the game, so as to encourage the user to explore the world and interact with the characters.

Reuben Chinaski
An interactive exploratory autobiography for the digital age. We see the main character, Reuben Chinaski, on his deathbed. He is too weak to write his own biography so instead he tells the user about the ups and downs of his life. The user is free to explore parts of Reuben’s life in a non-linear fashion by selecting various objects in the room and letting Reuben explain what the object means to him through filmed narrated memories.

Seven Deadly Sins
Purge your soul of the seven deadly sins by engaging with this interactive experience. Memorize sound sequences inside the confession box while hit with some far out imagery.

Tactical Control Force
‘Watch tomorrow’s news today”, in TCF you control a live newsfeed by taking place in a series of games. The outcome of these tests shapes the narrative. A real adrenaline rush as you try to avert a national emergency in real time. The installation for this piece is a sort of survival bunker / command post, which helps intensify the atmosphere further.

Zeppelin’s Universe
This game’s target audience is kids in a public waiting room who are feeling restless. The main character, Mikey, goes into a dream state where he meets Zeppelin the Wizard. He has four tasks to accomplish to allow him to return to reality, or else he will be trapped in Zeppelin’s Universe forever.