blog

Possible Downtime

2010-01-06T16:50:00.000-08:00

msutherl.net may be down for a day or so as I transfer to a new host (from one Dreamhost account to another).

Responsive Gel

2009-12-12T20:52:00.000-08:00

RGel (Responsive Gel) is an apparatus capable of synthesizing new forms of quasi-physical "computational matter" in a neutral material substrate. Computational matter is different from ordinary matter in that it is defined only phenomenally – the only properties are affective properties. Consisting of an array of bend sensors and vibrotactile actuators embedded in pliable silicone gel, RGel responds to contortion with rich, realistic or hyperrealistic vibrations that simulate the tactile sensations of arbitrary materials.

RGel is conceived not as a device, but a prototype material. As evidenced by databases like Materia, institutions like Material ConneXion[1], and the work of designers such as Linnaea Tillett and Neri Oxman, an industry and a practice focused on generating and using 'new' materials with novel properties is blooming. As further testament, Erica Robles and Mikael Wiberg from New York University and Umeå University will be presenting a paper entitled "Texturing the 'Material Turn' in Interaction Design" at the TEI 10 conference this year. RGel envisions such a "material turn", a future where new materials exhibit more than novel material properties, but controllable phenomenal properties as well. Call it phenomenological material design. As electronic components grow more advanced, smaller, cheaper, and more efficient, the possibility of designing materials with computationally-enabled active or responsive properties approaches reality. RGel invites us to bracket the harsh unchangeable materiality of material itself[2] and think of matter as something programmable, to imagine new class of hybrid materials with responsive capabilities enabled by distributed sensors and actuators. These new active materials could be thought of and used as raw material with which to build responsive interfaces, tools, spaces, and buildings.

For the first prototype of RGel, we established the hypothesis that we could synthesize the sensation of breaking a brittle material such as glass or a cracker. Read more about our initial vision in the proposal. Find more technical details and follow our progress at the progress blog.

1. "Every idea has a material solution" reads their slogan
2. As a designer from the future might put it, 'by its current reductive definition'

Technical

The first RGel prototype consists of an 8"x10"x2" slab of silicone gel laced with electronics. It detects contortions with a grid of four fiber optic bend sensors, which are used to drive software sound synthesis. The sounds generated by the synthesizers are then used to control an array of vibrotactile actuators in the gel.

Fiber optic bend sensors were chosen over traditional bend sensors and paper sensors mainly for aesthetic and educational reasons. They were constructed using an IF-E96 LED (emitter) and IF-D91 photodiode (receiver) from Industrial Fiber Optics with 1mm plastic fiber optic cable purchased from Abra Electronics. To create a bend sensor, the jacket was stripped completely and the cladding scraped off on one side of the fiber optic cable to allow a small amount of light to escape. Once this is done, when the cable is bent in the direction away from the removed cladding, more light is allowed to escape, while when the cable is bent toward the removed cladding, more light is retained. This allows for bi-directional sensing, unlike traditional bend sensors. By reading the voltage difference generated by the photodiode in response to the light fluctuation, a measure of the deformation of the fiber optic cable can be acquired. The voltage difference was read by an Arduino, an ATmega-based microcontroller which streamed the data to a laptop over USB.

Max was used to process the sensor data. Because the amount of light leaked while bending away from the scraped cladding is greater than the amount of light retained while bending toward the scraped cladding, an algorithm for measuring and compensating for this effect was devised. In addition to typical smoothing and scaling operations, the signal was rectified to give identical values for bending "up" and "down". Future iterations will take advantage of the bi-directionality of the sensors. The cooked sensor data was then mapped to a function resembling f(x)=(log(x)+2)/2 using the Max "object" [function] to yield a sharp onset.

Synthesis was implemented in Max/MSP. The first presentation of the device used a signal file playback buffer and an instance of IRCAM's Modalys running a plate simulation. Sound from the file playback buffer was used to excite the plate and the output consisted of a mix of both signals. The physical modeling was removed for the second presentation. We plan to use multiple playback buffers and a physical model with an array of spatialized exciters in the future. By placing the exciters at different locations on the plate corresponding to the locations of the bend sensors in the gel, we can create an accurately spatialized multi-channel output signal. Audio from the single playback buffer was split into four channels and filtered appropriately for the destination actuators. Audio sent to the speakers was highpass filtered and DC offset was removed while audio sent to the inertial motors was lowpass filtered. A fifth channel intended for the DC vibration motor consisted of a comparator and a lowpass filter. We found that large pager-type vibration motors respond best to DC pulses.

To retain the DC and sub-sonic frequencies in the output signal, we bypassed the "DC blockers" (highpass filter capacitors) in a Presonus Firebox firewire audio interface. This was done easily by soldering the leads together on the PCB. The DC blockers are the 10 identical capacitors on the top PCB of the device, off to one side and grouped in fours.

Choosing vibrotactile actuators (vibration motors) was the most difficult portion of the project. We conducted in depth research into the available options and acquired as many as we could for testing. Unfortunately due to shipping delays, we were not able to test some actuators before making the mould and we had to make do with what we had. Among the pager motors that we ordered were 12mm and 20mm motors from Precision Microdrives and a generic 20mm motor purchased from All Electronics.

In the prototype, we used two hand-made "inertial" motors and one hand-made "pager" motor (DC motor with eccentric weight). The inertial motors were designed by Hsin-Yun Yao, Vincent Hayward, and Randy E. Ellis at the McGill Haptics Laboratory for the Microtactus project. The parts for the motors were generously given to us by Joseph Malloch from the Input Devices and Music Interaction Laboratory who had manufactured some for use in digital musical instruments. They allow for extremely accurate reproduction of audio signals from the subsonic (<20hz) domain to well into the sonic domain (<1khz). Unlike pager motors, they have excellent transient response which allows them to easily transduce complex signals. The larger pager motor was added to provide "kick" to the signal. By filtering the audio through a comparator, we were able to send short DC pulses that correlated to transients in the signal that provided extra impact. (Researching vibration motors, we found that some had a 'nominal vibration' rating measured in g's. We were not sure what to make of this scale–how much vibration was necessary to achieve what we wanted? After receiving a few rated motors after the prototype was completed, we discovered that even a motor rated at 4g was still insufficient to vibrate the gel in a visceral way (most pager motors are rated at less than 1.5g). Only the larger motor for which we milled a weight had the power we were looking for, but unfortunately we are not able to provide a measure of the vibrational amplitude. What would have worked best would have been large inertial motors. The inertial motors are exceptionally powerful for their size and were they about twice the size, they would combine the best of both worlds: amazingly fast response to changes in voltage and high (hopefully sufficient) vibrational amplitude for their size. Unfortunately we do not have access to the equations governing the proportions of these motors, so without help from the designers, we may not be able to produce larger ones. The options, then, would be to reduce the size of the silicone and aim for a subtler effect, use more inertial motors coupled with large pager motors, or give up on the inertial motors completely.)

In addition to the vibration motors, we used a pair of ordinary speakers (102-1549-ND) to reproduce some of the signal aurally. This was done to experiment with coupling between vibrotactile and aural phenomena. To drive the speakers and the vibration motors, we used a simple 1W, 5V amplifier designed by Mark T. Marshal using the TDA7052 IC. Due to the high current loads required, we used a MOSFET to control the large pager motor. A 5V voltage regulator was used to step down the voltage from a 12V power supply for the amplifiers.

The components were soldered together and set in a rectangular slab of silicone gel. To do this, we constructed a resize-able mould from plexiglass based on advice from the silicone dealer. After trying a number of different silicone products at Sial in Laval, QC, we settled on Smooth-On Dragon Skin 10, a high quality water-clear silicone with a shore hardness of 10A as our choice of silicone.

Originally we had planned to sew the electronics onto a piece of cloth or vinyl, but for the first prototype we experimented with laying the electronics directly into the silicone. This proved difficult as the components floated or, in the case of the bend sensors, insisted on retaining their bent shape. Nevertheless, the final product was functional.

Components

- 4x LED and photodiode mounted in barrel-mounts from Industrial Fiber Optics
- 1mm plastic fiber optic cable
- custom software written with Max/MSP/Jitter
- Presonus Firebox audio interface with DC blockers removed
- 2x custom fabricated "inertial" motors
- 1x generic ~30mm DC motor with custom milled eccentric weight (pager motor)
- 2x 38mm low-profile speakers (102-1549-ND)
- 4x TDA7052-based amplifier circuits
- 4x photodiode amplifier circuits
- 2 lbs. Smooth-On Dragon Skin 10 silicone gel
- plexiglass for display stand and adjustable mould

Conclusion

Though constructing the first RGel prototype, we learned an immense amount about vibrotactile actuators, bend sensing, silicone moulds, analog signal conditioning, and driving motors in atypical ways (i.e. with sound). In constructing the prototype, we made a number of mistakes that rendered the device only semi-functional, but the mistakes have been identified and a second prototype is being planned that will correct the mistakes. In the second prototype, we will: (1) use less silicone and/or stronger motors. The silicone turned out to be too much mass for the vibration motors to actuate sufficiently and we were not able to achieve the desired vibrotactile effects, (2) sew the components onto a piece of cloth before placing them into the silicone to ensure even distribution and the appropriate shape, (3) experiment more with physical synthesis models and a large number of actuators to achieve more convincing spatialization, (4) develop a better signal conditioning circuit for the bend sensors (we were measuring voltage fluctuations of only 10mV from the photodiode). We believe that a new prototype taking these lessons into consideration will achieve our goal of synthesizing the sensation of deforming a brittle object.

References

- Natural Interactive Walking, vibrotactile floor tiles, by Yon Visell and Jeremy Cooperstock at the Shared Reality Lab
- Semblance, art installation experimenting with full body 'just noticeable' vibrotactility by Chris Salter, Marije Baalman, Harry Smoak, etc. (in production)
- TapTap [PDF], wearable that records touches and replays them using vibration motors, by Jeff Lieberman, Oritz Zuckerman, Cati Vaucelle, Leonard Bonanni at the MIT Media Lab and Harvard
- Microtactus, a surgical instrument for amplifying texture, by Hsin-Yun Yao, Vincent Hayward and Randy E. Ellis

Acknowledgements

We would like to thank our mentors and teachers, Vincent Leclerc, Gary Scavone and Sha Xin Wei for immeasurable guidance. Elio Bidinost from the Sensor Lab contributed sage advice and patient help without which the project could not have been realized. We would also like to thank Martin Peach and Avrum Hollinger for advice on constructing fiber optic bend sensors as well as Joseph Malloch and Dr. Marcelo Wanderly for helping us build and contributing the inertial motors.

–Morgan Sutherland & Faiq Hussein

Responsive Gel Proposal

2009-11-10T03:27:00.000-08:00

This is a proposal for a project that I will be submitted for MUMT306, CART360, and CART214:

"We're always looking for new physics; new behavior that has never been seen before. Once we find it, of course, we start to daydream."
– Marc Kastner

I propose to create an apparatus capable of synthesizing new forms of quasi-physical "computational matter" in a neutral substrate. Computational matter is different from ordinary matter in that it is defined only phenomenologically – the only properties are affective properties. By bracketing materiality itself, we can begin to think of matter as something programmable. We can imagine new hybrid materials that defy physical laws. These new active materials can then be used as raw material with which to build responsive interfaces, tools, spaces, structures and art objects.

I am interested in creating a prototype 'material synthesizer' using sensate silicone gel embedded with coupled vibrotactile and sonic actuators. The gel, when contorted, will exhibit various vibrotactile and sonic behaviors associated with materials other than gel, be they real or imaginary. My hypothesis is that by exciting the gel strategically in response to interaction, I can synthesize the sensation of manipulating a new material. More specifically, I hypothesize that I can make silicone gel, a flexible, stretchy material, feel brittle.

The apparatus will consist of an array of sensors to sense contortions of the gel, a microcontroller and radio to send the sensor data to a computer, software to extract features from the sensor data and map them to parameters of a physical-modeling synthesizer, another microcontroller and radio to send audio data back to the object, and a custom 'cross-over' to share the audio between an array of vibrating motors and speakers. All circuitry will be sewn onto a layer of translucent fabric and cast into a silicone gel mold.

For sensing, I plan to experiment with fiber-optic bend sensors. I will arrange them in a grid in order to generate a two dimensional contortion map. This will be mapped to a physical model defined in IRCAM's Modalys system. The sound generated by the physical model will drive an array of small 'shaftless' vibration motors as well as small speakers. I may use special transducers designed for vibrating hard surfaces rather than traditional speakers.

Finally, I plan to present the project not as an interface or an art project, but as a demo for a new material about to enter production. I will create a fictional company and set up an engineering-demo-style booth to advertise the project in various high-traffic areas at Concordia and/or McGill University. The project presentation will be partly a performance. I will prepare a routine promoting the near-future (next year) mass adoption of active materials as materials for building and designing interface products.

I'm interested in enchanting everyday spaces. Picture a walk through the forrest. You come across a patch of moss and gingerly step across it, breathing in the fresh smell of amphibious plant matter. What if our cities could feel like this?

Imagine a section of a metro station with floor, bench, and walls made from "responsive gel". Artists could be contracted to script interaction scenarios for these surfaces. The behavior would not be invasive, distracting, or confusing, but calm, passive, and delightful, like the watery squishing of moss. For instance, the floor could be scripted to feel like thin layers of ice that shatter beneath your feet and the bench like a bank of snow as you fall into it.

See progress blog and project documentation (one post up). Alternative documentation can be found at my CART360 portfolio site.

Gestures and Responsive Media

2009-05-14T06:40:00.000-07:00

I will be working with Sha Xin Wei and Clarinda Mac Low on Friday to realize a somewhat impromptu participatory performance 'in' responsive media. I will be processing realtime audio from the room and re-projecting it through a 4-channel surround system. (Thanks to Navid Navab for granular + spatialization patches.)

MAY 14 - 16, 2009, 8.30PM @ ROULETTE
20 Greene Street, (between Canal & Grand), NEW YORK
$15 General Admission, Roulette members FREE
$10 Harvestworks members, students, seniors and under 30

“…lustrous sonic landscapes using [her] highly trained vocal cords”
-The San Francisco Chronicle

“…the naked freedom of jazz improvisation and the alien crunch of digital mechanisms”
Pitchfork Media

Produced in partnership with Roulette’s Mixology Festival, the 2009 Harvestworks Inside performance series presents “Gestures and Responsive Media”, three concert programs focusing on modern experiments in performance technology. Central to this series is the exciting way that live performers are exploring the use of gesture to control computers and building technological environments for free, improvisatory play. From Pamela Z’s gesture-based vocal experiments to Sha Xin Wei’s phenomenological experiments with movement and media, the presenting artists not only expand the boundaries of traditional performance and instrumentation, but also humanize electronics through creative means.

Thursday May 14 - Pamela Z / Elliott Sharp
Friday May 15 – Zach Layton / Sha Xin Wei
Saturday May 16 – Sawako / Bill Hsu & James Fei

About Harvestworks:
Harvestworks is a nonprofit Digital Media Arts Center that provides resources for artists to learn digital tools and exhibit experimental work created with digital technologies. [...]

About Roulette:
Roulette Intermedium is an Experimental and New Music presenting organization dedicated to the development of emerging and established artists. [...]

http://www.harvestworks.org/

via topologicalmedialab.net

Time-Sand: Responsive Microcosm

2009-04-20T20:30:00.001-07:00

Time-Sand is a 'responsive microcosm' with its own space-time dynamics. Visitors are invited to meddle: to manipulate the landscape and alter the flows of water, light, and even time. Interactions perturb the microcosm, sending waves that gush and bleed, revealing the past and slowing time in a manner like kicking up dust on the sea-floor or dripping ink on thick, porous paper.

The piece consists of an architectural model of a soil-remediation process (developed by Gregory Rubin for his master's thesis in architecture) augmented with responsive video and sound. A time-lapse video was created of cleansing and drying experiments over the period of a month and is projected down onto the model itself, giving the effect that the model is evolving on a time-scale faster than our own. Synchretic, synaesthetic, sound was composed to follow the ebb and flow of the time-lapse video and is looped in sync. Movement, captured from a camera above, causes local responses including simultaneous timbral and temporal shifting and blurring in both the video and sound. The model is periodically filled with water, which forms small streams and pools as it flows through sand and around clay foundations. Visitors manipulate not just the computational media, but the physical model itself. Like playing with mud puddles on a dirt road in the rain, they can add material (sand) and shape the flows. The virtual and the real are thus phenomenologically fused to form a hybrid computational-physical material.

Time-Sand: Responsive Microcosm from Morgan Sutherland on Vimeo.

Exhibitions

- There is No Soundtrack, CART356B final show, Concordia EV-6.720, April 9, 2009
- Computation Arts Graduate Certificate show, Concordia EV-6.720, April 30, 2009
- Exposed 2009: Social Body, Art Mur, May 12-23, 2009

Credits

Time-Sand is a collaboration between Morgan Sutherland (concept, responsive video), Gregory Beck Rubin (concept, model), and Aaron Munson (responsive sound). Time-Sand uses software developed by Jean-Sébastien Rousseau, Michael Fortin, and Yoichiro Serita at the Topological Media Lab.

Support came from the Topological Media Lab, FASA Concordia, and Hexagram Concordia. Thanks to Sha Xin Wei, Chris Salter, Jean-Sébastien Rousseau, and Harry Smoak for guidance.

Alternative documentation can be found at the Topological Media Lab calligraphic video research page.

Skylight

2009-04-20T19:33:00.000-07:00

Skylight was a responsive video installation at the Canadian Center for Architecture's All Nighter for Nuit Blanche 2009 in Montreal. A 14x10 rear-projection screen was hung horizontally at gallery-level above the entrance stairway. A camera mounted on the screen and facing toward the stairs and the entrance doorway captured the movements of visitors. The data from the camera was used to drive a realtime Navier-Stokes fluid simulation which was mixed with the original video feed and projected onto the screen for under-side and top-side viewing. As visitors entered and exited the museum, they looked up to see themselves reflected in a cloudy, viscous dream-world. By moving, they were able to shape and mold the video-matter surrounding them. Along with the video projection, LED lamps were positioned in the gallery facing downward to illuminate the stairs.

Credits

Skylight was a collaboration between Morgan Sutherland (responsive video, concept), Michael Fortin (Navier-Stokes fluid simulation), and the Topological Media Lab. Thank you to Saulo Madrid, Michael Kogan, and Sha Xin Wei whose help and guidance was invaluable. Support came from the Topological Media Lab, the CCA, and Hexagram Concordia.

Deleted Education Posts

2009-03-20T20:08:00.000-07:00

I took down three posts that I had previously written about my educational plans. If you wanted to see them, leave a comment and I'll send you a pdf.

As of now I have decided to stay at Concordia and pursue a BFA Specialization in Computation Arts. I will continue working as a research assistant with the Topological Media Lab. I plan to take electives in Music Technology at McGill and study mathematics independently.