The Surveillance Orchestra

The robotic accordion / [Deep|Remote] Listening Device might actually be the first instrument in a strange electro-mechanical orchestra that tries to find music in everyday occurrences and processes that people do not normally notice. The following is a description of the orchestra. It informs the process of how an architectural proposal forms around my project.


The Surveillance Orchestra is a group of robotic musical instruments that finds and explores the music hidden in the banal, the mundane and the ordinary. The Orchestra uses surveillance devices to observe specific phenomena such as the movement of trains on a track, the traffic on a street, or the growth of a burdock plant. Each instrument of the orchestra is unique and responds to its own remote surveillance device or set of devices that observe its phenomenon of interest. The surveillance devices convert their observations into audible tones that can be periodically sent over radio to the orchestra instruments.


The owners of the instruments of the Surveillance Orchestra are a secretive and diverse group of hobbyists and amateur musicians. In Winnipeg, they have chosen a piece of city property in Point Douglas as the site of their observations. This is because of the variety of phenomena available to observe and to exploit the nature of Point Douglas as a forgotten neighborhood near the centre of the city. Someone skulking around and placing an object on a fence does not look quite so out of place in Point Douglas as they might in Tuxedo or Charleswood.

The owners of the instruments keep them tucked away at their homes where they continually revise and update the mechanisms, electronics and software of their devices. They constantly search for the best way to translate the surveillance data into a music appropriate for the surveilled phenomenon. Each member of the orchestra transmits the sounds produced by their instruments over a single (pirate) AM radio frequency (which changes periodically to help avoid detection by authorities).

Winnipeg’s Surveillance Orchestra started in 2007 with three members. The orchestra grew to twelve members by 2017 when their presence in Point Douglas was discovered by a CP Rail employee performing maintenance on one of the tracks. The employee did not usually work in the area, and questioned why there was such a large structure carrying cables over the tracks.


Each member of the orchestra constructs and adapts their own infrastructure in Point Douglas to accommodate their surveillance devices. Their constructions are primarily made of pieces of existing infrastructure that had been forgotten, abandoned or at least looked like they wouldn’t be missed if they were “borrowed.” The constructions start small and are built up slowly and covertly enough that locals scarcely notice the changes. Most parts are carried in by hand and no more than four visible pieces are installed during a week. Power for the devices is provided by solar panels and batteries, strategically placed to avoid theft and suspicion. The surveillance devices communicate to the orchestra instruments over Citizen’s Band (CB) radio, a mode of communication largely abandoned in favour of the cell phone.

Hardware Update

Once I got the robotic accordion up and running, I discovered a couple of issues. The bellows were more leaky than I realized. I took out the bellows and added a plastic liner. This dramatically improved the volume of sound produced by the instrument.


After a period of extended use, the relay that controls the direction of the motor failed. I suspect this occurred because the abrupt motor direction changes that I was subjecting the relay to. Any motor produces a back current while it is turning (acting like a generator). I believe that while the relay was reversing the polarity of the motor back and forth, substantial electric arcs were probably occurring inside the relay (as the motor keeps turning for a fraction of a second while the current reverses). This probably damaged the relay contacts over time. Also, the relay was handling close to its maximum rated current of 5 amps (the motor probably draws more than 5 amps when it first starts turning).

I was lucky enough to find some 10 amp relays that have the same pinout as the 5 amp relays, so I replaced all the relays on the board. In addition, I modified the Arduino code to shut the power off to the motor for a fraction of a second while the motor direction is changed. Hopefully these changes will extend the life of the control circuit.

Following is the current Arduino code that is running on the instrument. It includes changes to make the motor direction control and limit sensing more robust.

[Deep|Remote] Listening Device Firmware 1.12

From Breadboard to Printed Circuit Board

I went through a self-directed crash course in printed circuit design in order to create a printed circuit board to hold my control circuit for the robotic accordion. A printed circuit board provides more stable and reliable connections than those on a breadboard. I used CadSoft's Eagle software to layout my circuit schematic and convert it to a two-layer board design.



There are many companies online that specialize in building circuit boards. I uploaded my design files to Advanced Circuits in Colorado who built the board, which turned out quite well.



I transferred the parts from my breadboard circuit to the new printed circuit board, soldering them in place.



I built an oak frame to hold the circuit board, the Arduino microcontroller and the power supply, using pin connections to hold the frame in place.


Here is the circuit board in place in the robotic accordion device.

Accordion Automation

This post shows some of the steps I went through to automate the treble section of the accordion.

I came across this website that describes the use of car windshield wiper motors to animate large Halloween projects:

http://www.scary-terry.com/wipmtr/wipmtr.htm

The windshield wiper motor provides more than enough torque to raise the bellows. In fact, I removed some of the pulleys I had previously installed on the device. With fewer pulleys, less rope has to travel to move the bellows. This makes them much more responsive to the motor movement.



When the bellows are at their greatest extension, the wiper motor must be reversed to allow gravity to squeeze out the air contained within. I used a couple of reed switches to sense when the bellows are fully extended and send a signal to the Arduino board that controls the accordion. One reed switch is attached to each side of the device for redundancy.


Small rare earth magnets were attached to the extension structure of the bellows. The reed switch contacts close whenever the magnets pass by, sending a signal to the Arduino controller.


Solenoids are used to pull the accordion pallets to select which notes are played. Bicycle brake cable is used to transfer the pulling motion of the solenoids to the accordion pallet arms. I experimented with a couple different strategies of connecting the brake cable to the pallet arms. I developed some laser-cut clips that wrap around the pallet arm and accept the ball-end of the brake cable. I designed break-away arms on the clips to aid with their installation in the tight spaces between pallet arms.


The clips worked with light tension on the brake cable, but they failed with a heavy tug. The acrylic did not have enough strength in the tiny dimensions that I had to use in the clips and did not fit inbetween some of the pallet arms. I designed another piece that uses the cable itself to wrap around the pallet arm, saving quite a bit of space.


I built a couple of oak brackets to fit over the front of the accordion to align the brake cables and hold the brake cable housing.


The placement of the brake cables translates the order of the accordion pallets to the order of the solenoids.



I adapted a computer power supply to power the windshield wiper motor and the relays that control them. I clipped and insulated the wires that weren't needed, leaving only two yellow 12V wires and two black ground wires exposed from the power supply. The green power wire was permanently connected to a ground wire to keep the power supply on.

It's Alive!

The [Deep|Remote] Listening Device is now operational. I've spent quite a bit of time detailing the connections from the accordion pallets (the covers that open to allow air to pass through the reeds) to the solenoids. I used bicycle brake cable and housing to pull each rod that operates the accordion pallets.



The pallets close using the force of the springs located under the accordion piano keys. Some additional springs were added for a few of the solenoids to make up for weak and broken springs under the piano keys.

I programmed the device to have a start-up sequence to test the solenoids and bellows. The sequence cycles each solenoid in rapid succession, moves the bellows to their lowest point, and then plays a couple of chords. I posted a few videos on YouTube:

Save $$$ by Learning to Repair Your Own Accordion!

While working on controlling the pallets arms of the accordion, I broke off a couple of the pallets. The pallets hold on to the pallet arms with a special mixture of beeswax, resin and mineral oil. It is common for the wax to dry out and crack in older accordions. To repair the pallets, I reactivated the old wax mixture by adding some fresh beeswax to it and melting it with a heat gun.


As the mixture cooled, I applied it to the pallet and then secured it to the pallet arm.


The repaired pallet is shown in the bottom centre of the following photo:

Remote Observation

I've been investigating how to connect my Deep Listening Device to our given site of Point Douglas (6.7 miles or 10.7 kilometres from the University). I thought about different ways of communicating between sites including telephone and the Internet, but radio seemed to be the most accessible and feasible for this project. I looked into Citizen's Band (CB) radio since I knew it had a transmission range close to what I need. I borrowed a couple of old CB radios to start playing around with. I set up one radio in studio and could not hear any activity on any of the 40 CB channels. I suspect CB radio is still in use, but that it has been largely abandoned in favour of cell phones.

I'm working towards the creation of a device to be located in Point Douglas to watch the passing trains and transmit data about their motion to the [Deep|Remote] Listening Device which would translate the data into a strange music. I started prototyping a device using an Arduino board to read a photocell and transmit a signal over CB radio whenever there is an abrupt change in light that would indicate motion past the photocell. Most CB radios have a connector for an external microphone that contains all the basic control lines for the radio. I modified a cable taken from an old AT keyboard that happens to match the 5-pin DIN connector on the CB radio.


To transmit with the CB radio, the radio's TX line must be connected to the ground line, causing any signal applied to audio line to be transmitted. (This is the basic operation when using the CB radio's external microphone; pressing the button on the microphone causes any sound picked up by the microphone to be transmitted over the radio). I attached a 2N3904 transistor to one digital pin on the Arduino board to control the TX line. Tones are sent out from another digital pin on the Arduino.



This picture shows the Arduino connected to the portable CB radio. In the middle is the photocell (shielded by the blue straw) connected through the coil of green and black wire. This is the code I developed for the Arduino to send a sequence of two tones over the CB radio whenever there is a substantial change of light detected by the photocell:

Motion_Bleep.txt

You can hear the tones produced over the CB radio by clicking the following link:

Two_tone.mp3

Deep (Remote) Observation

Ted Krueger from the Rensselaer School of Architecture (RPI) visited our department to give a couple of lectures. He stopped by our studio to talk about our projects from last term and where they are headed this term. We talked about the Deep Listening Device and this site that I have been looking at in Point Douglas:



The site is located on city property next to the CPR rail lines and the rail bridge that crosses Higgins. There is a chainlink fence that prevents access to the rail lines. The rail line is elevated above the natural grade of the area, and there is a small hill that provides a decent view of the trains next to the fence. There are a few utility poles on the site. One pole has a large metal box containing a circuit breaker and an electricity meter housing (the meter itself is missing).



Our current studio project is to adapt our devices and/or technologies from last term so they can exist in Point Douglas. I have been thinking about creating a number of devices that would listen and watch for trains and perhaps create a music from that. The devices would be integrated into the existing "fabric" of Point Douglas, hidden in plain sight. A device could live on the chainlink fence, looking for changes in light to determine the rhythm of a passing train. Some other device could listen for changes in sound level and signal the approach of a train.



Ted suggested the possibility of transmitting data about the trains back to our studio, where my in-progress "Deep Listening" Device would translate the presence, motion and rhythm of the trains into a music (kind of a remote listening or observing). Ted wondered about how the trains communicate, which I had not even considered.

Raymond Scott's Electronium

I came across Raymond Scott's "Electronium" while browsing the Internet today. From http://emfinstitute.emf.org/exhibits/electronium.html:

"Developed by Raymond Scott in 1959, the Electronium was a large-scale composing machine. As Scott described it: "A composer 'asks' the Electronium to 'suggest' an idea, theme, or motive. To repeat it, but in a higher key, he pushes the appropriate button. Whatever the composer needs: faster, slower, a new rhythm design, a hold, a pause, a second theme, variation, an extension, elongation, diminution, counterpoint, a change of phrasing, an ornament, ad infinitum ...""

http://raymondscott.com/Electron.html
http://raymondscott.com/em.html


Scott harvested parts from the Electronium for use in other music-generating devices, so it is no longer operational. There is a YouTube video showing the device as it is today:

Pure Data + Arduino

I worked on some software for the Arduino microcontroller to control the solenoids of the Deep Listening Device. The code accepts "note" inputs from the Pure Data patch that will do the "deep listening." The Arduino code converts the note input to the proper signals to operate the solenoid attached to the desired accordion valve.

The code sends a 48-bit sequence out to the shift register circuit I built previously. Each bit controls a single solenoid, turning it on or off.

I used code snippets for the shift register from http://www.arduino.cc/en/Tutorial/ShiftOut. The communication to Pure Data uses the Simple Message System library from http://tof.danslchamp.org/SimpleMessageSystem.

Here is the code:
Deep Listening Device Arduino Code

Here is a video of the startup sequence of the circuit. LED's are attached to each output for troubleshooting and show which solenoids are being activated.

My Head is Spinning

I built this Head BEAMbot a while back. It has two photocells (shaded with the blue straws) that sense light. The robot is mounted on top of a motor, and spins itself around to find a position where it can get equal amounts of light hitting each photocell. Devices such as this would be used as the "evil sensors," detecting light and motion for the "deep listening" device.

Here is a short clip of it in action:

The Bacterial Orchestra

I discovered the Bacterial Orchestra today, a project put together by an architect, an electronic artist and a musician in Stockholm, Sweden. From their website:

"Bacterial Orchestra is a self-organizing evolutionary musical organism. It consists of several audio cells. Every cell listens to its surroundings and picks up sounds trying to play them back in sync with what it hears. It can be the background noise, people talking or sound played by other cells."

It looks like every audio cell contains a microphone, a set of speakers, and a controller of some sort that listens to the microphone and plays back what it hears with a pre-programmed behaviour. This is quite like an all-electronic audio-only version of my "deep listening" device.

http://www.we-make-money-not-art.com/archives/009294.php

Under the Bridge

I went back to Point Douglas yesterday to explore under the Louise bridge, which connects Point Douglas to Elmwood. Cars driving northbound over the bridge produce some nice thumps under the bridge. The slope next to the bridge seems to be used as a toboggan run for the neighborhood kids. The area is probably more heavily inhabited during the spring, summer and fall. Several drain pipes hang underneath the bridge and beneath each is a pile of sand and gravel washed down from the bridge surface.

Considering Point Douglas

Our next studio project is the creation of an autopoietic device (based on our previous work) to be located in the Point Douglas neighborhood of Winnipeg. Point Douglas is a somewhat down-trodden (but interesting) section of Winnipeg located in a bend of the Red River just north-east of Downtown. Point Douglas is bisected by the national Canadian Pacific Railway line and bypassed by the Disraeli Freeway. The neighborhood contains a mix of residential, commercial and industrial properties.

I explored the area a couple of times looking for possible places to locate my future device(s). There were a few areas that caught my interest.

I spent some time under the Disraeli freeway, listening to the traffic zoom and thump overhead.

I also spent quite a bit of time next to the train tracks near the rail bridge over Higgins. There is a stand of trees right next to Higgins with a tiny clearing inside.



This site slopes up rapidly to meet the elevated rail line and has a view of Higgins, the nearby industrial buildings, the Louise bridge, the River Parkway and out across the Red River.


I was lucky enough to experience two trains rolling through while I was taking pictures there. I was engaged by the shifting slivers of sunlight cast between the rail cars, sweeping across the snow as the trains rumble past. Photocells could be used to observe passing trains by sensing these fleeting curtains of light. I was also interested in the varied rhythms of produced by the two trains rolling past one another at different speeds.



I'm not sure exactly what my device(s) will do yet. I've been thinking about continuing with the "deep listening device" theme by making a collection of bellows of different shapes, sizes and materials. Each bellows would contain a different reed, so that each would sound with a different note when activated. This collective might watch or listen to passing trains and cars. It might sporadically and spontaneously "remember" a passing train and attempt to play it back through the motion of the bellows and sounds of the reeds, with certain bellows pumping away to describe the rhythm of the train, and others reacting to the train's length or the direction it was traveling. I have serious doubts about the feasibility of building such an installation: Could it survive in the snow and cold? Would parts of it be stolen? Would it be quickly destroyed by mischievous neighborhood kids?

Electrifying Work

I've been working on the electronics of the Deep Listening Device sporadically. I've installed all 41 of the solenoids on the wood rails, so they are ready to be wired up and set in the device to control the valves of the accordion.



I've been working on the circuitry to operate the solenoids as well. The circuit is essentially a 48-bit shift register (but I am only using 41 of the bits). It accepts a serial signal and converts it to a parallel signal that goes out to the Darlington transistor array that operates the solenoids. The shift register has a latch on the output that allows me to control when the signal gets output to the solenoids.

This is how it works: A string of bits is sent into the circuit, instructing it about which solenoids to activate. For example, to activate the 2nd and 24th solenoids, the string of 48-bits (0's and 1's) would be sent in with a '1' located in the 2nd and 24th position, and '0's' everywhere else. Once all the bits are sent into the device, the latch is activated to output the signals, turning on only the 2nd and 24th solenoids.



I am using the Arduino microcontroller as a bridge between the Pure Data software and the shift register circuit. I've modified and added to the Arduino code from this page to accept up to six notes and convert those notes into a string of bits to send to the shift register to turn on the appropriate solenoids. The Arduino USB board I am using powers itself through the computer's USB connection and is able to provide enough juice to power the shift register as well.

Over the winter break I upgraded the RAM on the $8 computer I had bought from Value Village. I've installed Ubuntu Linux, Pure Data and Arduino software on it (all open source and free). The computer will be integrated into the device to run the Pure Data code that will listen to the room and determine which notes the device should play.

I bought a car window motor and a windshield wiper motor from Princess Auto. Both motors provide quite a bit of tourque, so I'm hoping one of them will be able to power the bellows.