In the beginning of the week the class was divided into two groups. The first group being those who had programming experience, the second one those with less or no programming skills. The more experienced students would experiment with generating music by processing inputs that would be provided by the other group. They on the other hand were programming basic visual interfaces using p5. After the first half of the week people from both groups were teamed up to combine both sides into a sound setup with two connected laptops serving as input and output.

The input side of the setup is a combination of a text input field and four circles of different colour, size and position on the screen. The user can write anything into the text input field. The engine would continue to read out loud whatever has been written. Using the mouse, the user can cycle through the four circles and use drag and release gestures to change their position and size. Each circle colour represents a different instrument that will start playing when the circles are first activated. The red circle is special as it is used to distort the text-to-speech output. Modifying the circles' size and position results in a change of volume, pitch and other distortion filters of the corresponding instruments.

On the output side a Processing application receives all data over an OSC channel. The text is being processed by a speech synthesizer and the resulting sound is sent to Ableton Live together with the circle parameters. These parameters are used to tweak the Ableton channels which generate the final sound distortion. This tool provides an unconventional way to create unique speech and sound compositions.

Contribution

I was involved in the concepting, the design and the implementation of Klangmaschine. I wrote the complete engine that drives the communication between the web interface and the Ableton software and implemented the animations for the frontend.