Instrument
Draw a line that goes from left to right in the space below. An AI will play a synthesizer following the shape of the line you drew. Try straight lines, diagonals, and lines that look like waves. Try short broken lines and continuous lines.
Could we consider this a musical instrument? Musical instruments transform expressive gestures from the performer into musical sounds. In traditional instruments the gesture shapes the sound, but also needs to provide the “acoustic energy”: plucking a string, blowing a flute, hitting a drum. Over the last century electronic instruments changed this (because the acoustic energy comes from electricity) and artists were able to dream up a multitude of ways of producing and shaping sound.
Many of the pioneers of electronic music, who composed the earliest pieces and also created the instruments that produced them, were women. They combined artistic sensibility with knowledge of science and technology.
Up ahead, we’ll:
Female pioneers of electronic music
The theremin, patented in 1928, is one of the earliest electronic instruments that’s still used. It’s really groundbreaking in terms of the gesture used to control its sound: you play it by moving your hands in the air. One hand controls the pitch, the other the volume, of a continuous tone created by oscillators.
Try playing a virtual theremin in the panel below:
Audiences showed some resistance accepting the theremin as a valid artistic medium. One particular artist convinced many people.
Clara Rockmore (1911-1998) was a classically trained Lithuanian theremin virtuoso. As a child she was a violin prodigy, but had to give up the instrument in her teens due to tendinitis. She then met Leon Theremin and discovered his new invention. Thanks to her training, control, and perfect pitch (the innate ability to identify notes without external references), she was able to develop a unique playing technique and collaborate with Leon in improving the instrument. As early as 1938, she was playing large public concerts and performed as a soloist with the New York Philharmonic, the Philadelphia Orchestra, and the Toronto Symphony.
Tape recorders revolutionized music, starting around the 1940s. Any existing sound (musical instruments, nature, or household items) recorded on tape can be easily slowed down (which lowers its pitch), sped up (which raises its pitch), reversed, or passed through filters creating a new sound that did not exist before. Tape recordings can also be looped, layered, or cut and rearranged.
This began the expressive manipulation of sound itself, disconnected from its original source.
Delia Derbyshire (1937-2011), a British mathematician and composer, is famous for creating the electronic arrangement of the Doctor Who theme in 1963. This was one of the first entirely electronic music pieces used on television. She worked for over a decade in the BBC Radiophonic Workshop, a pioneering sound effects unit that produced incidental sounds and music for radio and television. One of her best-regarded works from that period was Inventions for Radio, a series of acoustic collages with people talking about dreams, God, life after death, and the experience of old age, over an electronic soundscape. As a freelancer, she collaborated with major figures of the British art scene, like Yoko Ono or the Royal Shakespeare Company, in film, theater, and early electronic music events.
Delia Derbyshire’s early tools were sound-producing electronic devices like oscillators, wobbulators, white noise generators, and tape recorders. These were not designed for musical performance, so they were operated with knobs and switches, and creating a musical piece required a lot of lengthy careful planning. However, they created synthesized sounds that, through the radio, brought new worlds into existence.
Daphne Oram (1925-2003) was one of the founders of the BBC’s Radiophonic Workshop in 1958 and one of the earliest British composers to produce electronic sounds and work from field recordings. In 1959, she created her own electronic music studio and developed Oramic Synthesis, a system that controlled electronic sound production using several strips of clear 35mm film where the artist would paint expressive lines with a brush. These strips would be fed into the machine, and a photo-electric sensor would read the lines. Each strip controlled a different parameter in the synthesis (pitch, loudness, envelope, etc.).
Oramic synthesis was an early form of “sequencer,” which allowed to separate the moment the expressive gesture is made from the moment when the sound is produced. Daphne Oram understood the freedoms that came from being able to produce music independently from performing an instrument:
“Once the composer can write without the limitations of performance his palette is extended enormously. Rhythms become anything the composer can visualise without them having to be playable. Timbres have no registration and theoretically any sound, musical or otherwise, is within his grasp.”
New instruments were developed around these freedoms brought by electronic music. An explosive revolution started as they left the labs and reached the hands of musicians. Some of them defined entirely new genres. One notable example is the Moog, the first commercial synthesizer. Wendy Carlos helped in its development while also being one of its most celebrated performers.
Wendy Carlos (1939-present) is an American musician and composer. She studied physics and music at the university and had an early interest in electronic music. She collaborated on developing the Moog synthesizer, suggesting a touch-sensitive device for greater musical dynamics, among other improvements. She became famous with Switched-On Bach, an album of Bach music performed on the Moog that achieved commercial and critical success, plus three Grammy awards. The Moog could only play one note at a time, so multiple tracks had to be recorded separately and combined to create the pieces. She composed the scores for Stanley Kubrick’s A Clockwork Orange (1971) and The Shining (1980) and Disney’s classic sci-fi movie Tron (1982). As an openly trans-woman, she helped raise public awareness of transgender issues.
These days, a new revolution in electronic music is being wrought by artificial intelligence. This technology can give artists even greater freedom:
- It can enable new expressive gestures;
- it offers new ways to explore timbres, genres, and forms;
- it allows us to shape sound with the versatility that imagination can provide instead of the capabilities of the physical body.
In other chapters, we explore some of these possibilities in more detail.
How does our AI work?
This AI was built using two neural networks working together: an encoder and a decoder. The encoder converts a sequence of notes played on the piano to a simple “shape” (a curve that moves up or down in the same way as the pitch of the notes played), and the decoder converts this shape back into piano notes.
The networks were trained using hundreds of piano pieces in a variety of styles, ranging from classical to contemporary. The training (done automatically) involved converting a melody to a shape and this shape back to a melody until the input and output were as similar as possible.
The interactive above uses only the trained decoder. It uses the shape you draw as an input to generate a sequence of piano notes similar to the ones it was trained on. The decoder can decode any shape, even if it was drawn directly, and not produced by the encoder. Thanks to its training it learned to reproduce the patterns and structures commonly found in piano music to produce new, original melodies.
Food for thought
The same technological advancements that changed the way that music was produced changed the way it was consumed: headphones, 3D sound, CDs, portable music players, music videos, streaming services, karaoke, rhythm games, etc.
In what ways can AI empower listeners and enrich how we experience music?
External links
- Sisters with Transistors is an excellent documentary that tells the story of the female pioneers of electronic music.
- Play with Piano Genie, the original version of the AI in this page.
Credits
Illustrations by Constanza Rojas-Molina. Licensed under a Creative Commons Attribution 4.0 International License. Copyright 2023 Constanza Rojas-Molina.
Based on Piano Genie, created by Chris Donahue, Ian Simon, Sander Dieleman and part of the Google Magenta project.
Copyright and license information in our GitHub repository.
References
- “Sisters With Transistors.” Written and directed by Lisa Rovner. Anna Lena Films. 2020.
- Wikipedia contributors. “Clara Rockmore.” In Wikipedia, The Free Encyclopedia. Wikipedia, The Free Encyclopedia. 2022-10-20. Retrieved 2022-12-10.
- Warshaw, Dalit (March 2011). “Clara Rockmore: A Legendary Performer of an Enigmatic Instrument.” The Juilliard Journal. The Juilliard School. 2011-03. Archived on 2016-04-02. Retrieved on 2022-12-10.
- Dr. David Butler. “Delia Derbyshire.” In “100 Years of our BBC.” BBC website. c. 2018-12. Retrieved 2022-12-10.
- Steve Marshall. “The Story Of The BBC Radiophonic Workshop.” Sound on Sound. 2008-04. Retrieved 2022-12-01.
- Wikipedia contributors. “Wendy Carlos.“ Wikipedia, The Free Encyclopedia. 2022-12-11. Retrieved 2022-12-17.
- Wikipedia contributors. “Theremin.” Wikipedia, The Free Encyclopedia. 2022-12-14. Retrieved 2022-12-19.
Text is available under the Creative Commons Attribution License . Copyright © 2022 IMAGINARY gGmbH.
