Melody

Place some notes on steps 1 through 4 of the sequencer below. Then press the “Generate” button, and the AI will complete your melody. Press “Play” to listen to your AI-assisted creation. Each time you press “Generate,” the AI will create a new pattern.

The AI is a neural network trained with thousands of monophonic melodies (monophonic means that only one note is played at a time). From this training, it learned the structure or shape of typical melodies. Neural networks can learn to do things by training with a large number of examples. This makes them ideal for applications where we cannot explain through “rules” what the task exactly is (either because there are too many rules or we don’t know them), but we can provide examples of what we want.

The neural network used in this chapter is very similar to the one used in the “Rhythm” chapter. You can read the explanation there to find out how it works.

How random are melodies?

There are no rules or scientific laws to say which sequences of notes are valid melodies and which aren’t. Music theory can explain many decisions behind compositions, but the boundaries of art can never be perfectly delineated.

Many different things can influence composers:

And within these frameworks, artists also use repetition, anticipation, surprise, and rhythm to engage listeners.

All these things generate structure: a web of relationships. Each note is related to the ones that came before (e.g., how likely or unlikely they are to be together, how similar or contrasting they are, etc.) and the ones that might come after.

The sequencer above does some very big simplifications: it only offers seven notes from the C major scale, and all notes have the same duration and loudness. This means that from the start, there’s a strong relationship between the notes we’ll use to build melodies: they’re close friends.

Still, in terms of structure, we’re only scratching the surface. Let’s see a sequencer with a new button that generates a melody randomly. At each step, it choses a note (or silence) as if by the roll of a die.

It’s reasonable to generate melodies randomly when working with a limited framework of related notes (such as a scale). If you generate many random melodies, you will come across some good ones and some that even seem to match your initial notes incredibly well (by pure chance). But you should notice differences if you generate many melodies using the AI and many with the random algorithm. The AI will generate more regular shapes, use a smaller selection of notes, and build on what you entered as prompt in the first steps. Also, you might notice that random melodies sound good only after you hear them in a loop several times (giving your brain a chance to establish patterns).

Before neural networks became popular, legacy AI systems often used a mix of rules and randomness. The sequencer below offers a third button that uses an algorithm based on a Markov chain. Try it several times.

We can explain the way this algorithm works through the following instructions:

The random algorithm can produce any possible combination of notes with equal probability. The results created by the Markov chain are hopefully more interesting, but they will all be limited by the possible combinations that these rules allow. There are a lot of interesting melodies it’s unable to produce.

We could create many more Markov chains, using different sets of rules, based on different ideas about composition and what makes melodies work. An algorithm could select one of these Markov chains at random and then use it to generate a melody. As we continue to add more chains, we would cover more and more interesting melodies.

Neural networks don’t require rules from us; They learn through a training process using examples. The potential melodies that a neural network will generate are those that follow similar structures or patterns to the ones used to train them. To build a neural network that can produce (almost) every interesting melody, we need to train it with a large amount of varied examples.

A brief history of sequencers

Sequencers, like the one used above, are a common tool in electronic music. They shorten the distance between writing and performing music. An artist can even “play the sequencer” and make writing music an artistic performance.

Food for thought

Sequencers allow us to make music by “molding” it as it plays. We can change the notes of a sequence while it loops, its rhythm, or modulate the sound of the instruments smoothly transitioning to a different mood or color. The user interface of the sequencer (knobs, pads, mouse, keyboard, touch screen, etc.) limits how many simultaneous changes we can make while the music is playing, and sometimes it’s necessary to trigger pre-programmed variations.

How can AI enable new forms of interfacing with a sequencer to control the flow of an “elastic” live performance with a lot of expressive freedom?

Early sequencers required tedious note-by-note input. MIDI controllers, like keyboards and pressure-sensitive pads, allowed more expressive, fast, and natural input of sequences. What new mechanisms can AI introduce?

External links

• Play with online versions of classic synthesizers with built-in sequencers like the TB-303 and SH-101.

Credits

Based on the Google Magenta Melody RNN demo.

Copyright and license information in our GitHub repository.

References