Rough Prototype to Generate Songs From Text
Annotated screen capture of the working prototype.
To explore a concept for an installation about data and sound, I started prototyping ideas in a Codepen project.
If you happen to notice the date I published the project - yep it was a while ago. Made this back in 2016. A good separate topic: ideas can be timeless and it can take many cycles of exploration and shelving a project before it becomes the thing you learn it needed to become.
As to this exploration, some things I wanted the prototype to do:
- Create sound from data without sounding too terrible. Generating white noise might be OK but shrieking high pitches and nails on a chalkboard types of sounds are out of bounds.
- Show some dynamic visuals that relate and reinforce the data informed recipe of the data's sound.
- Get the meaning and patterns from text based on at least two ingredients: the sentiment score and length of text.
- Get interesting variety through randomality - yet constrain the randomness to get the same dice roll results given the same input text.
For want number 1, I constrained the sound generation by having a recipe approach of overall how many instruments, which rhythm, and which synth sounds. After a bit of exploring I found an approachable sound library called Gibber.js. It helps make sounds both as synthesized instruments and as synthesizer playing a sequence of notes for multiple instruments at once.
Then for want number 2 I chose P5.js, which is a fun creative coding javascript library suitable for all kinds of interactive visuals and sound.
For want 3 I used the AFINN library. It's not perfect in how it "understands" the emotional intent of a given portion of text, but it has a kind of quirky close enough consistency. Consistent-enough is a good place to start for this use.
And for want 4 I used the seedrandom JavaScript library to treat the text as a random seed. Then any time I "roll dice" in the code using Math.seedrandom() I get the same seeded random result given the same input text. Change anything about the text and it's a different consistent seed.
All those parts are in the prototype. To use the sound generator, run the prototype, change the text in the text field, then click the "generate music using this text" button. The sounds and visuals should start right away.
It's a fairly fun to play with prototype being curious about what text will create what results. Here are a few clips I recorded while playing with the text to find fun results. You can recreate any of these songs in the demo by using the same text.
This clip uses the text "the peaceful lake blue sky happiness" which sounds more uptempo metal thrash than I'd expect.
And another clip of "this demo rocks so much i want to keep trying" which sounds kind of poppy.
Finally "so glad to get the band back together and make a new song" is far more down tempo than the text implies:
This project is available on CodePen:
See the Pen Generating Music Based on Text and Seeded Randomality by Rob Stenzinger (@RobStenzinger) on CodePen.
Also on Github as a Gist:
Further Iteration
Some future version of this I would like to improve both the visuals and the songs with more playful predictability mixed with surprise variety. I could see this becoming part of the Holiday/Halloween/ Gremlin cardboard tree project which I discussed on a Polytechnicast Episode.
For now, it's a fun toy. If you find a fun song generating phrase, please send it my way.