Project 2: A Generative Music Box
Welcome to Project #2: A Generative Music Box. As with all projects, start by reading the “Start Here” section and then proceed down the page as you complete each section. Good luck, and remember the community is here if you need help (or want to give help). Hint: you’ll find the community chat icon at the top of any section that is expanded.
Welcome to Project #2 (and if you haven’t done Project #1, well, go back and do it!). In this project, we’re going to build a cool little music generator in the spirit of Brian Eno’s and Peter Chilvers’ Bloom application.
Now this app doesn’t involve a lot of code, yet it does some interesting things. So get started now by moving on to the next section. See you there!
Prerequisites & Tools
You’re also going to need a web server of some form, and this is something we’ll get to later in the project, nothing to worry about right now.
What about source code?
You can download all the completed code for this project from github. However, resist the urge. By writing the code yourself as you progress through the project, the concepts will sink in more deeply and you’ll build better mental muscle memory. We’ll also be prompting you to try to figure some of it out on your own, which will help the concepts sink in even further. If you get stuck, first try the community, then as a last resort fall back on the source code.
With that out of the way, let’s get warmed up with a little exercise in the next section.
Before we dig in, have a look at the project preview video below.
Notice some of the interesting things about how this app works:
- Wherever you click on the canvas, a dot slowly expands into an increasingly large circle and then fades.
- The animated circle replays, over and over, with a short period of time between each replay.
- When you click, a sound is generated.
- Every time the animated circle reappears the sound is replayed.
Given this behavior, how would you implement such an app? How would you write code to generate the circles? How would you write code to replay the circles and sounds over and over?
The Game Plan
You’ve watched the video showing what you’re going to build, and thought through some of the features, so you know what the goal is. Let’s get a game plan together for how we’re going to tackle the task of building it.
Here’s the game plan for the Generative Music Box project: we’re going to build a one-page application in the browser, one that uses the <canvas> element for the user interface, and plays sounds.
The HTML and CSS is quite simple, so we’ll get that out of the way first. Then we’ll dive into getting the user interface working. The user will be interacting exclusively with the <canvas> element, so all the code we write for the View, that is, the part of the application the user sees and interacts with, will be done in the canvas.
The other main feature of this application is the audio. To add audio to our application, we’re going to use audio samples that correspond to notes from the C major scale. To play these samples, we’ll first have to load them, and then write some code to play a note when a circle is added to the interface, or a circle already added replays.
Organizing the files
- musicbox.js: This file will contain the window.onload function, as well as some simple controller code. Remember from Project #1, that controller code is what helps direct things to happen when you interact with the interface.
- View.js: This file will contain all the code related to the view. That means all the code to handle clicks on the canvas, drawing on the canvas, and animating the circles.
- BufferLoader.js: This file will contain the code to load the audio samples. We won’t go into a huge amount of detail on this code, but just enough so you know how it works.
- Audio.js: And finally, this file will contain the code to play the audio samples. We’ll coordinate playing a sample with animating a circle from the view code.
Notice that we’re roughly using a similar conceptual structure to the one we used in Project #1: the model-view-controller. In this project, our model is an array holding audio samples, which is created and loaded by the code in BufferLoader.js, and then used by the code in Audio.js. The view is our HTML, CSS, and canvas, and primarily it’s the canvas. All the code to manage the view is in View.js. Our controller is a bit more spread out: we’ll have some code in musicbox.js to define how we’ll handle clicks, but the rest of the controller code will be in a method in the View. We’ll point out these components as we get to them.
Techniques for keeping code modular
We’re going to use a couple of different techniques in this project for “modularizing” code. That just means we’re going to build the code so that all the related functions are grouped together into objects. So, all the view code is grouped into a View object, all the sample loading code is grouped together into a BufferLoader object, and all the audio playing code is grouped together into an Audio object. As you can see these modules also correspond to the files we’re creating.
We’ll use two techniques to do this: one is to just create an object using an object literal, with properties and methods in the object all related to a particular function of the code. The second technique is to use a constructor function that will create an object, again, with properties and methods related to a particular function of the code. We’ll dig into the details of why we choose one or the other as we get to each object.
Okay, that’s the plan! Let’s get going.
How do I know what the best way to organize my code is? Should I always use multiple techniques like we’re using here (file organization, MVC structure, and using objects to modularize)?
There’s an infinite number of different ways you can choose to organize your code in any project. Well, maybe not infinite, but a lot. And sometimes you may not know the best way to organize it until after you start writing it and see what you’re going to need for the project.
In this project, we’re illustrating three ways to keep code organized and well structured. First, we’re separating the code into multiple files, to keep related things together in one file. But we could just as easily put all the code in one file. Another technique we’re using is to put related properties and methods in one object. This can work to keep things modular—that is, each major piece of the implementation is one “module” of related code—whether you have all your code in one file, or split across multiple files as we’re doing. Finally, we’re using MVC—Model, View, Controller—as a way to conceptually think about organizing the code. This structure isn’t precise, but it does help you think about how the various pieces of the implementation fit together and what each role is in the application.
Create the HTML & CSS
We’re going to get the HTML and CSS into place for the Music Box application. Most of the interface is yet to come, so we need to get these basics into place first.
HTML - Coding the Markup
Here’s the HTML:
Next up, we’ll get the CSS added.
CSS - Adding some Style
The CSS for the Music Box project is super simple:
Get the canvas working
It’s time to write some code. To begin, we’re going to get the canvas into place and make sure we can do a very simple drawing to the canvas. Don’t worry, if you haven’t worked with the canvas before we’ll do a quick review of the basics.
You already know that this application has several different components: we need a canvas, which is going to act as our interface for users to add circles and sounds; we need to draw on the canvas to create the circles; we need to load and play sounds; and we need a way to coordinate everything.
As we described in the Game Plan section, we’re going to organize our code by splitting it up into several different files. Each file is going to contain the code associated with one responsibility of the application. The code that kicks everything off once the page is loaded is the code in the window.onload function, and we’re going to put that in a file named “musicbox.js”. Then, we’ll put the code for the other responsibilities: managing and drawing in the canvas, loading sounds, and playing sounds, into separate files. When we organize code this way, we say we’re separating the concerns of each part of the code so each piece stays relatively small and focused.
In this module, we’re going to start putting this structure into place by creating two pieces: the “musicbox.js” file containing the window.onload function to get the application going, and the “View.js” file containing the code to manage and draw in the canvas.
So let’s get started!
The window.onload function
Okay, we’re ready to write some code. Create a new file “musicbox.js”. This is where we’ll put the window.onload function. For now, all we’re going to do in this function is get the canvas element object from the DOM, and create a new View object. After we’ve created the View object, we’ll call a method,
updateDisplay. We haven’t written the code for the View yet, but we’ll get to that in a later section. For now, just get this code typed in so it’s ready to go:
Working with the canvas
If you haven’t worked with the canvas before, we need to cover the basics before you go any further. If you’ve read Chapter 7 of Head First HTML5 Programming, or you’ve had prior experience working with the canvas element, feel free to skip over this section and move on to the next one. But if you’re new to canvas, we recommend watching the video below before typing in the code below, so you know what’s going on.
Creating the View object
You’ve learned the basics about canvas, so now it’s time to write some code. Create a new file, “View.js” and get the code below added to the file:
Let’s step through the code. First, notice that we’re first defining a constructor function,
View, that takes one argument, the canvas object that the View is managing. So, when we create a new View object, we’ll pass in the canvas object it’s responsible for, and the View will stash that object in a property named canvas.
The next thing to notice is that we’re adding a method,
updateDisplay, to the View. We’re adding the method by adding it to the prototype. That means that if we wanted to say, have two different canvases on the page, with two different View objects managing them, we can share the same code for
updateDisplay function we’re first copying the value of
this to a variable named
view. Remember, when we call the method
updateDisplay, the object in
this will be the object whose method we call. If you take a look back at the code in “musicbox.js”, you’ll see that we’re calling
view.updateDisplay(). In other words, we’re calling
updateDisplay on the View object we’ve just created, so the value in
this will be the View object. We’re copying it to the
view variable so it’s clear that we’re referring to properties of the View object.
Next we’re using the
Okay, we have the basics of the Music Box project started: code to get the canvas from the DOM, create the View, and call the
updateDisplay method to access the canvas, so it’s time to test the code.
<body> <canvas id="canvas" width="1000" height="600"> </canvas> <script src="View.js"></script> <script src="musicbox.js"></script> </body> </html>
Save “musicbox.html”, and then open it in your favorite browser. If all goes well you’ll see a canvas in the page. It will be white with a black border, briefly, and then the canvas will be black.
If you don’t see the canvas, then here are a few things you can do to debug the situation:
- Make sure you’re using a modern browser.
- Finally, reach out to the forum if you are really stuck.
Draw a circle on the canvas
So far, all we’ve drawn in the canvas is a background color. In this module, we’ll draw a circle. Drawing a circle using the canvas isn’t quite as straightforward as you might think it would be, so we’ll dive into all the details.
How to draw a circle on the canvas
Unfortunately, there’s no handy built-in
Drawing a circle
As you just saw in the video in the previous section, drawing a circle in the canvas isn’t exactly straightforward, but now we have written a function to do this, we can use that function to draw all the circles we need for the Generative Music Box project.
We’re going to add the
drawCircle function to the View in our project. Just like we did with the function
updateDisplay, we’ll add
drawCircle to the prototype of the View constructor function, so it’s available to the view object we created to manage the view.
Once we’ve got the function added to the View prototype, we can use it to draw a circle. To test
drawCircle, we’ll call it with an x, y position of 150, 150, a radius of 100, and an alpha of 1, so the circle is completely opaque. Right now, we’ll just draw one circle from the
updateDisplay function, to test it. Later, we’ll update the code so we’re calling
drawCircle every time you click on the canvas to add a new circle, and at that point, we’ll pass in the x, y position you click on the canvas to determine the circle’s location, which will also determine the circle’s color.
Here’s the full code for this step:
If you’ve got it all added to your “View.js” file, move on to the next section where we test the code.
Let’s test the code to make a circle. For right now, all we’re doing is calling
drawCircle once, from
updateDisplay, so we should see one circle at x, y coordinates 150, 150 appear in the canvas.
Here’s what our test looks like:
Do you see the same circle?
Remember, at this point, we’re placing the circle at a specific point on the canvas, and using no alpha value. If you see a circle, then your
drawCircle method is working!
Handle Clicks on the Canvas
You know how to draw a circle on the canvas, so next, we’ll add a click handler function on the canvas, so we can draw a circle every time you click.
Handling clicks on the canvas
You know how to draw a circle on the canvas; now we need to figure out how to draw a circle each time you click on the canvas, and at the location on the canvas where you click.
For this section, we’ll modify both the View in View.js, as well as the code in musicbox.js. First, in musicbox.js, we’re going to add a click handler to the canvas. Adding a click handler to a canvas is just like adding a click handler to any other DOM element. We’ll use the
addEventListener method to add the click handler to the canvas, and we’ll define the click handler function,
handleClick, in the view.
handleClick function is responsible for adding a circle to the view whenever you click on the canvas. But we can’t just call
drawCircle to add a circle, because we want to animate the circles over and over again, as you saw in the preview video of the project. So instead,
handleClick is going to create a simple object representing a circle, containing the x and y coordinates on the click in the canvas (so we know where to add the circle). Each circle that gets added to the View will be stored in an array,
clicks so we can keep track of all the circles.
Instead of having
drawCircle to draw the circles in the clicks array, we’re going to use
updateDisplay to do that. Right now, we’re calling
updateDisplay once, to draw one test circle. We’ll change that code so that
updateDisplay will iterate through the clicks array to draw each circle stored in the array. We’ll get to that in a little bit; for now, let’s get the click handler working and make sure we store each new circle in the
Add a click handler to the canvas
There are two things we need to do to handle clicks on the canvas: add the
handleClick function and the new
clicks array to the View code, and add that function as the click handler for the canvas, which we’ll do in musicbox.js.
Here’s the code for the
handleClick function, and notice that we also added the
clicks array as a property of the View object in the constructor function. Each time you click on the canvas, we get the x, y coordinate where you clicked, and add a circle to the
For now we’ll have the click handler show a message: draw a circle at x, y. We’ll actually draw the circles in a later section.
Get the new code above added to your file, View.js.
And, here’s the code to add the
handleClick function as a click handler for the canvas. We’ll add this to the window.onload function in musicbox.js.
Once you’ve got the new code added, we’ll test it in the next section.
Crash Test the click handler
Let’s test the click handler. Now we should be able to click on the canvas and see a message in the console that we’ve added a circle, although remember, we won’t actually see the circle just yet.
Something’s wrong! We’re not seeing the message to add a circle, and you are probably seeing an error message like this:
Uncaught TypeError: Cannot read property ‘push’ of undefined
So what went wrong?
Overheard on the Forum
handleClickfunction, where we’re trying to push the click info object onto the end of the
clicksarray. I think that means the
clicksarray is undefined.
clicksarray isn’t defined that means the View isn’t being created correctly, right?
thisisn’t the correct object in the View. I’ve had trouble with
thisbefore; sometimes it doesn’t get set to the object you think it’s going to be set to.
console.logto the function to see what
thisis inside the
canvasobject instead of the
thisset to the
thisalways set to the object whose method you call? We’re calling
thisshould be set to
handleClickmethod; we’re passing it to
addEventListeneras the handler for the click event.
Why 'this' is not bound to the correct object
To see the problem with the
handleClick function, try adding a
console.log statement at the top of the function, like this:
When you look in the console, you should see that
this is the canvas object:
<canvas id="canvas" width="1000" height="600">
But why? If you look at where we’re setting up
handleClick as the click handler for the canvas (in musicbox.js), you can see that we’re passing
view.handleClick as the handler function.
handleClick is a method of the
view object, so why isn’t
this set to
Well, take a closer look. Normally, when you call a method of an object,
this is set to the object whose method you’re calling in the body of the method. But we’re not actually calling
handleClick. Instead, we’re passing a reference to the function
handleClick so that
addEventListener can call it later, when you click on the canvas. When
handleClick gets called, it is getting called as a method of the
canvas object, not as a method of the
view object. It’s a method of the
canvas object, because we’re assigning the function to the
click property of the
canvas object in the line where we call
canvas.addEventListener. That means that
handleClick, ends up getting set to the
canvas object instead of the
If you need a more in-depth explanation of why
this is set in a method call, it’s easy to lose track of what the value of
this will be.
Fortunately, our problem has an easy fix. We can force
this to be the value we want it to be in the
handleClick function by using
bind. We use
bind like this:
bind returns a new function reference to the function
handleClick, only with
this bound to the
view object. If we pass this modified function reference to
handleClick will be called when you click on the canvas, and
this will be properly set to the
view object instead of the
We’ll fix the code in the next section.
Fix the bug: Bind "this" to the View
It’s a quick and easy fix to make sure that
this is set to the
view object in
handleClick. All we have to do is use
bind to create a new function with
this set to
view, like this:
Make this change in View.js, and reload the musicbox.html page in your browser. Now you should be able to click on the canvas and see the correct console message:
Updating the View
We know how to handle clicks on the View to create circles at a specific location on the canvas; now we need to actually draw the circles on the canvas each time we update the view.
Updating the canvas
Take a look at the video of the finished music box, and take careful note of the behavior of the circles:
Each circle appears with a sound, grows to a certain size, and then disappears and the whole cycle repeats. This happens for every circle that we want to add to the canvas.
To make our circles so they behave like this, we need to repeatedly draw each circle. Each time we draw a circle we’ll draw it slightly bigger than the time before. When a circle reaches a certain maximum size, it disappears, and then starts growing again from zero.
You can think of this behavior as being almost like a movie: we have a series of “frames” that create an animation as we play them. If we play them fast enough then the animation will look smooth, just like if we play the individual frames of a movie fast enough.
We’ll break this into two steps. In the first step, we’re going to get all the circles drawn on the canvas, and make sure we can add new circles whenever you click, so you see those circles too.
To do that, we need to iterate through the
clicks array and draw all the circles in that array on the canvas. We’ll do that in the
We’re also going to change how we call
updateDisplay. Right now, we’re calling it just once, from the
window.onload function. But if we call it only once, you won’t see the circles change size (which we’ll do in the next modules), and you won’t see any new circles added to the canvas over time, as you click. So what we really need to do is call
updateDisplay over and over again—this is the movie frame part of the code.
Each time we call
updateDisplay, we’ll reset the canvas, and then redraw all the circles in the
clicks array. That way, if you load the page, and then a few seconds later, click to add a circle, you’ll see that circle appear in the canvas the next time
updateDisplay is called.
Let’s dive into the code so you can see how this is going to work.
The first step is to draw all the circles stored in the
clicks array. We’ll do this in
updateDisplay. Remember that
updateDisplay is clearing out the canvas, by drawing a black background over the whole canvas. So any circles that are currently drawn in the canvas will disappear. So we’ll iterate through the
clicks array, drawing all the circles stored there. If any new circles have been added since the last time we called
updateDisplay, they’ll get drawn too.
We get each circle from the
clicks array, and call
drawCircle with that circle’s x, y coordinates, and the radius of the circle. Right now the radius is set to 100, but we’re going to change that later so the radius varies over time. Also notice that alpha is set to 1; again we’re going to change that later so it varies depending on the radius of the circle.
We’ve also added a
frameRate property to the view, in the constructor function. The frame rate is how often we’re going to call
updateDisplay. Remember the movie analogy? This is how many frames per second we’re showing the user: 1000/30, or about 33 frames per second.
To make this work, we have to change how we’re calling
updateDisplay. Instead of just calling the function once from the window.onload function, we’re going to call it over and over again at a specific rate, defined by
frameRate. We’ll do that with
setInterval calls a function over and over again, at the rate you determine. Notice that we have to use the same trick we used before with
bind to make sure that
this in the body of
updateDisplay is set to the
view object. If we don’t use
bind, in this case,
this would be set to the
window object, which isn’t what we want.
Follow along as we add this code:
Now get your code updated and let’s test this code! If everything’s working well, you should see a circle appear in the canvas every time you click.
Crash Test: test updateDisplay
Let’s test the code. If it all works, you should see a new circle added to the canvas each time you click.
updateDisplay, we could call
drawCirclein the click handler.
To get a circle drawn in the canvas each time you click, we could indeed just call
drawCircle from the click handler. However, if we do that, we have no easy way to animate the circles.
Remember that when you draw on the canvas, all you’re doing is updating pixels at a certain location. It’s not like when you create an element in the page using HTML or SVG. If each circle was its own object, then we could, for example, write a separate function to go through all the circle objects in the canvas and animate them.
But the canvas doesn’t work that way. All we’re doing when we’re drawing on the canvas is manipulating pixels. So to make the circles look like they’re moving, we need to redraw them over and over. It really is just like an animated movie made out of individual frames. You’ve probably seen those old videos showing how the animation studios used to make movies from individual “cells” drawn just a little bit different for each frame of the movie, right? Well, we have to do exactly the same thing to make our circles move.
That means we’re creating a new drawing for each frame of the movie. And to create a new drawing, we have to re-draw all the circles in whatever configuration we want them to be in for that frame. By drawing each frame over and over at a fast enough rate, it will look just like a smooth animation does in a movie.
This is a standard animation technique when working with pixels. An alternative way to do this would be to create a separate SVG object for each circle and control each circle with a function that changes the size of the circle objects over time.
Take a Break
Okay, it’s time to take a breather and let everything sink in. After all, you’ve already got a lot of the code implemented for the Generative Music Box. So, before we move on and start implementing the circle animations, how about a little inspiration?
Grab your favorite snack and watch this video clip about how a famous animation studio used to make their movies:
Animate the circles
Next, we’re going to animate the circles we’re adding to the canvas each time you click. We’ll do it by modifying both the size of the circle each time we draw it, and by modifying the opacity of the circle so it fades out at it reaches its maximum size.
Each time you click on the canvas, a new circle is added. We do this by handling your click with the
handleClick function, which adds the x, y position of the circle to an array,
clicks. From the
window.onload function, we call
updateDisplay 33 times per second, which clears the canvas and then iterates through the
clicks array, calling
drawCircle to draw each circle.
We have the code in place to draw the circles, and now we’re going to animate them. We set up the code to call
updateDisplay 33 times per second (specified by the
frameRate property in the View), so we can animate the size of the circle by changing the circle’s radius each time we draw it so it looks a little bigger each time.
When the circle gets to a certain size, we’ll fade it by modifying the opacity of the circle. We can do that by changing the alpha value of the circle when we draw it with
Finally, when the circle reaches its maximum allowed size, we’ll stop drawing it when
updateDisplay is called—that is, we’ll skip over the circle, so it will disappear. Of course, we’ll need to make it reappear at some point, so we’ll take care of that too.
Let’s step through all of this so you can see how it’s going to work.
Code to animate the circles
Our goal for this section is to animate the circles in the Musicbox application.
Here’s the full code for the View. Go ahead and get it typed in, and then make sure you understand what we’re doing at each step. Once you do, move on to the next section and give it a try.
We’re using the objects in the
clicks array to keep track of the changing radius of each circle. Remember, each item in the
clicks array represents the x, y coordinate of a circle we’re drawing on the canvas, and now we’re using the
radius property to store the circle’s current radius.
We want each circle to grow to a maximum radius and then stop growing. We set the maximum radius in the View property
maxRadius. Each time we call
updateDisplay, we check to see if the circle’s radius is greater than this maximum radius and if it is, we don’t call
drawCircle. If it’s still less than the maximum radius, we increase the radius size by 1.
We also check the size of the radius to set the opacity using the alpha value in the rgba color. Recall that the alpha determines how opaque or transparent a color is. Until the circle reaches the maximum radius minus 15, we set the alpha value to .7, which means it’s 70% opaque. After that point, until the circle is its maximum size, we set the alpha based on the size. Try out different techniques here to vary the opacity of the circles and see what you like!
Once we’ve set the circle’s radius and its opacity, we then call
drawCircle, passing in the context, the x and y coordinates, the radius and the opacity.
Let’s give this a try!
Here’s a short video of our test of the code:
The circles are animating—that is, they are growing from 0 to their maximum size, and then disappearing, and you can see that towards the end of the growth, they change in opacity. So our code is working great.
But right now, the circles are only animating once. So our next step is to repeat the animation so the circles reappear and grow again, over and over.
To make the circle animations repeat, we need to reset each circle’s radius back to 0 at some point after it reaches the maximum radius. There are a few options for how to do this; the way we’re going to do it is to set a timer when we first create a circle. This timer will call a function to reset the circle’s radius back to 0 after a certain period of time has passed.
Follow along as we write this code:
Here’s the code we just added; get this added to your code and then we’ll test it, and hopefully we’ll see our circles repeating the animation, growing to a maximum size, disappearing, and then repeating.
Here’s our test; how is yours looking? Is it working like you expected?
Try playing around with the
loopRate. If you make it too short, the circles won’t animate correctly. Too long and it won’t feel right. See what timing works for you. Remember that
loopRate is milliseconds so just multiply the number of seconds you want by 1000.
We have our circles animating beautifully, so the next step is to add the sound. Time for the next module!
You could! For instance, you could write another
for loop to run after the loop we already have to go through each circle in the
clicks array, check to see which circles have a radius greater than the maximum radius, and reset the radius there.
However, that will mean there is no delay between when a circle disappears and when it starts growing again, and we would like to have a short delay. It also means that the code will be slightly less efficient since we have to loop through the
clicks array twice; first to update the radius and draw the circles, and second to reset the circles that have reached maximum size.
In addition, when we get around to adding the audio to the music box, you’ll see why it’s a bit better to start the timer to reset the circles in the
handleClick function. It’ll be a little bit easier to synchronize the circles and the audio if we reset the circles from a timer there.
Of course, there are many different options for how to write this code, depending on how you want the music box to behave, so explore your options and test things out! When you’re working with timers (and with audio) it can get a bit tricky to keep things straight about what’s happening when so it’s a good exercise to step through this code on paper, and any other variations you try, and make sure you understand it.
Add the audio
The circles for our Music Box project are working great and the View is done, so now it’s time to add some audio.
We’ll be adding audio in four steps. The first step is to add a new object, Audio, which will contain properties and methods we need to play audio samples. Next, we’ll create a BufferLoader object. This object will load audio samples from sample files, and get them stored in an array in a format that the Audio object can use to play sounds. We’ll update the
window.onload function in musicbox.js to kick things off by loading and initializing the audio.
Finally, once we’ve done all that, we can play the Audio. We want to play a sound with each circle on the canvas; to do this, we’ll need to make just a small change to the
handleClick function in the View.
We’re going to go pretty quickly through the code to load, initialize, and play audio. For a more in-depth description of the AudioContext web API and using the BufferLoader to load audio data, check out this HTML5 Rocks article.
Okay, let’s jump in and get going!
Prep: Audio samples
Before you begin writing any code, make sure you have the audio samples ready to go. If you haven’t already, download the samples from github, and put them in the same folder where your code is. You should have ten MP3 samples that contain notes from the C major pentatonic scale. We chose these notes so when they’re played together in the music box, they sound good!
Prebaked Goodness: Audio.js
AudioContext constructor to create an
AudioContext object. After creating the
AudioContext object, we’ll initialize it with sounds, so it’s ready to play. We’ll put all the code associated with generating audio from the sounds in a new file, “Audio.js”, so go ahead and create this file now, and copy in the code below.
Let’s quickly step through the code so you understand how it works. First, notice that we’re encapsulating all the code related to audio in an object named
Audio object has five properties, two of which are methods. The
gainNode will be used to control the volume; the
bufferList will be an array holding the sounds to play; and the
audioContext is the object that knows how to decode binary audio data and make the browser play a sound.
Audio operations, like making a sound or adjusting the volume, are handled with nodes that are connected together. Audio people think of the groups of nodes as “audio routing graphs”, which is where this “node” language comes from. For this example, we’re keeping it extremely simple. We just need two nodes: one to create the sound, and one to generate the sound.
We’ll be calling the
init method to initialize the
Audio object with the sounds it needs to play. We’ll also use this method to set up the volume at which the sounds will be played. The function takes an array,
bufferList, of sounds, which it stores in the property
bufferList. Each item in this array will be one of the sounds from the audio files you downloaded earlier.
To set the volume for the audio, we create a gain node and connect it to the audio context object. The gain (the amount of volume to apply to your sound) on the gain node is set to 1, to play at maximum volume. If you wanted to play at half volume, you’d set this to 0.5.
To actually play a sound, we’ll be calling the
play method. We’ll be doing this in the View object, and we’ll get to that shortly. The
play method takes an index of one of the sounds in the
bufferList and prepares that for playing by creating a buffer source. This source is then connected to the gain node and the sound is started. A started sound will play to completion and then end. Each time we need to play a sound, we recreate the buffer source and connect it to the gain node.
We have the code to take sounds and play them; next up, we’ll tackle loading the sounds from the MP3 files.
Prebaked Goodness: BufferLoader.js
Next up is the buffer loader. This is the code that reads the audio data (the sound samples) from the MP3 files and loads the data into the buffer list array we use in “Audio.js” to play the sounds.
We’re going to put all the code responsible for loading the sound data into a separate file, “BufferLoader.js”. Go ahead and create that file now, copy the code below, and save. You’ll then need to add a link to this file in “musicbox.html”, just like you did for “Audio.js”. Put this link above the link to “Audio.js”.
Here’s the full code for the BufferLoader:
BufferLoader is a constructor function, that takes the
AudioContext as its first argument, an array of filenames as the second argument, and a callback function as the third argument. We’ll use this constructor to create a new
BufferLoader object in “musicbox.js” in the next section.
loadBuffer method is what does the bulk of the work, so let’s start there. We call
loadBuffer from the
load function (which we’ll call from “musicbox.js”) for each sound in the list of files. What gets passed to the
loadBuffer method is the URL of the file containing the sound, e.g. “A4.mp3”, and the index of that sound in the list. We’ll use that index later to make sure we get the sounds added to the buffer list in the correct order.
XMLHttpRequest to get the data from the file. Just like a typical
XMLHttpRequest request, we make the object, set a callback (in the
onload property) and then call
send() to kick off the request.
Notice that the
responseType we’re expecting is “arraybuffer”. This is probably different from what you’re used to when working with
XMLHttpRequest to get JSON or plain text. We use this response type when we’re working with binary data. In this example, we’re working with MP3 binary data, and even though the “arraybuffer” type is generic (meaning, it can be any kind of binary data), we can rely on the
AudioContext to correctly recognize that binary data as MP3.
When the request response is ready, the callback function we define for the
request.onload property is called. Here, we call the
decodeAudioData to decode the binary data in the file into a sound the
AudioContext can use. The first argument is the binary data in the
response and the second argument is a callback function that is called if the decoding is successful.
The callback is called with the sound data passed into the parameter
buffer. Next, we save this sound into the
bufferList array, which will contain all the sounds once the
BufferLoader has completed loading the sounds from the files. Notice here that we don’t want to just push the sound onto the end of the
bufferList array because the sounds might end up in the wrong order, and we want to keep them in the order we specify (when we pass them into the
BufferLoader in the
urlList). Remember that making a XMLHttpRequest is asynchronous so the sounds could be loaded out of order. Since we passed in the index where this sound needs to go in the
bufferList we can use that to put it in the right spot.
We’re keeping track of how many sounds we’ve loaded with
loadCount. Once this count is the same as the length of the original array of sounds we passed in, we know we’ve loaded all the sounds, and we can then call the callback function we stored in the
onload property of the
BufferLoader object, and pass in the fully loaded array of sounds,
We’ll write this callback function in the next section, where we modify “musicbox.js” to make this
BufferLoader object and specify the callback function we’ll use to kick off the music box once all the sounds are loaded.
So hang in there! Just a couple more changes and then you’ll be ready to test your music box project.
Prebaked Goodness: musicbox.js
We’ve got two new files, “BufferLoader.js” and “Audio.js” that we’ll use to load and play sounds. Now, we need to make some changes to “musicbox.js” to use the BufferLoader, and supply a callback function that will be called once the sounds are loaded. This callback function will initialize the sounds in the audio context, and then do what our code already does: set up the view, the canvas click handler, and the call to
updateDisplay to get our animation started.
Open up the file “musicbox.js” and make sure you update your code with the full code below:
The first thing we’re doing in the
window.onload function now is creating a new
BufferLoader object. Recall that the constructor function for the
BufferLoader takes three arguments: the audio context, which is a property of the
Audio object, an array of URLs (which in our case are just file names) for the sound files, and a callback function to call once the sounds have been successfully loaded. Note that we’re assuming the sound files are in the same folder as your code, so the path to the sounds is the same as the path to the music box application.
The callback function we’re specifying is the
finishedLoading function which is defined in the
window.onload function also. We’ll go over what that does in a moment.
Once we’ve created the
BufferLoader object, we call the
load method. Remember,
load iterates through all the file names in the array, and passes the file name, along with the index of the file name in the list, to the
loadBuffer method which is responsible for loading the sounds as binary data, using
loadBuffer has successfully loaded the sounds, the callback function,
finishedLoading is called (to see where we do this, look at the “BufferLoader.js” file for the call to
loader.onload). What gets passed into the
finishedLoading function is the
bufferList array in the
BufferLoader object, which contains all the decoded sounds from the MP3 files.
finishedLoading function contains the code we had previously written to set up the view, the click handler for the canvas, and start the calls to
updateDisplay to kick off the animation. In addition, we’ve added one line at the top to call the
Audio.init method, passing in the
bufferList array. This array of sounds gets stored in the
Audio object and used when we call the
Play audio with each circle
We’ve added code to include the capability to play an audio sample, so now, we’re ready to update the code in the View to do just that. All we need to do is call
Audio.play and pass in an index into the
bufferList array, which contains our list of audio samples.
Take a look at the
handleClick function in the View (“View.js”). Remember that this is where we add a new circle to the music box, and create a timer for each new circle that resets that circle’s radius to 0 every four seconds so that the circle will continue to animate over and over again. Our goal is to play a sound that begins to play roughly in synch with when a circle begins to animate; that is, when the circle’s radius is 0. And of course, we want that sound to repeat each time the circle starts animating again.
We can do this by calling
handleClick, just before we call
setInterval. This will play a sound when a circle is first added. Then, we’ll also add a call to
Audio.play in the anonymous function we’re passing to
setInterval, so that the sound repeats each time the circle’s animation begins.
But which sound do we want to play for a given circle? Well, we know we have ten different samples, so that means the number we pass to
Audio.play should be between 0 and 9. We could randomly choose a number in this range, but we can do better by basing the audio sample we select on the location of the circle on the canvas. You could pick x or y or some combination of the two for this; we’re going to keep it simple and use x. To get a number between 0 and 9 based on the x location of the circle, we’ll use x % 10, which is x “modulus” 10: divide x by 10 and take the remainder. This number is a number between 0 and 9.
Now you know what we’re going to do, and why we’re doing it, so it’s time to add this code to the
handleClick function in your “View.js” file:
Get ready for the final test!
Let’s test all the code we just added. We’ve added quite a lot in this section: we added two completely new files, “Audio.js” and “BufferLoader.js”. We updated “musicbox.js” to use the Audio object, and created a
BufferLoader object, using our previous code in a callback function to call once the audio is loaded and ready to go. And finally, we updated “View.js” to play an audio sample each time a circle begins animating.
Before you test, make sure you’ve got all the code added in the right places, and with the correct filenames. Also make sure you’ve got everything linked from “musicbox.html” correctly; at this point, your code in “musicbox.html” should look like this:
Okay, you’ve double checked all your code, so let’s give it a try:
Hopefully your music box is working, and you’re hearing sounds as you click to add circles. Have fun, play around with the music box. It can be quite addicting!
The Finish Line
You’ve completed The Generative Music Box project, and implemented an application using audio and canvas graphics in the browser. We hope you’re having fun playing with the application, making some ambient music and some cool visuals. Once you’re ready, take a look at some things you might do next.
Why Stop Now? How about…
Extending your project?
We’ve really just scratched the surface of this project. You can take it a lot further. For instance, think about how you might go further with the audio. You could:
- Create an algorithm that only plays the notes some of the time. Perhaps each sound has a probability of playing each time it is scheduled to play.
- Change how sounds die over time. This would lead to the composition decaying or being forgotten over time. How would you code this?
- Change how sounds are played. We’re picking the sample that plays on a given click by mod’ing the x value of the mouse click by the number of sounds. That’s pretty simplistic. What other ways can you think of doing this?
- Create your own sound samples and see how this affects the feel of the app.
Doing more research?
Research the Web Audio API. Think about using an oscillator instead of samples or adding audio events (reverb, delay, etc.).
Or, check out HTML’s canvas element and see how far you can push the graphics.
Or, blow our minds and yours…
How about revisiting project #1, The Game of Life, and combining code? Can the Game of Life generate sound too, perhaps by assigning sounds to grid locations? You might also check out apps and devices like the Tenori-on for inspiration.
In any case, stay in touch and let us know what cool things you’ve written!