Project 2: A Generative Music Box

In this project we take inspiration from Brian Eno and Peter Chilvers’ Bloom application and create a single page app, using HTML, CSS and JavaScript, that allows you to create generative, ambient music. With a small amount of code we’re going to handle your mouse clicks, use the canvas for graphics, and leverage the Web Audio API to create sound. We’re also going to spend a lot of time on scheduling events in time and organizing our code.

PreviewStart Now

Welcome

Welcome to Project #2: A Generative Music Box. As with all projects, start by reading the “Start Here” section and then proceed down the page as you complete each section. Good luck, and remember the community is here if you need help (or want to give help). Hint: you’ll find the community chat icon at the top of any section that is expanded.

Start Here

Welcome to Project #2 (and if you haven’t done Project #1, well, go back and do it!). In this project, we’re going to build a cool little music generator in the spirit of Brian Eno’s and Peter Chilvers’ Bloom application.

Like Project #1 this is a one-page app that runs right in your browser and relies only on HTML, CSS and JavaScript. This time, we’re going to use HTML’s canvas element for graphics and the Web Audio Sound API to generate sound. We’ll also structure things in a slightly more sophisticated way than we did in Project #1 and use a couple of more advanced JavaScript techniques.

Now this app doesn’t involve a lot of code, yet it does some interesting things. So get started now by moving on to the next section. See you there!

Prerequisites & Tools

Before we get started, let’s talk about what you need to know to take on this project. This is a browser-based project involving HTML, CSS and JavaScript—–our use of HTML and CSS is quite minimal, although you’ll need to know about basic HTML elements and how to use classes and ids from JavaScript. We also use the HTML canvas element for the graphics in this project, and while you don’t need to know this element well, you might want to read through Chapter 7 of Head First HTML5 Programming (or a similar reference) to get familiar with it.

Of course, you’ll need to understand the fundamentals of JavaScript programming; we’ll be using functions, loops, arrays, and, because this is a web app, we’ll also be lightly manipulating the DOM and handling events using JavaScript code. If you’ve read an introductory JavaScript book, you’ll be prepared; of course, we’re partial to Head First JavaScript Programming, and if you’ve read that book, you’ll be in great shape to take on The Generative Music Box project.

Tools

So what tools do you need to complete this project? Simple: one or more modern browsers (the latest version of Chrome, Firefox, Safari, or IE will do just fine) and any editor. On this latter point we’re agnositic: if you love IntelliJ, good for you. If a stripped down text editor or vi is your thing? More power to you. Just make sure whatever editor you use is either designed specifically for HTML, CSS and JavaScript programming, or is in “plain text” mode (that means, don’t use Microsoft Word!).

You’re also going to need a web server of some form, and this is something we’ll get to later in the project, nothing to worry about right now.

What about source code?

You can download all the completed code for this project from github. However, resist the urge. By writing the code yourself as you progress through the project, the concepts will sink in more deeply and you’ll build better mental muscle memory. We’ll also be prompting you to try to figure some of it out on your own, which will help the concepts sink in even further. If you get stuck, first try the community, then as a last resort fall back on the source code.

With that out of the way, let’s get warmed up with a little exercise in the next section.

Exercise

Before we dig in, have a look at the project preview video below.

Notice some of the interesting things about how this app works:

  • Wherever you click on the canvas, a dot slowly expands into an increasingly large circle and then fades.
  • The animated circle replays, over and over, with a short period of time between each replay.
  • When you click, a sound is generated.
  • Every time the animated circle reappears the sound is replayed.

Given this behavior, how would you implement such an app? How would you write code to generate the circles? How would you write code to replay the circles and sounds over and over?

Think about the technologies you have at hand (the browser with HTML, CSS and JavaScript), make some notes, and then proceed.  Don’t worry about the details of how to play the sound; we’ll deal with that later.

The Game Plan

You’ve watched the video showing what you’re going to build, and thought through some of the features, so you know what the goal is. Let’s get a game plan together for how we’re going to tackle the task of building it.

Game Plan

Here’s the game plan for the Generative Music Box project: we’re going to build a one-page application in the browser, one that uses the <canvas> element for the user interface, and plays sounds.

The HTML and CSS is quite simple, so we’ll get that out of the way first. Then we’ll dive into getting the user interface working. The user will be interacting exclusively with the <canvas> element, so all the code we write for the View, that is, the part of the application the user sees and interacts with, will be done in the canvas.

The other main feature of this application is the audio. To add audio to our application, we’re going to use audio samples that correspond to notes from the C major scale. To play these samples, we’ll first have to load them, and then write some code to play a note when a circle is added to the interface, or a circle already added replays.

Organizing the files

Unlike in Project #1, for this project, we’re going to split the JavaScript code over multiple code files to try to keep it organized by function. So, we’re going to have the following files:

  • musicbox.js: This file will contain the window.onload function, as well as some simple controller code. Remember from Project #1, that controller code is what helps direct things to happen when you interact with the interface.
  • View.js: This file will contain all the code related to the view. That means all the code to handle clicks on the canvas, drawing on the canvas, and animating the circles.
  • BufferLoader.js: This file will contain the code to load the audio samples. We won’t go into a huge amount of detail on this code, but just enough so you know how it works.
  • Audio.js: And finally, this file will contain the code to play the audio samples. We’ll coordinate playing a sample with animating a circle from the view code.

Notice that we’re roughly using a similar conceptual structure to the one we used in Project #1: the model-view-controller. In this project, our model is an array holding audio samples, which is created and loaded by the code in BufferLoader.js, and then used by the code in Audio.js. The view is our HTML, CSS, and canvas, and primarily it’s the canvas. All the code to manage the view is in View.js. Our controller is a bit more spread out: we’ll have some code in musicbox.js to define how we’ll handle clicks, but the rest of the controller code will be in a method in the View. We’ll point out these components as we get to them.

Techniques for keeping code modular

We’re going to use a couple of different techniques in this project for “modularizing” code. That just means we’re going to build the code so that all the related functions are grouped together into objects. So, all the view code is grouped into a View object, all the sample loading code is grouped together into a BufferLoader object, and all the audio playing code is grouped together into an Audio object. As you can see these modules also correspond to the files we’re creating.

We’ll use two techniques to do this: one is to just create an object using an object literal, with properties and methods in the object all related to a particular function of the code. The second technique is to use a constructor function that will create an object, again, with properties and methods related to a particular function of the code. We’ll dig into the details of why we choose one or the other as we get to each object.

Okay, that’s the plan! Let’s get going.

Audience Question

How do I know what the best way to organize my code is? Should I always use multiple techniques like we’re using here (file organization, MVC structure, and using objects to modularize)?

 

There’s an infinite number of different ways you can choose to organize your code in any project. Well, maybe not infinite, but a lot. And sometimes you may not know the best way to organize it until after you start writing it and see what you’re going to need for the project.

In this project, we’re illustrating three ways to keep code organized and well structured. First, we’re separating the code into multiple files, to keep related things together in one file. But we could just as easily put all the code in one file. Another technique we’re using is to put related properties and methods in one object. This can work to keep things modular—that is, each major piece of the implementation is one “module” of related code—whether you have all your code in one file, or split across multiple files as we’re doing. Finally, we’re using MVC—Model, View, Controller—as a way to conceptually think about organizing the code. This structure isn’t precise, but it does help you think about how the various pieces of the implementation fit together and what each role is in the application.

Create the HTML & CSS

We’re going to get the HTML and CSS into place for the Music Box application. Most of the interface is yet to come, so we need to get these basics into place first.

HTML - Coding the Markup

The HTML and CSS for the Music Box project is about as simple as it gets. For the HTML, all we really need is the HTML to make a page containing a <canvas> element. The rest of the interface will all be built inside the canvas—that is, we’ll respond to clicks on the canvas, make sounds, and make animated circles, all using JavaScript.

Here’s the HTML:

Save this as “musicbox.html”. Notice that we’re linking to “musicbox.css”, which we’ll get to in a moment. In the body of the HTML, we’ve got just one element: the <canvas> element. We’ll be adding some <script> elements below the canvas to link to our JavaScript later on, but this is all we need for HTML structure. The canvas has an id, “canvas”, to make it easier to get the canvas from the DOM, and a width and height to size it for the page. Feel free to change the width and height to suit you and your screen.

Next up, we’ll get the CSS added.

CSS - Adding some Style

The CSS for the Music Box project is super simple:

Yep, that’s all we need. We’re just adding a 1 pixel black border around the canvas so you can see it as the page is loading. Once it’s loaded we’ll be modifying the canvas with JavaScript. Notice that we’re using the canvas id, “canvas”, to select the canvas and apply the style.

Get the canvas working

It’s time to write some code. To begin, we’re going to get the canvas into place and make sure we can do a very simple drawing to the canvas. Don’t worry, if you haven’t worked with the canvas before we’ll do a quick review of the basics.

Code Design

You already know that this application has several different components: we need a canvas, which is going to act as our interface for users to add circles and sounds; we need to draw on the canvas to create the circles; we need to load and play sounds; and we need a way to coordinate everything.

MVC diagram for Musicbox

As we described in the Game Plan section, we’re going to organize our code by splitting it up into several different files. Each file is going to contain the code associated with one responsibility of the application. The code that kicks everything off once the page is loaded is the code in the window.onload function, and we’re going to put that in a file named “musicbox.js”. Then, we’ll put the code for the other responsibilities: managing and drawing in the canvas, loading sounds, and playing sounds, into separate files. When we organize code this way, we say we’re separating the concerns of each part of the code so each piece stays relatively small and focused.

In this module, we’re going to start putting this structure into place by creating two pieces: the “musicbox.js” file containing the window.onload function to get the application going, and the “View.js” file containing the code to manage and draw in the canvas.

We won’t do the complete code for these pieces just yet; we’re just getting started, and we’re going to take it a step at a time. The goal here is to introduce the canvas so you can learn how we can use a canvas element and JavaScript code to create an interface for the application.

So let’s get started!

The window.onload function

Okay, we’re ready to write some code. Create a new file “musicbox.js”. This is where we’ll put the window.onload function. For now, all we’re going to do in this function is get the canvas element object from the DOM, and create a new View object. After we’ve created the View object, we’ll call a method, updateDisplay. We haven’t written the code for the View yet, but we’ll get to that in a later section. For now, just get this code typed in so it’s ready to go:

Working with the canvas

If you haven’t worked with the canvas before, we need to cover the basics before you go any further. If you’ve read Chapter 7 of Head First HTML5 Programming, or you’ve had prior experience working with the canvas element, feel free to skip over this section and move on to the next one. But if you’re new to canvas, we recommend watching the video below before typing in the code below, so you know what’s going on.

Creating the View object

You’ve learned the basics about canvas, so now it’s time to write some code. Create a new file, “View.js” and get the code below added to the file:

Let’s step through the code. First, notice that we’re first defining a constructor function, View, that takes one argument, the canvas object that the View is managing. So, when we create a new View object, we’ll pass in the canvas object it’s responsible for, and the View will stash that object in a property named canvas.

The next thing to notice is that we’re adding a method, updateDisplay, to the View. We’re adding the method by adding it to the prototype. That means that if we wanted to say, have two different canvases on the page, with two different View objects managing them, we can share the same code for updateDisplay for both views, and both canvases. If you need a refresher on how prototypes in JavaScript work, check out Chapters 12 and 13 of Head First JavaScript Programming or an online reference.

In the updateDisplay function we’re first copying the value of this to a variable named view. Remember, when we call the method updateDisplay, the object in this will be the object whose method we call. If you take a look back at the code in “musicbox.js”, you’ll see that we’re calling view.updateDisplay(). In other words, we’re calling updateDisplay on the View object we’ve just created, so the value in this will be the View object. We’re copying it to the view variable so it’s clear that we’re referring to properties of the View object.

Next we’re using the canvas object we stored in the View, and getting the canvas’s context. The context is what we use to actually draw in the canvas. We’re getting a “2d” context because we’re going to draw in 2D. Then we clear the canvas (erasing any previous things that we might have drawn in it), set the fill color to black, and then fill the canvas with black. If you need a refresher on canvas, read Chapter 7 from Head First JavaScript Programming or watch the video above.

Okay, we have the basics of the Music Box project started: code to get the canvas from the DOM, create the View, and call the updateDisplay method to access the canvas, so it’s time to test the code.

Crash Test

At this point you should have a set of files, including “musicbox.html”, “musicbox.css”, “musicbox.js”, and “View.js” all in the same folder. Open up “musicbox.html” in your editor. You should already have a link to “musicbox.css” in the <head> of the document. We need to add links to the two JavaScript files you just created, “musicbox.js”, and “View.js”. Add <script> tags to do this below the canvas, just above the closing </body> tag near the bottom of your file. The bottom part of your code should look like this:

<body>
  <canvas id="canvas" width="1000" height="600">
  </canvas>
  <script src="View.js"></script>
  <script src="musicbox.js"></script>
</body>
</html>

Save “musicbox.html”, and then open it in your favorite browser. If all goes well you’ll see a canvas in the page. It will be white with a black border, briefly, and then the canvas will be black.

If you don’t see the canvas, then here are a few things you can do to debug the situation:

  1. Make sure you’re using a modern browser.
  2. Double check all your code, markup and JavaScript for missing elements or typos.
  3. Make use of your JavaScript console to see if there are any error messages.
  4. Finally, reach out to the forum if you are really stuck.

Draw a circle on the canvas

So far, all we’ve drawn in the canvas is a background color. In this module, we’ll draw a circle. Drawing a circle using the canvas isn’t quite as straightforward as you might think it would be, so we’ll dive into all the details.

How to draw a circle on the canvas

Unfortunately, there’s no handy built-in drawCircle function in JavaScript that will draw a circle on a canvas. So we’re going to write that function ourselves. Watch the video below to follow along as we write the drawCircle function.

Drawing a circle

As you just saw in the video in the previous section, drawing a circle in the canvas isn’t exactly straightforward, but now we have written a function to do this, we can use that function to draw all the circles we need for the Generative Music Box project.

We’re going to add the drawCircle function to the View in our project. Just like we did with the function updateDisplay, we’ll add drawCircle to the prototype of the View constructor function, so it’s available to the view object we created to manage the view.

Once we’ve got the function added to the View prototype, we can use it to draw a circle. To test drawCircle, we’ll call it with an x, y position of 150, 150, a radius of 100, and an alpha of 1, so the circle is completely opaque. Right now, we’ll just draw one circle from the updateDisplay function, to test it. Later, we’ll update the code so we’re calling drawCircle every time you click on the canvas to add a new circle, and at that point, we’ll pass in the x, y position you click on the canvas to determine the circle’s location, which will also determine the circle’s color.

Here’s the full code for this step:

If you’ve got it all added to your “View.js” file, move on to the next section where we test the code.

Crash Test

Let’s test the code to make a circle. For right now, all we’re doing is calling drawCircle once, from updateDisplay, so we should see one circle at x, y coordinates 150, 150 appear in the canvas.

Here’s what our test looks like:

A circle drawn on the canvas

Do you see the same circle?

Remember, at this point, we’re placing the circle at a specific point on the canvas, and using no alpha value. If you see a circle, then your drawCircle method is working!

Handle Clicks on the Canvas

You know how to draw a circle on the canvas, so next, we’ll add a click handler function on the canvas, so we can draw a circle every time you click.

Handling clicks on the canvas

You know how to draw a circle on the canvas; now we need to figure out how to draw a circle each time you click on the canvas, and at the location on the canvas where you click.

For this section, we’ll modify both the View in View.js, as well as the code in musicbox.js. First, in musicbox.js, we’re going to add a click handler to the canvas. Adding a click handler to a canvas is just like adding a click handler to any other DOM element. We’ll use the addEventListener method to add the click handler to the canvas, and we’ll define the click handler function, handleClick, in the view.

The handleClick function is responsible for adding a circle to the view whenever you click on the canvas. But we can’t just call drawCircle to add a circle, because we want to animate the circles over and over again, as you saw in the preview video of the project. So instead, handleClick is going to create a simple object representing a circle, containing the x and y coordinates on the click in the canvas (so we know where to add the circle). Each circle that gets added to the View will be stored in an array, clicks so we can keep track of all the circles.

Instead of having handleClick call drawCircle to draw the circles in the clicks array, we’re going to use updateDisplay to do that. Right now, we’re calling drawCircle from updateDisplay once, to draw one test circle. We’ll change that code so that updateDisplay will iterate through the clicks array to draw each circle stored in the array. We’ll get to that in a little bit; for now, let’s get the click handler working and make sure we store each new circle in the clicks array.

Add a click handler to the canvas

There are two things we need to do to handle clicks on the canvas: add the handleClick function and the new clicks array to the View code, and add that function as the click handler for the canvas, which we’ll do in musicbox.js.

Here’s the code for the handleClick function, and notice that we also added the clicks array as a property of the View object in the constructor function. Each time you click on the canvas, we get the x, y coordinate where you clicked, and add a circle to the clicks array.

For now we’ll have the click handler show a message: draw a circle at x, y. We’ll actually draw the circles in a later section.

Get the new code above added to your file, View.js.

And, here’s the code to add the handleClick function as a click handler for the canvas. We’ll add this to the window.onload function in musicbox.js.

Once you’ve got the new code added, we’ll test it in the next section.

Crash Test the click handler

Let’s test the click handler. Now we should be able to click on the canvas and see a message in the console that we’ve added a circle, although remember, we won’t actually see the circle just yet.

Something’s wrong! We’re not seeing the message to add a circle, and you are probably seeing an error message like this:

Uncaught TypeError: Cannot read property ‘push’ of undefined

So what went wrong?

Overheard on the Forum
What do you think that error means, “Uncaught TypeError: Cannot read property ‘push’ of undefined”?
It looks like the error’s happening in the handleClick function, where we’re trying to push the click info object onto the end of the clicks array. I think that means the clicks array is undefined.
If the clicks array isn’t defined that means the View isn’t being created correctly, right?
Could be that, or it could be that this isn’t the correct object in the View. I’ve had trouble with this before; sometimes it doesn’t get set to the object you think it’s going to be set to.
Let’s test it and see. I’ve added a console.log to the function to see what this is inside the handleClick function…
And?
It looks like this is the canvas object instead of the view object.
Why is this set to the canvas object? Isn’t this always set to the object whose method you call? We’re calling view.handleClick in the addEventListener call, so this should be set to view, right?
Take another look. We’re not actually calling the handleClick method; we’re passing it to addEventListener as the handler for the click event.
Hmmm. There must be something about the way we’re passing the method that’s messing up the value of this in the handleClick function.
Yes, I think so. We need to look at this a bit more deeply to figure it out…
Why 'this' is not bound to the correct object

To see the problem with the handleClick function, try adding a console.log statement at the top of the function, like this:

console.log(this);

When you look in the console, you should see that this is the canvas object:

<canvas id="canvas" width="1000" height="600">

But why? If you look at where we’re setting up handleClick as the click handler for the canvas (in musicbox.js), you can see that we’re passing view.handleClick as the handler function. handleClick is a method of the view object, so why isn’t this set to view?

Well, take a closer look. Normally, when you call a method of an object, this is set to the object whose method you’re calling in the body of the method. But we’re not actually calling handleClick. Instead, we’re passing a reference to the function handleClick so that addEventListener can call it later, when you click on the canvas. When handleClick gets called, it is getting called as a method of the canvas object, not as a method of the view object. It’s a method of the canvas object, because we’re assigning the function to the click property of the canvas object in the line where we call canvas.addEventListener. That means that this, in handleClick, ends up getting set to the canvas object instead of the view object.

If you need a more in-depth explanation of why this isn’t getting set to the correct object, take a look at this free Head First JavaScript Programming Extra about this very issue. This is a common mistake to make, and if you’re not very careful about tracking why and how this is set in a method call, it’s easy to lose track of what the value of this will be.

Fortunately, our problem has an easy fix. We can force this to be the value we want it to be in the handleClick function by using bind. We use bind like this:

view.handleClick.bind(view);

Calling bind returns a new function reference to the function handleClick, only with this bound to the view object. If we pass this modified function reference to addEventListener, handleClick will be called when you click on the canvas, and this will be properly set to the view object instead of the canvas object.

We’ll fix the code in the next section.

Fix the bug: Bind "this" to the View

It’s a quick and easy fix to make sure that this is set to the view object in handleClick. All we have to do is use bind to create a new function with this set to view, like this:

Make this change in View.js, and reload the musicbox.html page in your browser. Now you should be able to click on the canvas and see the correct console message:

Updating the View

We know how to handle clicks on the View to create circles at a specific location on the canvas; now we need to actually draw the circles on the canvas each time we update the view.

Updating the canvas

Take a look at the video of the finished music box, and take careful note of the behavior of the circles:

Each circle appears with a sound, grows to a certain size, and then disappears and the whole cycle repeats. This happens for every circle that we want to add to the canvas.

To make our circles so they behave like this, we need to repeatedly draw each circle. Each time we draw a circle we’ll draw it slightly bigger than the time before. When a circle reaches a certain maximum size, it disappears, and then starts growing again from zero.

You can think of this behavior as being almost like a movie: we have a series of “frames” that create an animation as we play them. If we play them fast enough then the animation will look smooth, just like if we play the individual frames of a movie fast enough.

A series of circles getting larger

We’ll break this into two steps. In the first step, we’re going to get all the circles drawn on the canvas, and make sure we can add new circles whenever you click, so you see those circles too.

To do that, we need to iterate through the clicks array and draw all the circles in that array on the canvas. We’ll do that in the updateDisplay function.

We’re also going to change how we call updateDisplay. Right now, we’re calling it just once, from the window.onload function. But if we call it only once, you won’t see the circles change size (which we’ll do in the next modules), and you won’t see any new circles added to the canvas over time, as you click. So what we really need to do is call updateDisplay over and over again—this is the movie frame part of the code.

Each time we call updateDisplay, we’ll reset the canvas, and then redraw all the circles in the clicks array. That way, if you load the page, and then a few seconds later, click to add a circle, you’ll see that circle appear in the canvas the next time updateDisplay is called.

Let’s dive into the code so you can see how this is going to work.

Calling updateDisplay

The first step is to draw all the circles stored in the clicks array. We’ll do this in updateDisplay. Remember that updateDisplay is clearing out the canvas, by drawing a black background over the whole canvas. So any circles that are currently drawn in the canvas will disappear. So we’ll iterate through the clicks array, drawing all the circles stored there. If any new circles have been added since the last time we called updateDisplay, they’ll get drawn too.

We get each circle from the clicks array, and call drawCircle with that circle’s x, y coordinates, and the radius of the circle. Right now the radius is set to 100, but we’re going to change that later so the radius varies over time. Also notice that alpha is set to 1; again we’re going to change that later so it varies depending on the radius of the circle.

We’ve also added a frameRate property to the view, in the constructor function. The frame rate is how often we’re going to call updateDisplay. Remember the movie analogy? This is how many frames per second we’re showing the user: 1000/30, or about 33 frames per second.

To make this work, we have to change how we’re calling updateDisplay. Instead of just calling the function once from the window.onload function, we’re going to call it over and over again at a specific rate, defined by frameRate. We’ll do that with setInterval.

setInterval calls a function over and over again, at the rate you determine. Notice that we have to use the same trick we used before with bind to make sure that this in the body of updateDisplay is set to the view object. If we don’t use bind, in this case, this would be set to the window object, which isn’t what we want.

Follow along as we add this code:

Now get your code updated and let’s test this code! If everything’s working well, you should see a circle appear in the canvas every time you click.

Crash Test: test updateDisplay

Let’s test the code. If it all works, you should see a new circle added to the canvas each time you click.

Audience Question
I’m still not sure I understand why we need a loop. Couldn’t we just draw a new circle on the canvas each time we click? So instead of calling updateDisplay, we could call drawCircle in the click handler.

 

To get a circle drawn in the canvas each time you click, we could indeed just call drawCircle from the click handler. However, if we do that, we have no easy way to animate the circles.

Remember that when you draw on the canvas, all you’re doing is updating pixels at a certain location. It’s not like when you create an element in the page using HTML or SVG. If each circle was its own object, then we could, for example, write a separate function to go through all the circle objects in the canvas and animate them.

But the canvas doesn’t work that way. All we’re doing when we’re drawing on the canvas is manipulating pixels. So to make the circles look like they’re moving, we need to redraw them over and over. It really is just like an animated movie made out of individual frames. You’ve probably seen those old videos showing how the animation studios used to make movies from individual “cells” drawn just a little bit different for each frame of the movie, right? Well, we have to do exactly the same thing to make our circles move.

That means we’re creating a new drawing for each frame of the movie. And to create a new drawing, we have to re-draw all the circles in whatever configuration we want them to be in for that frame. By drawing each frame over and over at a fast enough rate, it will look just like a smooth animation does in a movie.

This is a standard animation technique when working with pixels. An alternative way to do this would be to create a separate SVG object for each circle and control each circle with a function that changes the size of the circle objects over time.

Take a Break

Okay, it’s time to take a breather and let everything sink in. After all, you’ve already got a lot of the code implemented for the Generative Music Box. So, before we move on and start implementing the circle animations, how about a little inspiration?

Grab your favorite snack and watch this video clip about how a famous animation studio used to make their movies:

Animate the circles

Next, we’re going to animate the circles we’re adding to the canvas each time you click. We’ll do it by modifying both the size of the circle each time we draw it, and by modifying the opacity of the circle so it fades out at it reaches its maximum size.

Getting Prepared

Each time you click on the canvas, a new circle is added. We do this by handling your click with the handleClick function, which adds the x, y position of the circle to an array, clicks. From the window.onload function, we call updateDisplay 33 times per second, which clears the canvas and then iterates through the clicks array, calling drawCircle to draw each circle.

We have the code in place to draw the circles, and now we’re going to animate them. We set up the code to call updateDisplay 33 times per second (specified by the frameRate property in the View), so we can animate the size of the circle by changing the circle’s radius each time we draw it so it looks a little bigger each time.

When the circle gets to a certain size, we’ll fade it by modifying the opacity of the circle. We can do that by changing the alpha value of the circle when we draw it with drawCircle.

Finally, when the circle reaches its maximum allowed size, we’ll stop drawing it when updateDisplay is called—that is, we’ll skip over the circle, so it will disappear. Of course, we’ll need to make it reappear at some point, so we’ll take care of that too.

Let’s step through all of this so you can see how it’s going to work.

Code to animate the circles

Our goal for this section is to animate the circles in the Musicbox application.

Here’s the full code for the View. Go ahead and get it typed in, and then make sure you understand what we’re doing at each step. Once you do, move on to the next section and give it a try.

We’re using the objects in the clicks array to keep track of the changing radius of each circle. Remember, each item in the clicks array represents the x, y coordinate of a circle we’re drawing on the canvas, and now we’re using the radius property to store the circle’s current radius.

We want each circle to grow to a maximum radius and then stop growing. We set the maximum radius in the View property maxRadius. Each time we call updateDisplay, we check to see if the circle’s radius is greater than this maximum radius and if it is, we don’t call drawCircle. If it’s still less than the maximum radius, we increase the radius size by 1.

We also check the size of the radius to set the opacity using the alpha value in the rgba color. Recall that the alpha determines how opaque or transparent a color is. Until the circle reaches the maximum radius minus 15, we set the alpha value to .7, which means it’s 70% opaque. After that point, until the circle is its maximum size, we set the alpha based on the size. Try out different techniques here to vary the opacity of the circles and see what you like!

Once we’ve set the circle’s radius and its opacity, we then call drawCircle, passing in the context, the x and y coordinates, the radius and the opacity.

Let’s give this a try!

Crash Test

Here’s a short video of our test of the code:

The circles are animating—that is, they are growing from 0 to their maximum size, and then disappearing, and you can see that towards the end of the growth, they change in opacity. So our code is working great.

But right now, the circles are only animating once. So our next step is to repeat the animation so the circles reappear and grow again, over and over.

Code

To make the circle animations repeat, we need to reset each circle’s radius back to 0 at some point after it reaches the maximum radius. There are a few options for how to do this; the way we’re going to do it is to set a timer when we first create a circle. This timer will call a function to reset the circle’s radius back to 0 after a certain period of time has passed.

Follow along as we write this code:

Here’s the code we just added; get this added to your code and then we’ll test it, and hopefully we’ll see our circles repeating the animation, growing to a maximum size, disappearing, and then repeating.

Crash Test

Here’s our test; how is yours looking? Is it working like you expected?

Try playing around with the loopRate. If you make it too short, the circles won’t animate correctly. Too long and it won’t feel right. See what timing works for you. Remember that loopRate is milliseconds so just multiply the number of seconds you want by 1000.

We have our circles animating beautifully, so the next step is to add the sound. Time for the next module!

Audience Question
Could I animate my circles by resetting the radius back to 0 in updateDisplay?

You could! For instance, you could write another for loop to run after the loop we already have to go through each circle in the clicks array, check to see which circles have a radius greater than the maximum radius, and reset the radius there.

However, that will mean there is no delay between when a circle disappears and when it starts growing again, and we would like to have a short delay. It also means that the code will be slightly less efficient since we have to loop through the clicks array twice; first to update the radius and draw the circles, and second to reset the circles that have reached maximum size.

In addition, when we get around to adding the audio to the music box, you’ll see why it’s a bit better to start the timer to reset the circles in the handleClick function. It’ll be a little bit easier to synchronize the circles and the audio if we reset the circles from a timer there.

Of course, there are many different options for how to write this code, depending on how you want the music box to behave, so explore your options and test things out! When you’re working with timers (and with audio) it can get a bit tricky to keep things straight about what’s happening when so it’s a good exercise to step through this code on paper, and any other variations you try, and make sure you understand it.

Add the audio

The circles for our Music Box project are working great and the View is done, so now it’s time to add some audio.

Game Plan

To add audio to the Music Box project, we’re going to use the AudioContext API, a built-in JavaScript API that was added to JavaScript as part of the HTML5 overhaul. All modern browsers support AudioContext, but note that this does not include IE, so you’ll need to use Edge instead.

We’ll be adding audio in four steps. The first step is to add a new object, Audio, which will contain properties and methods we need to play audio samples. Next, we’ll create a BufferLoader object. This object will load audio samples from sample files, and get them stored in an array in a format that the Audio object can use to play sounds. We’ll update the window.onload function in musicbox.js to kick things off by loading and initializing the audio.

Finally, once we’ve done all that, we can play the Audio. We want to play a sound with each circle on the canvas; to do this, we’ll need to make just a small change to the handleClick function in the View.

We’re going to go pretty quickly through the code to load, initialize, and play audio. For a more in-depth description of the AudioContext web API and using the BufferLoader to load audio data, check out this HTML5 Rocks article.

Okay, let’s jump in and get going!

Prep: Audio samples

Before you begin writing any code, make sure you have the audio samples ready to go. If you haven’t already, download the samples from github, and put them in the same folder where your code is. You should have ten MP3 samples that contain notes from the C major pentatonic scale. We chose these notes so when they’re played together in the music box, they sound good!

Prebaked Goodness: Audio.js

To play sounds in JavaScript, we need to use the built-in AudioContext constructor to create an AudioContext object. After creating the AudioContext object, we’ll initialize it with sounds, so it’s ready to play. We’ll put all the code associated with generating audio from the sounds in a new file, “Audio.js”, so go ahead and create this file now, and copy in the code below.

Once you’ve saved “Audio.js”, you’ll need to add a link to the JavaScript in the file “musicbox.html” using a <script> tag. Add it to the bottom of the file, above the link to “View.js” file, and save your “musicbox.html” file.

Let’s quickly step through the code so you understand how it works. First, notice that we’re encapsulating all the code related to audio in an object named Audio. Using an object literal like we’re doing here is a good strategy for grouping related code together, and we can use this object to access the audio-related properties and methods from other JavaScript code.

The Audio object has five properties, two of which are methods. The gainNode will be used to control the volume; the bufferList will be an array holding the sounds to play; and the audioContext is the object that knows how to decode binary audio data and make the browser play a sound.

Audio operations, like making a sound or adjusting the volume, are handled with nodes that are connected together. Audio people think of the groups of nodes as “audio routing graphs”, which is where this “node” language comes from. For this example, we’re keeping it extremely simple. We just need two nodes: one to create the sound, and one to generate the sound.

We’ll be calling the init method to initialize the Audio object with the sounds it needs to play. We’ll also use this method to set up the volume at which the sounds will be played. The function takes an array, bufferList, of sounds, which it stores in the property bufferList. Each item in this array will be one of the sounds from the audio files you downloaded earlier.

To set the volume for the audio, we create a gain node and connect it to the audio context object. The gain (the amount of volume to apply to your sound) on the gain node is set to 1, to play at maximum volume. If you wanted to play at half volume, you’d set this to 0.5.

To actually play a sound, we’ll be calling the play method. We’ll be doing this in the View object, and we’ll get to that shortly. The play method takes an index of one of the sounds in the bufferList and prepares that for playing by creating a buffer source. This source is then connected to the gain node and the sound is started. A started sound will play to completion and then end. Each time we need to play a sound, we recreate the buffer source and connect it to the gain node.

We have the code to take sounds and play them; next up, we’ll tackle loading the sounds from the MP3 files.

Prebaked Goodness: BufferLoader.js

Next up is the buffer loader. This is the code that reads the audio data (the sound samples) from the MP3 files and loads the data into the buffer list array we use in “Audio.js” to play the sounds.

We’re going to put all the code responsible for loading the sound data into a separate file, “BufferLoader.js”. Go ahead and create that file now, copy the code below, and save. You’ll then need to add a link to this file in “musicbox.html”, just like you did for “Audio.js”. Put this link above the link to “Audio.js”.

Here’s the full code for the BufferLoader:

BufferLoader is a constructor function, that takes the AudioContext as its first argument, an array of filenames as the second argument, and a callback function as the third argument. We’ll use this constructor to create a new BufferLoader object in “musicbox.js” in the next section.

The loadBuffer method is what does the bulk of the work, so let’s start there. We call loadBuffer from the load function (which we’ll call from “musicbox.js”) for each sound in the list of files. What gets passed to the loadBuffer method is the URL of the file containing the sound, e.g. “A4.mp3”, and the index of that sound in the list. We’ll use that index later to make sure we get the sounds added to the buffer list in the correct order.

We’re using XMLHttpRequest to get the data from the file. Just like a typical XMLHttpRequest request, we make the object, set a callback (in the onload property) and then call send() to kick off the request.

Notice that the responseType we’re expecting is “arraybuffer”. This is probably different from what you’re used to when working with XMLHttpRequest to get JSON or plain text. We use this response type when we’re working with binary data. In this example, we’re working with MP3 binary data, and even though the “arraybuffer” type is generic (meaning, it can be any kind of binary data), we can rely on the AudioContext to correctly recognize that binary data as MP3.

When the request response is ready, the callback function we define for the request.onload property is called. Here, we call the AudioContext‘s method decodeAudioData to decode the binary data in the file into a sound the AudioContext can use. The first argument is the binary data in the response and the second argument is a callback function that is called if the decoding is successful.

The callback is called with the sound data passed into the parameter buffer. Next, we save this sound into the bufferList array, which will contain all the sounds once the BufferLoader has completed loading the sounds from the files. Notice here that we don’t want to just push the sound onto the end of the bufferList array because the sounds might end up in the wrong order, and we want to keep them in the order we specify (when we pass them into the BufferLoader in the urlList). Remember that making a XMLHttpRequest is asynchronous so the sounds could be loaded out of order. Since we passed in the index where this sound needs to go in the bufferList we can use that to put it in the right spot.

We’re keeping track of how many sounds we’ve loaded with loadCount. Once this count is the same as the length of the original array of sounds we passed in, we know we’ve loaded all the sounds, and we can then call the callback function we stored in the onload property of the BufferLoader object, and pass in the fully loaded array of sounds, bufferList.

We’ll write this callback function in the next section, where we modify “musicbox.js” to make this BufferLoader object and specify the callback function we’ll use to kick off the music box once all the sounds are loaded.

So hang in there! Just a couple more changes and then you’ll be ready to test your music box project.

Prebaked Goodness: musicbox.js

We’ve got two new files, “BufferLoader.js” and “Audio.js” that we’ll use to load and play sounds. Now, we need to make some changes to “musicbox.js” to use the BufferLoader, and supply a callback function that will be called once the sounds are loaded. This callback function will initialize the sounds in the audio context, and then do what our code already does: set up the view, the canvas click handler, and the call to updateDisplay to get our animation started.

Open up the file “musicbox.js” and make sure you update your code with the full code below:

The first thing we’re doing in the window.onload function now is creating a new BufferLoader object. Recall that the constructor function for the BufferLoader takes three arguments: the audio context, which is a property of the Audio object, an array of URLs (which in our case are just file names) for the sound files, and a callback function to call once the sounds have been successfully loaded. Note that we’re assuming the sound files are in the same folder as your code, so the path to the sounds is the same as the path to the music box application.

The callback function we’re specifying is the finishedLoading function which is defined in the window.onload function also. We’ll go over what that does in a moment.

Once we’ve created the BufferLoader object, we call the load method. Remember, load iterates through all the file names in the array, and passes the file name, along with the index of the file name in the list, to the loadBuffer method which is responsible for loading the sounds as binary data, using XMLHttpRequest.

Once loadBuffer has successfully loaded the sounds, the callback function, finishedLoading is called (to see where we do this, look at the “BufferLoader.js” file for the call to loader.onload). What gets passed into the finishedLoading function is the bufferList array in the BufferLoader object, which contains all the decoded sounds from the MP3 files.

The finishedLoading function contains the code we had previously written to set up the view, the click handler for the canvas, and start the calls to updateDisplay to kick off the animation. In addition, we’ve added one line at the top to call the Audio.init method, passing in the bufferList array. This array of sounds gets stored in the Audio object and used when we call the Audio.play method.

Play audio with each circle

We’ve added code to include the capability to play an audio sample, so now, we’re ready to update the code in the View to do just that. All we need to do is call Audio.play and pass in an index into the bufferList array, which contains our list of audio samples.

Take a look at the handleClick function in the View (“View.js”). Remember that this is where we add a new circle to the music box, and create a timer for each new circle that resets that circle’s radius to 0 every four seconds so that the circle will continue to animate over and over again. Our goal is to play a sound that begins to play roughly in synch with when a circle begins to animate; that is, when the circle’s radius is 0. And of course, we want that sound to repeat each time the circle starts animating again.

We can do this by calling Audio.play in handleClick, just before we call setInterval. This will play a sound when a circle is first added. Then, we’ll also add a call to Audio.play in the anonymous function we’re passing to setInterval, so that the sound repeats each time the circle’s animation begins.

But which sound do we want to play for a given circle? Well, we know we have ten different samples, so that means the number we pass to Audio.play should be between 0 and 9. We could randomly choose a number in this range, but we can do better by basing the audio sample we select on the location of the circle on the canvas. You could pick x or y or some combination of the two for this; we’re going to keep it simple and use x. To get a number between 0 and 9 based on the x location of the circle, we’ll use x % 10, which is x “modulus” 10: divide x by 10 and take the remainder. This number is a number between 0 and 9.

Now you know what we’re going to do, and why we’re doing it, so it’s time to add this code to the handleClick function in your “View.js” file:

Get ready for the final test!

Crash Test

Let’s test all the code we just added. We’ve added quite a lot in this section: we added two completely new files, “Audio.js” and “BufferLoader.js”. We updated “musicbox.js” to use the Audio object, and created a BufferLoader object, using our previous code in a callback function to call once the audio is loaded and ready to go. And finally, we updated “View.js” to play an audio sample each time a circle begins animating.

Before you test, make sure you’ve got all the code added in the right places, and with the correct filenames. Also make sure you’ve got everything linked from “musicbox.html” correctly; at this point, your code in “musicbox.html” should look like this:

Okay, you’ve double checked all your code, so let’s give it a try:

Hopefully your music box is working, and you’re hearing sounds as you click to add circles. Have fun, play around with the music box. It can be quite addicting!

The Finish Line

You’ve completed The Generative Music Box project, and implemented an application using audio and canvas graphics in the browser. We hope you’re having fun playing with the application, making some ambient music and some cool visuals. Once you’re ready, take a look at some things you might do next.

Going Further

Why Stop Now? How about…

Extending your project?

We’ve really just scratched the surface of this project. You can take it a lot further. For instance, think about how you might go further with the audio. You could:

  1. Create an algorithm that only plays the notes some of the time. Perhaps each sound has a probability of playing each time it is scheduled to play.
  2. Change how sounds die over time. This would lead to the composition decaying or being forgotten over time. How would you code this?
  3. Change how sounds are played. We’re picking the sample that plays on a given click by mod’ing the x value of the mouse click by the number of sounds. That’s pretty simplistic. What other ways can you think of doing this?
  4. Create your own sound samples and see how this affects the feel of the app.

Doing more research?

Research the Web Audio API. Think about using an oscillator instead of samples or adding audio events (reverb, delay, etc.).

Or, check out HTML’s canvas element and see how far you can push the graphics.

Or, blow our minds and yours…

How about revisiting project #1, The Game of Life, and combining code? Can the Game of Life generate sound too, perhaps by assigning sounds to grid locations? You might also check out apps and devices like the Tenori-on for inspiration.

In any case, stay in touch and let us know what cool things you’ve written!

Code HTML

You can download the complete code and all the sounds for the Generative Music Box Project at github. There are lots of files so we recommend clicking the download button and getting the zip file. This will download a zipped folder with everything in the musicbox project.

Pin It on Pinterest

Give your Brain a Treat!

Don't miss out on brain-friendly updates, new WickedlySmart Projects, early access to books and general cool stuff! Just give us your email and we'll send you something about once a week. Don't worry, we'll never sell your name and you can remove yourself at any time.

Check your email to confirm your subscription.