PA 1 Part 2: Building the Tone Matrix grid

In Part 1, we built a single plucked string that can play one note. Now we’ll arrange a whole grid of them into a Tone Matrix instrument, a grid of lights you click to compose a looping melody.

With this, we’ll see how the Tone Matrix works: each light in the grid stands for a note at a specific point in time. Lights further to the left play earlier; lights further to the right play later. A light’s row sets its pitch: lights near the bottom of the grid are lower notes, lights near the top are higher. A lighter column sweeps across the grid from left to right, and any lit cell it passes over plays its note. When the sweep reaches the right edge, it wraps back around to the start, so whatever pattern you’ve clicked in keeps looping. If you haven’t already, take a look at the demo video on the assignment overview page to see this in action before we dig into the code.

Overview: representing the Tone Matrix¶

[Task] Logical grid and audio sample grid¶

To build the Tone Matrix, we need two 2D arrays that stay in sync with each other, cell for cell:

• lightGrid, a boolean 2D array: lightGrid[row][col] is true if that cell is lit (on), false if it’s off. This is the array mousePressed() and keyPressed() change.

• audioSampleGrid, a 2D array holding AudioSample objects: audioSampleGrid[row][col] holds the actual sound to play for that cell, as an AudioSample. This is Processing’s class for a chunk of sound you can play() on demand, and we build one for every cell using the StringInstrument you wrote in Part 1.

Every column in a row shares the same pitch: you’ll get that frequency from a helper method we provided called frequencyForRow(int row), which maps each row to a note in a pentatonic scale.^[1] You don’t need to understand its internals, just that calling it with a row number returns a frequency in Hz, ready to pass into the StringInstrument() constructor.

To get started, let’s declare the 2D array instance variables representing the grids: lightGrid and audioSampleGrid as indicated by the TODOs:

// TODO declare lightGrid as a 2D array of boolean values

// TODO declare audioSampleGrid as a 2D array of AudioSample objects

Understanding Processing methods and variables¶

It helps to know the structure of a Processing sketch as you complete the assignment. The setup() method runs exactly once, when the window first opens. The draw() method then runs on a loop, about 60 times a second, redrawing the whole window each time. This is what makes the sweeping column and the blinking lights possible: draw() is constantly checking “where’s the sweep now?” and redrawing the grid accordingly. Alongside those two, Processing calls a few special methods automatically whenever an event happens: mousePressed() is run when a user clicks, and keyPressed() is run when a user presses a key.

PApplet variables¶

ToneMatrix extends PApplet, Processing’s base sketch class, which means we get some instance variables already defined for us from PApplet that describe the current input state:

• mouseX: the x-coordinate of the mouse at the time of the click.

• mouseY: the y-coordinate of the mouse at the time of the click.

• key: the key that was pressed.

You’ll use these inside mousePressed() and keyPressed().

[Task] setup() method¶

setup() runs once when the sketch starts, so it’s the right place to do anything that only needs to happen a single time. For this sketch, we want setup() to initialize the lightGrid and audioSampleGrid 2D arrays according to GRID_SIZE, and fill the audioSampleGrid with AudioSample objects for each cell in the grid. Looking at the starter code:

public void setup() {
    // TODO initialize the lightGrid as a 2D array of boolean values
    
    // TODO initialize the audioSampleGrid as a 2D array of AudioSample objects
    
    
    // TODO fill the audioSampleGrid with AudioSample objects

        // TODO create a new StringInstrument with frequencyForRow for the cell using its row index
        
        // TODO create a soundArray by calling generateSoundArray with NOTE_SECONDS

        // TODO create a new AudioSample with the soundArray
}

Your job is to complete the TODOs. For filling the audioSampleGrid, loop over every row and column in the grid and fill in audioSampleGrid[row][col] with the appropriate AudioSample object. Building one AudioSample entails the following:

1. Get the frequency for the row using the provided frequencyForRow(row) method.

2. Create a new StringInstrument with that frequency.

3. Generate the sound array for the note using generateSoundArray() you implemented in Part 1, passing in NOTE_SECONDS.

4. Create a new AudioSample with the sound array.

The syntax for creating a single new AudioSample object is as follows:

// suppose the sound array has already been created via generateSoundArray()
AudioSample sample = new AudioSample(this, soundArray);

AudioSample’s constructor takes the Processing sketch itself (this, since ToneMatrix extends PApplet) and that array, and wraps it up into something Processing knows how to play.

x and y coordinates vs row and column indices¶

The Tone Matrix grid is GRID_SIZE cells wide and tall, and the window is WINDOW_SIZE pixels wide and tall. That means every cell is a square with side length cellSize, in pixels:
 

We’ll need to convert back and forth between two different ways of talking about a cell:

• pixel coordinates: mouseX, mouseY, which is where the user clicked

• grid indices: row, col, which cell that is in our 2D arrays.

One way to picture this is to imagine a ruler marked off in 30-pixel steps over the window; whichever mark a pixel falls between tells you the row or column.

Going from pixels to indices:

col = mouseX / cellSize
row = mouseY / cellSize

Since cellSize isn’t one of the variables provided in the starter code, feel free to compute it where you need it throughout the assignment.

[Task] draw() method¶

Here is the starter code for the draw() method:

public void draw() {
    // Sets the background color to dark grey
    background(20);

    // calculate which column is "active" based on the current time
    int activeColumn = (int) ((millis() / (SWEEP_SECONDS * 1000.0f)) * GRID_SIZE) % GRID_SIZE;

    // TODO draw the grid, highlighting the active column and playing the notes
    for (int row = 0; row < GRID_SIZE; row++) {
        for (int col = 0; col < GRID_SIZE; col++) {
                // TODO your fill color and sound playing logic


		

                // TODO draw the cell -- make sure to keep this 
                // code section here at the end of the loop, 
                // outside of any if statements above

        }
    }
}

background(20) is already there for you: it’s a Processing call that fills the whole window with a greyscale color, where 20 (out of 255) gives us a dark grey background, resetting the background on every frame.^[2]

You’ll need to use two additional Processing methods in your draw() method:

• fill(int rgb): sets the color of the next shape you draw. It is passed a single number from 0-255 that is the intensity of the color (0 is black, 255 is white), similar to the background(20) you just saw.

• rect(int x, int y, int width, int height): draws a filled rectangle, using the current fill() color, with its top-left corner at pixel (x, y). Since the grid is square, the width and height are the same cellSize you computed earlier.

The starter code already computes which column is “active” right now:

int activeColumn = (int) ((millis() / (SWEEP_SECONDS * 1000.0f)) * GRID_SIZE) % GRID_SIZE;

millis() is another PApplet method: it returns how many milliseconds have elapsed since the sketch started. This formula turns that running clock into a column index that sweeps from 0 to GRID_SIZE - 1 and wraps around back to 0, taking SWEEP_SECONDS seconds for one full pass. You don’t need to modify this line, just use the activeColumn variable it produces.

We’ll use the following color scheme to determine the fill of each cell:

• unlit and non-active cells should be filled in as dark grey with a brightness of 50

• cells that are unlit but in the active column should be filled in lighter with a brightness of 90

• cells that are lit should be the brightest, filled in with a brightness of 200

Your job is to complete the rest of draw(). Loop over every row and col in the grid, and for each cell, do the following:

1. Decide the fill color according to the color scheme above.

2. If the cell is lit and its column is the active one, this is the instant its note should play. Play audioSampleGrid[row][col], but only if it isn’t already playing.

3. Draw the cell as a rect(), converting row and col to pixel coordinates using the formulas we’ve seen above.

See the figure below for an example of the color scheme and logic for draw() in action:

[Task] Drawing an example grid¶

Once you’ve verified your logic with your drawn grid, implement the logic in your draw() method. For triggering the sound itself, an AudioSample object inside each cell of the audioSampleGrid has two methods you’ll need:

• isPlaying(): returns true if this sample is currently playing. Check this with an if statement before you call play(). Without the check, draw() would call play() on the same note repeatedly while the sweep sits on that column, which restarts the note from the beginning every frame instead of letting it ring and fade.

• play(): starts (or restarts) playback from the beginning of the sample.

For example, to play a note from an AudioSample object called sample, you would use the following code:

// Suppose sample is an AudioSample object
if (!sample.isPlaying()) {
    sample.play();
}

[Task] mousePressed() method¶

When the user clicks somewhere in the window, mousePressed() is called once with mouseX and mouseY already set to the click location. Your job is to convert that pixel position into a row and col using the formulas from the Overview section, then flip that cell’s state in lightGrid (on becomes off, off becomes on).

[Task] keyPressed() method¶

keyPressed() is called whenever a key goes down, with key set to whatever was pressed. We only care about one key here: when key equals ' ' (the space bar), clear the whole grid by setting every cell back to false. You don’t have to add any other functionality to handle other keys.

Run and explore your Tone Matrix¶

Because Part 2 is all about a visual, audible application, we won’t write a testCode() method for this part the way we did in Part 1. Instead, testing your implementation means actually running it and watching (and listening to) what happens.

Once you’ve completed the functionality described above, run the sketch by clicking the play button in VSCodium. A window should appear with a 16 by 16 grid of dark squares and a lighter column sweeping across it. Verify the following:

• Clicking a cell toggles it between on (bright) and off (dark), and clicking it again toggles it back.

• Watching the window for a few seconds, the sweep column moves steadily left to right and wraps back around to the start.

• When the sweep reaches a lit cell, you hear a note; different rows sound noticeably different pitches.

• Pressing the space bar clears every lit cell back to off.

Reflection¶

Finally, answer the following reflection questions in a file named README.md and submit it to Gradescope:

1. Approximately how much time did you spend on the final PA 1 submission?

2. What were the most challenging portion(s) of the assignment?

3. What is one thing you have a better understanding of after completing this assignment and going through the class content?

4. Do you have any follow-up questions about concepts that you’d like to explore further?

5. Indicate the number of late days (if any) you are using for this assignment.

Part 2 task checklist¶

Going further¶

If you want to keep going after completing the assignment, here are a few directions to explore. None of these are required!

• Add a mouseDragged() method so you can click and drag across cells to paint or erase a whole run at once, instead of clicking one at a time. It uses the same pixel-to-index conversion as mousePressed(), called continuously while the mouse moves with the button held down. See the mouseDragged() reference.

• Change SWEEP_SECONDS and see how the tempo of the loop changes. Could you make the tempo controllable while the sketch is running, say, by a key press that speeds up or slows down the sweep?

• Give lit cells a color instead of greyscale, perhaps based on their row, so pitch is visible as well as audible. You can take a look at the fill() documentation to see how to set the color of a shape.