Making waves in Unity

Durations

Last week, as final challenge, I asked to to think about how you would play multiple notes. One possible solution would have been something like this:

Tip	this just replaced the call to SineWave from last week, the rest of the code remains unchanged!

int someValue = sampleRate / 2;

if ( position < someValue ) {
	// play note 1
	data[count] = volume * SineWave( 440, position );
} else {
	// play note 2
	data[count] = volume * SineWave( 500, position );
}

What if we wanted to play three notes? Ok, well we can do that as well…

int someValue = sampleRate / 2;
int nextValue = sampleRate;

if ( position < someValue ) {
	// play note 1
	data[count] = volume * SineWave( 440, position );
} else if ( someValue <= position && position < nextValue ) {
	// play note 2
	data[count] = volume * SineWave( 500, position );
} else {
	// play note 3
	data[count] = volume * SineWave( 550, position );
}

Note	make sure the duration is longer than one second, else this will not play the third note.

Refactoring

What about 4, or 5, or 100 notes? Would you write 100 if statement cases? that would get very unwieldly fairly quicky. Instead, it would be better if we could devide time up into "chunks" (1, 2, 3, 4, etc…) then we could look the frequency assoicated with that chunk up.

our code could be something like:

// define our note sequence
float[] sequence = new float[]{ 440, 500, 550 };

// NEXT LINE IS NOT REAL CODE, IT WILL NOT RUN - KEEP READING
int index = calculateTheChunkIndex(position);

// this would actually work if the above line worked...
data[count] = SineWave( sequence[index], position );

We would remove the need for all those messy if statements all together! How would we write that function?

Calculating the Index

Because we’re intrested in dividing the time up, division seems like a good candidate. This is usually when I’d break out a spreadsheet for a more complicated problem and play with some values. For this problem, we’ll just think about the things we know, and the things we want to know.

We know:

the position in the sequnce (between 0 and length in seconds * SampleRate)
the duration we want each note to play for (in length in seconds * SampleRate)

We want:

A number corrisponding to the position in the array.

Let’s say we want each node to play for half a second (0.5 * SampleRate):

float index = position / ( 0.5f * sampleRate);

If you plot a bunch of positions in excel (or libreoffice calc) and calculate the values, you’ll see they increase (0, 1, 2, 3) but include the fractional part. We want to get rid of the fractional part (we just want the int value). This is easy to do in programming, just cast it to an int:

Let’s pretend for a second that sample rate is 6 and we want each note to play for half a second (so my table isn’t at least 14400 entries long…). The table for the first few entries would look like this:

position	position / (0.5f * sampleRate)	index (whole number part)
0	0	0
1	0.333333	0
2	0.66666	0
3	1	1
4	1.333333	1
5	1.666666	1
6	2	2

Seems to be what we’re after! Lets' put that into code (using half a second per note):

int index = (int)( position / ( 0.5f * sampleRate ) );

Note	how would you make each note last for 2 seconds? what about a quater of a second?

Refactoring the callback

Ok, now we’ve tackled the problem of figuring out how to calculate the index. Let’s also refactor our play function to make use of our new code:

void OnAudioRead(float[] data) {
	float[] sequence = new float[]{ 440, 500, 550, 440 };
	int count = 0;
	while ( count < data.Length ) {
		int index = (int)( position / ( 0.5f * sampleRate) );
		data[count] = SineWave( sequence[index], position );

		position++;
		count++;
	}
}

Warning

make sure your note list and track duration make sense (ie, you have enouph notes to cover the required duration)

Let’s experiment with that warning.

What would happen if the sequence is too short? Ie, this assumes that your duration (in seconds) is currently 2 seconds. Try setting it to 3 before the program runs. You should see a bunch of errors about reading off the end of the array:

Unity reports 'index was outside the bounds of the array'

Note	Try setting duration shorter than the length of the array (eg, 1 second rather than 2). What happens? Why does this happen?

Exposing values in the inspector

We’re doing this in Unity after all, so let’s make our sequence a public variable so it appears in the inspector:

class AudioManager : public MonoBehaviour {
    // properties from last week here...

    [SerializeField]
    private float[] sequence = new float[]{ 440, 0, 500, 0, 550, 0, 440 };

    [SerializeField]
    private float noteDuration = 0.5f;

    // functions from last week
}

You can use SerializeField if you want to make something accessible in the editor, but not public. Have a look at what other attributes are possible, and how this might help organise your component in the inspector.

Tip	it doesn’t actually matter where in the class you put the variables (properties) or functions (methods). Common convention is to put the properties at the top of the class, just after the class definition (the class MyClassName bit).

We’ll also need to update our index calculation to make use of the duration:

int index = position / ( noteDuration * sampleRate );

Working around the array length

The issue mentioned above is one that can be fixed using the mod (%) operator:

int index = (position / ( noteDuration * sampleRate ) ) % sequence.Length;

The mod (remainder) operator ($n % m$ will return the 'remainder' of $n\over{m}$ ie, 7 % 3 == 1)

Note	what happens now if we ask for a value after the length of the notes array now? What will the track do once it reaches the end of the array?

Note generation

During the last lab session, you had the opporunity to modify the frequency of the notes directly either in the inspector, in code, or both. This week we’ll be expanding our approach to make better use of storing notes that we can use to make intresting sounds.

Generating Scales

Recall that different notes result from different frequencies. What frequencies should we use to represent different notes?

The equation is on the slides, and on the website linked to in them.

\[f_n = f_0 (a)^n\]

where:

n: the note in the scale (0, 1, 2, …, 12), relative to $f_0$
a: is the base that is used for each 'step' in the scale

Recall from the lecture, it’s common to use a base of $2/^{1/12}$. We’ll use this in our code. You can experiment with different values if you’d like. The lectures give some intresting things to try out.

Let’s see that equation in code:

private float[] generateScale(float baseNote) {
	float estimator = Mathf.Pow( 2.0f, (1.0f / 12.0f ) );

	float[] octave = new float[12];
	for (int i=0; i < octave.Length; i++) {
		octave[i] = baseNote * Mathf.Pow( estimator, i );
	}

	return octave;
}

We’re storing an array that contains 12 elements, so each position relates to it’s relative location in the scale. The float array therefore represents one octave (grouping of notes in the same pitch class). The 0th element is an A.

If we want to make use of this in our script, we can do it once and cache the result (rather than, for example, calling it in our OnAudioRead function).

task Try using what you’ve learnt so far to play a full octave. If you have followed this guide until now, you should be able to do this by adding one line to 'Start'.

Playing note sequences

Playing octaves is intresting, but we can do more with this. Each note in the scale is stored relative to the base note (ie, 1 note up from A is in index 1, etc…). If we store the generated scale in a variable, we can modify our OnAudioRead function to play a sequence of notes rather than a sequence of frequencies by adding another layer of indirection:

// at the top of the code
private float[] scale;
private int[] notesToPlay;

// in Start():
scale = generateScale( frequency );
notesToPlay = new int[]{ }; // TODO populate notes here
durationInSeconds = notesToPlay.Length * noteDuration;

// in OnAudioRead(float[] data):
int scaleNote = notesToPlay[index];
if ( scaleNote != -1 ) {
	float frequency = scale[ notesToPlay[index] ];
	data[count] = SineWave( frequency, position );
} else {
	data[count] = 0.0f; // -1 == no note
}

I’m not telling you exactly where you need to add the code above in the correct functions, can you figure it out?

Ok, that’s quite a lot so far, so let’s have some fun with what we’ve built.

Set the base note (frequency) to 440 (A4).
Set the noteDuration to something a little smaller than we’ve been using (eg, 0.25 seconds)
use the following note sequence: 7, 7, 2, 3, 5, 5, 3, 2, 0, 0, -1, 3, -1, 7, 7, 5, 3, 2, 2, 3, 5, 5, 7, 7, 3, 3, 0, -1, 0

Note	this ins’t perfect, I’ve played around with the timing and used a simplified set of notes to make this work, but it should at least be recognisable.

Making Waves

We’ve made use of sine waves so far. We have other options, recall that the audio toy also supports square and sawtooth waves. Triangle waves are also a good option for classic chiptune effects.

Square waves

Let’s recall what a square wave looks like in math notation (from the lecture):

\[s \leftarrow \begin{cases} a, \textrm{if} \sin(2 \pi t f) \ge 0,\\ -a, otherwise \end{cases}\]

The curley bracket can be seen as an `if' or a choice. There are two possible branches. The equation from the top branch is the sine wave equation from last week. Let’s try to think about the function in English before converting it into code:

the value is equal to:

a, if the sine wave function returns a positive value (or zero)
-a, if the sine wave function returns a negative value

A is simply the amplitude we want. In this case, we’ll use full volume (1.0 or -1.0).

Tip	we could make the volume adjustable in the inspector, what value would we use for a in that case?

private float SquareWave(float frequency, int position) {
	float value = 0.0f; // TODO replace with sine logic from last week
	if ( value > 0 ) {
		return 1.0f;
	} else {
		return -1.0f;
	}
}

task: implement the missing sine wave logic.

How did you complete the above task?

There are two ways you could integrate the sine logic from last week. Firstly, you could simply copy and paste the body of the sine function into the line above. There is a better way - we could reuse the function we wrote last week by calling it. If you implemented using copy and paste, implement it by calling the function directly.

task: In the lecture, you were given equations for the triangle and sawtooth waves. Try implementing these in your pair, based on what you learnt last week and what we just did.

Implement the following kinds of waves:

Sawtooth
Triangle

Tip	if you are struggling with the notation, reason about it in english/words first before converting it to code.

To make the tune I provided above sound best, use a triangle wave (implement sawtooth first).

Some Math Tips:

$\left| a \right|$ is the absolute value of a (ie, make a be positive)
$\lfloor a \rfloor$ means the 'floor' of a (ie, round down)
$\lceil a \rceil$ means the 'ceiling' of a (ie, round up)
$\pi$ is PI (3.1415…).
In C#: int/int = int division, if you want float division make sure one of the values is a float

Have a look at the Unity’s MathF class to help you.

Additional Techniques

There are a range of suggested techniques to implement beyond tone generation. These are available under the 'Programming' tab above.

Among others, these include:

Tone combination
Scaling Amplitude
Scaling Wavelength
Phase Inversion
Reverse
White Noise
Audio Splicing

UI Cleanup

Currently, our editor UI looks a little cluttered. Unity gives you Attributes to help you display your variables in the editor nicely. You could explore this to make it a little easier to understand. You could also comment your code using the XMLDoc format.

Let the user pick what wave function (sine wave, square wave, sawtooth, etc…) to use from the editor
You can either use an enum with a bunch of if statements in your play callback (easy)
Try defining your own delegate, and swapping them as needed (advanced)

Loading From Disk

Want to load audio tracks from disk? Follow the Loading audio from disk lab.