Audio Processing

Digtal Audio

Just as digital photography captures and quantizes light and color, digital audio recording may capture sound volume and frequencies. Once these characteristics are represented as values and stored numerically, they may be manipulated. Just as mathematical operations can alter the colors or brightness in an image, they may alter the frequencies and amplitudes of stored sound waves.

However, the techniques used in audio processing are more difficult than those in image processing, because timing is an important factor. You will spend some time doing some simple manipulation of audio files in Processing.

Download the audio processing starter code and three sample audio files (compressed into a single .zip) and execute the sketchbook.

Installing Processing Libraries

NOTE: Before running the AudioProcessing.pde sketchbook, you will need to install the Minim software library, which provides the additional code needed for processing audio effects.
  1. Open the AudioProcessing.pde sketchbook in Processing.
  2. Under the Sketch menu, select Import Library... and choose Add Library...
  3. Scroll down the list to find Minim and select it.
  4. Click on the Install button.
All future projects that require the Minim library will now be able to use this library.
Note: This is a different audio library than the one previously used for Audio Generation.

Click to Download: AudioProcessing

Examine the code to see how it functions. In particular, note the following code segment:

class MyEffect implements AudioEffect
{
  void process(float[] samp)
  {
    float[] newSamp = samp.clone(); 
    int j = 0; 
    while (j < newSamp.length)
    {
      newSamp[j] = samp[j];         
      j = j + 1;
    }
    oldSamp = samp.clone();         
    arrayCopy(newSamp, samp);
}

This is where the majority of the work takes place. The process() function reads into memory an array of the samples and processes each of them one at a time.

In the default code, no post-processing is done. Each sample is simply assigned the unaltered sample: newSamp[j] = samp[j];. Changing this one line can result in a variety of effects.

Instructions

Perform the following tasks, and note how the audio is altered as a result.

Instead of assigning newSamp[j] to be unaltered samp[j], assign it:

  • samp[j] × 2
  • samp[j] ÷ 2
  • samp[j] − itself
  • samp[j] + 1.0
  • samp[j] + random(0.1)
  • random(1.0)

Note that you are operating on each sample. This is akin to operating on each pixel in an image. You can also operate on the order of samples as well.

  • Reverse each batch of samples: assign newSamp[j] to be samp[(BUFFERSIZE - 1) - j].
  • The starter code keeps a copy of the previous batch samples as well as the latest. These are named oldSamp and samp, respectively. Assign newSamp[j] the average of samp[j] and oldSamp[j].
  • The audio is stored and processed in batches of 4,096 samples. This can be altered by changing the BUFFERSIZE variable at the beginning of the program code. After altering the variable to a higher value (e.g., 4 * 4096), re-execute the previous two tasks. What is the effect?

Note that some of the effects will be more pronounced on different types of audio, so you should try each of the tasks above on each of the three audio files.

Submission

Submit a reflection on the assignment. Include each of the following items:

  • A description of what each of the nine bulleted tasks does,
  • an explanation of how each task works, and
  • an explanation of how mathematics affects the wave.