Make Yourself Transparent

by Adam Rokhsar

materials: footage of MRI, visualization of audio waveform

software: Max/MSP/Jitter

by Adam Rokhsar

You are watching footage of Barack Obama after he was elected president in 2008. The movie is drawn out of points instead of pixels, and each point is colored the same as in the original footage. The height of the point — how much it is above the flat surface of the movie — depends on how much the original pixel is moving (optical flow). The more it moves, the greater the height.

The audio you hear is made directly from the amount of motion in one column of the video. The red scan line shows which column is being red, and that data is scaled and used as the amplitude in an inverse Fast Fourier Transform (iFFT). The processing occurs in real-time on movie file or camera input.

by Adam Rokhsar

You are watching a family home movie. The movie is drawn out of points instead of pixels, and each point is colored the same as in the original footage. The height of the point — how much it is above the flat surface of the movie — depends on how much the original pixel is moving (optical flow). The more it moves, the greater the height.

The audio you hear is made directly from the amount of motion in one column of the video. The red scan line shows which column is being red, and that data is scaled and used as the amplitude in an inverse Fast Fourier Transform (iFFT).

The processing occurs in real-time on movie file or camera input.

by Adam Rokhsar

by Adam Rokhsar

by Adam Rokhsar

by Adam Rokhsar

Adam Rokhsar
2009
materials: TV, MacBook
software: ffmpegX, avidemux2, Max/MSP/Jitter

A movie file incorrectly converted by ffmpegX and captured from avidmux2 is the basis for this experiment in video and audio feedback.

Adam Rokhsar, 2009

materials: computer, Canon PowerShot A630

software: Max/MSP/Jitter, ffmpegx, avidemux2

I took pictures of myself using my Canon digital camera using a very slow shutter speed. The pictures were processed with OpenGL and converted in Max to a movie, which I then hacked with avidemux2 to add static and the compression error effects.

I recorded myself speaking a short text and loaded the audio into Max. By extracting the amount of motion from the video using frame differencing, I could map that number inversely to the sampling rate of the audio, so that my voice could only be heard in fragments of visual motion.