by Adam Rokhsar
Features live video processing, pitch tracking, bitcrushing and downsampling audio signal as well as FFT data, and glitch work.
by Adam Rokhsar
Features live video processing, pitch tracking, bitcrushing and downsampling audio signal as well as FFT data, and glitch work.
video and processing by Adam Rokhsar
This piece uses pitch detection to control granular synthesis parameters, and Fast Fourier Transform to extract amplitude data, which is downsampled and “bit-crushed” before the voice is re-synthesized.
The video effect is based on edge detection, using the light from the monitor to create a kind of feedback loop, and motion detection.
The Recovery is: PJ Brindisi, Miguel Padro, Adam Rokhsar, Jamil Zaki
mew
by Adam Rokhsar
watch it performed April 3rd at Penn State.
Mew is working mostly in realtime, using computer vision algorithms to track my face, which is processed separately from the rest of the image. Additionally motion tracking data is mapped to a hidden 3D mesh covering the screen, so that when I move the parts of screen that contain the most light appear to pop out along the z-axis.
I analyze the audio signal and extract its brightness level, which is then used to control video processing and cut timing. Some of the visual debris or “glitches” were made using a hex editor and changing the file by hand. In the future I plan on implementing algorithms to do this for me.
The song was made in Max/MSP, Reason, and Protools.
This is a quick demo of some preliminary tests of using face and motion tracking. Here the music controls scaling along the z-axis, and my face is being detected, removed, altered, and replaced back into the original video. Feedback and the operation joining the processed face with the original video are mostly responsible for effects. Last part is the best, I think.