Gesture DJ

Gesture DJ is a multi-modal DJ controller built on a Raspberry Pi that lets you control music through gesture sensors, capacitive touch, voice commands, and camera-based hand tracking—paired with real-time visualizations on a web interface and a retro PiTFT display. (Team project with Eva Huang and Zoe Tseng.)

Inputs include an APDS-9960 gesture sensor for track and volume control, an MPR121 touch panel for direct track selection and playback, offline voice commands via Vosk, and MediaPipe hand-gesture recognition for switching UI themes and triggering DJ scratch effects. A Flask + WebSocket server streams the camera feed and beat-reactive audio visualizations (waveform, spectrum, audience, and particle modes).

I focused on the sensor interaction layer and core logic—building the APDS gesture and MPR121 touch interfaces, the offline voice control, and the shared event-handling system that lets all input methods run simultaneously without conflicts. A major challenge was integrating MediaPipe alongside the other sensors, which required carefully designing distinct responsibilities for each input across several rounds of testing.