POUNDING
← Mobile MIDI
Pounding Systems
Documentation

Face and hand tracking

The Tracking tab uses the phone's front camera to turn your face and hands into expressive MIDI controllers.

Face tracking (iOS only, TrueDepth required)

Face tracking uses Apple's ARKit framework and the TrueDepth camera found on iPhone X and newer Pro models.

Setup

  1. Open Tracking → select the Face mode chip.
  2. The support check runs automatically.
  3. Tap START.
  4. Grant the Camera permission if iOS prompts.
  5. A live face mesh draws on screen.

What gets tracked

Mobile MIDI reads Apple's 52 facial blendshapes — each a 0–1 value describing how much of a particular expression you're making. Examples:

  • mouthSmileLeft / mouthSmileRight
  • jawOpen
  • eyeBlinkLeft / eyeBlinkRight
  • browInnerUp
  • tongueOut

Each blendshape can be mapped to any MIDI CC, note, or pitch-bend.

Full-screen visualizer

Tap the expand icon to enter a full-screen view of the face mesh on a black background. Great for audience-facing screens at a gig — and for the social media clips.

Hand tracking

Hand tracking uses on-device ML to find a hand in the camera feed and estimate the position of each finger joint.

Setup

  1. Tracking → Hand chip → START.
  2. Hold your hand in front of the camera, palm toward the phone.
  3. A hand skeleton overlays in real time.

Gestures

Mobile MIDI detects hand gestures (pinch, fist, open palm, thumbs up, etc.) and exposes each as a mappable source. You can also map raw joint positions if you want continuous control from finger motion.

Full-screen visualizer

Same as face tracking — tap the expand icon for a minimalist black-background view.

Tips for good tracking

  • Lighting — good, even light on your face or hand. Backlighting (bright window behind you) makes detection unreliable.
  • Distance — 30–60 cm from the camera is ideal for face; arm's length works best for hand.
  • Background — a plain background helps hand tracking differentiate your hand from the scene.
  • Stability — prop the phone on a stand or put it on a music stand. Handheld tracking works but adds noise.

Device support

  • Face tracking: iPhone X, XS, XR, 11–17 (Pro and non-Pro), iPad Pro (2018+).
  • Hand tracking: any iPhone with a front camera.
  • If your device doesn't support a feature, the tab will say so rather than crash or mislead.

Permissions

If you deny camera access the feature is disabled. Re-grant via iOS Settings → Privacy & Security → Camera → Mobile MIDI.