Mouthpiece

Two phases here. Phase 0 was just filling in schema columns that existed but were never populated at ingest — lemma via WordNet, plus f0_var and a rough spectral-tilt voice-quality proxy alongside the existing f0_mean/RMS, and exposing all of it through the search/candidates APIs.

Phase 1 is the actual feature: POST /api/render/{id}/auto_pitch shifts every word’s pitch toward a target center Hz while preserving a scaled version of the render’s own intonation contour — so it’s not flattening everything to one pitch, just recentering the performance. New “pitch match” panel with center-pitch/intonation-range/speaking-rate controls, and the candidates panel’s pitch-sort now uses that target when the panel’s open instead of sorting against nothing in particular.

Added a /api/render/{id}/reaper endpoint that hands back a ready-to-run Lua ReaScript — drop it into Reaper and it imports the assembled WAV onto a new track, then adds a named timeline region per word (token + speaker), color-coded by speaker from a small muted palette. Way more useful than staring at a flat WAV when you’re trying to see which speaker said what, where.

The annoying part was compute_output_times() — had to re-derive, server-side in Python, the exact same per-word position math the client JS already does (crossfade padding, speed scaling, gaps for missing words) so the regions land in the right place. Two implementations of the same timing logic in two languages isn’t great, but there wasn’t a clean way to share it between a browser and a Lua export.

Three things landed together, all kind of in service of “make word selection actually good”:

Speaking rate was normalized by character count, which is bad for short function words like “the” or “we” — added a proper phonemes.py (CMUdict lookup, syllable-count fallback) and switched normalizeSpeed() to phoneme-count-per-duration instead.

Also started pulling real per-word F0 (librosa.yin) and RMS at ingest time instead of guessing, and wired a “match pitch” sort so the candidate browser can re-order clips by how close their pitch is to the words next to them — which matters a lot for concatenative synthesis actually sounding coherent instead of like a ransom note.

And: keyboard shortcuts, finally. Space to play/pause, arrows to navigate word tiles, up/down to nudge speed, shift+up/down for pitch in semitones, L/R/N for lock/reroll/normalize, and proper undo/redo (markDirty() snapshots state before a drag, normalize, or reset). Doing all of this by mouse-only was miserable.

The idea: type text, get back a WAV of real people actually saying it — cut word by word from a corpus of real speech, not synthesized. It’s for an album, and the “real clip only, always attributable” part is the actual point, not a nice-to-have. If a word isn’t in the corpus, it should fail soft (skip with a gap, not fabricate a TTS word).

First pass has the full shape already: WhisperX for ingest/transcription, SQLite FTS5 for word search, assembly with crossfades + RMS normalization, per-word speed/pitch stretching via rubberband, and a candidate browser in a FastAPI + vanilla-JS UI. No affect/emotion search yet (that’s supposed to work by biasing which real clip gets picked, not DSP faking it) — right now it’s just “find the word, splice it in.”