BhajanApp — Chord Sheets, ML Pipeline & Desktop Trainer
Three products around bhajans: a community web base of chord sheets with clickable chords and a dictionary of meanings; a Python ML service turning any recording into a chord sheet (Demucs, key-aware Viterbi chords, Whisper lyrics, section detection); and a local Tauri desktop trainer with a Synthesia-style player.
Bhajans and kirtans are a living musical tradition learned by ear. Lyrics in Sanskrit and Bengali, chords passed hand to hand, and "sheet music" existing at best as someone's phone recording. Ordinary music apps aren't built for this: they don't need word-by-word translations, the meaning of words, or a harmonium part.
"A shared base of chord sheets the community edits itself, and a tool that turns any recording into a draft of such a sheet. AI gives the draft — a person quickly finishes it."
So the project became three linked products in one repository: the Bhajan Sangam web base, the audio-service ML pipeline and the local Mac app Bhajan Desktop. 160 commits since July 2025.
[ SYS_IMAGE_LOAD ] System map: audio → stems → chords → sheet; web and desktop
02: Bhajan Sangam — the Web Base
A library with search and author filters (35 bhajans, 15 authors — Bhaktivinoda Thakur, Narottama dasa, Vishvanatha Chakravarti…), favourites, sharing. Each bhajan is IAST-transliterated text with chords above the words, a "Translation" tab, an audio snippet and a lecture.
Chords are clickable: a diagram pops up for the chosen instrument — guitar, ukulele or keys (a keyboard diagram from notes for harmonium and piano). Transposition by any number of semitones recalculates the whole sheet.
Every word in the text is clickable too: the dictionary returns transliteration, Russian and English translation and the spiritual meaning — why "Nanda-nandana" means more than "son of Nanda". The dictionary grows through AI translation with a confidence rating and is edited in the admin panel.
[ SYS_IMAGE_LOAD ] Chord diagram, word card, settings
The app is a PWA with a service worker and builds via Capacitor for iOS and Android. Admin: import from bhajanamrita.com (Latin + Cyrillic merged into one record), OCR of sheet music, MIDI upload, a per-syllable step editor for the keyboard lesson player.
03: audio-service — from Recording to Chord Sheet
Python + FastAPI: an mp3 or a YouTube link → stems, MIDI, chords, timed lyrics, sections. The pipeline tolerates missing heavy dependencies: a "light" tier on librosa and a full one with Demucs and MuScriptor.
The project's main lesson: don't multiply inaccuracies. Four ML steps used to stack into noise ("178 chord changes in 8 minutes"). Now chords are computed the way a musician writes them: key estimation, beat tracking, one chord per bar, a diatonic bias against maj↔min flicker, and Viterbi with a change penalty.
# chords.py — one chord per bar, key-aware, Viterbi with a change penalty
tonic, is_minor = _estimate_key(chroma)
in_key = _diatonic_labels(tonic, is_minor)
bonus = np.array([0.12 if labels[k] in in_key else 0.0 for k in range(len(labels))])
_tempo, beats = librosa.beat.beat_track(y=y, sr=sr, hop_length=hop)
bounds = sorted({0, *beat_list[::beats_per_bar], chroma.shape[1]}) # bars, not beats
seg_chroma = [np.median(chroma[:, a:b], axis=1) for a, b in zip(bounds, bounds[1:])]
scores = templates @ normalize(np.stack(seg_chroma, axis=1)) + bonus[:, None]
path = _viterbi_path(scores, settings.chord_change_penalty) # hold the chord until there's a reason to change
def _viterbi_path(scores, change_penalty):
"""'Stay' is free, 'switch' costs change_penalty — O(K·T) via the best previous state."""
K, T = scores.shape
dp = np.full((K, T), -1e18); back = np.zeros((K, T), dtype=int); dp[:, 0] = scores[:, 0]
for t in range(1, T):
prev = dp[:, t - 1]; best = int(prev.argmax()); switch = prev[best] - change_penalty
stay = prev >= switch
dp[:, t] = np.where(stay, prev, switch) + scores[:, t]
back[:, t] = np.where(stay, np.arange(K), best)
...
Lyrics are timed with faster-whisper word timestamps (output romanised across scripts); vocal MIDI comes from MuScriptor or pyin with a median filter, key snapping and beat quantisation.
04: Sections and Learning Without Repeats
Verse and chorus are found by self-similarity: beat-synchronous CQT, a recurrence matrix, Laplacian segmentation and spectral clustering — repeats of one section land next to each other in eigenvector coordinates. Each section carries a group — a self-similarity group id shared by repeats. The trainer builds on it: the route plays only unique sections, not the whole song with its repeats.
# sections.py — computed per beat so a repeated verse matches itself even at a different tempo
cqt_sync = librosa.util.sync(cqt, beats, aggregate=np.median)
rec = librosa.segment.recurrence_matrix(cqt_sync, width=3, mode="affinity", sym=True)
rec = librosa.segment.timelag_filter(median_filter)(rec, size=(1, 7))
affinity = mu * rec + (1 - mu) * rec_path # long-range similarity + local continuity
lap = scipy.sparse.csgraph.laplacian(affinity, normed=True)
_evals, evecs = scipy.linalg.eigh(lap)
seg_ids = KMeans(n_clusters=k, n_init=10, random_state=0).fit_predict(evecs[:, :k] / norms)
05: Bhajan Desktop — a Local Mac App
A personal practice tool: everything runs on your machine, models live on disk, nothing leaves. Tauri v2 (a ~10 MB Rust shell) spawns the same FastAPI as a sidecar on 127.0.0.1. Inside: mp3/YouTube import, a waveform track with a section-boundary editor, a Synthesia-style player with falling notes on a virtual keyboard, a piano-roll over the recognised MIDI with note playback and .mid export. Bundled as a .dmg, launched with a double-click.
06: Stack
Web: Next.js, React, TypeScript, Tailwind, shadcn/ui, React Query, Prisma + PostgreSQL, a GraphQL proxy to the bhajan base, PWA, Capacitor (iOS/Android), Vercel
ML service: Python, FastAPI, librosa, Demucs, faster-whisper, MuScriptor, mido; light and full tiers; Docker
Three tools around one idea: the community reads and edits chord sheets on the web, the ML service turns any recording into a draft sheet, and the desktop app teaches you to play by sections — without repeats and without the cloud.
