PRACTICAL GUIDE
How AI audio-to-MIDI transcription works
What makes an AI model different from a regular tuner, and its real limits.
On this page5 sections
What makes this different from a tuner
A tuner estimates the pitch of a sounding note. The transcriber also looks for starts and ends to propose a note sequence. HertzKit uses Spotify’s Basic Pitch, which supports simultaneous notes from one instrument; this does not mean it separates or identifies each instrument in a mix. A detected note remains an estimate, not confirmation of what was played.
Why it runs in your browser instead of a server
The model is under 1 MB, in addition to the processing library, and runs with TensorFlow.js on your device. Audio is not uploaded for transcription. The page, library and model must be downloaded before analysis; the first load requires a connection. This tool is not included in the offline package, and opening it once does not guarantee later transcription without internet.
What it's good at, and what it isn't
Start with an isolated voice or instrument and a 10–30 second excerpt you can check by ear. Chords, reverb, weak attacks and noise can produce missing, extra or octave-shifted notes. It does not recognize lyrics, instruments or unpitched percussion. Basic Pitch’s polyphonic support does not turn a full song into separate tracks.
HertzKit exports fixed-pitch notes, durations and velocities. It does not export the model’s pitch-bend curves or detect tempo, meter or key. The starting 120 BPM and 4/4 are editable references. Pressing Apply tempo and meter preserves audio note timing, subject to tick rounding; it does not stretch the audio or correct the performance.
What to do with the result
After transcription, the result panel brings together MIDI playback, a plot of all detected notes and download. The original has its own controls; starting either playback stops the other. Position only moves playback and does not trim the download. If you change tempo or meter, MIDI playback and download remain disabled until you apply those settings.
Transcription distinguishes audio preparation, note detection and MIDI preparation. The percentage refers to detection when reported by the model; other stages do not show an invented remaining time. Cancel is beside progress. Detection settings let you change sensitivity, discard notes shorter than a minimum or restrict pitches. They apply when you transcribe again; the result identifies the settings that produced it. Conservative can miss quiet notes and Sensitive can add extra ones. Filters can also discard correct notes; Reset detection restores Balanced with no additional filters. These controls do not separate instruments or extract drums.
Download the MIDI and listen in the MIDI viewer. The transposer shifts notes by semitones; MIDI workshop inspects events, applies repairs or quantization, and lets you edit each note’s pitch, velocity, start and end. Prepare the version and compare it with Original before downloading. Use your DAW to add or delete notes and continue arranging. Then prepare notation with MIDI to score and review MusicXML in a notation editor. None of these steps certifies the transcription.