Sub-align

GitHub

Align .srt / .lrc / .txt subtitles (or generate them) to audio/video with WhisperX forced alignment — so each cue can move independently instead of only applying one global timeline shift.

Why not only a global offset?

Tools like ffsubsync typically find a constant offset (or stretch) between speech activity and subtitle “on” times. That works well for whole-track drift, but leading/trailing silence or local timing errors can still leave lines early or late.

sub-align picks a strategy from the input type, then runs WhisperX phoneme / word-level forced alignment so each line is refined against the audio:

Limitation: subtitle text must roughly match spoken content. Alignment does not translate or correct wrong words.

More detail: docs/pipeline.md · scenarios & flags: docs/usage.md

Install

Requires Python 3.10+ and ffmpeg on PATH. First run downloads WhisperX alignment models (disk/RAM).

1
2
3

pip install 'sub-align[align]'
# or
uv pip install 'sub-align[align]'

Extras [align], [cpu], and [gpu] all install WhisperX. Install a matching PyTorch build first when you need a specific CPU/CUDA wheel:

1
2
3
4
5
6
7

# CPU
uv pip install torch --index-url https://download.pytorch.org/whl/cpu
uv pip install 'sub-align[cpu]'

# CUDA (example: cu124)
uv pip install torch --index-url https://download.pytorch.org/whl/cu124
uv pip install 'sub-align[gpu]'

Development

1
2
3
4
5

uv venv
uv sync --group dev                 # unit tests / lint (no WhisperX)
uv sync --group dev --extra align   # full local alignment
uv run pytest
uv run ruff check src tests

Usage

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

# Timed subtitles: auto global offset + per-cue refine
sub-align media.mp4 subs.srt --language zh -o out.srt

# Lyrics (LRC): same strategy as SRT
sub-align audio.wav lyrics.lrc --language en --margin 1.0

# Untimed script: ASR windows, then force-align original lines
sub-align media.mkv script.txt --language en --model small

# Skip podcast intro/outro before aligning a script
sub-align media.mp3 script.txt --language en --trim-start 13 --trim-end 5

# Known whole-track shift (skips auto-offset ASR)
sub-align media.mkv subs.srt --language en --offset 12.5

# Audio only: transcribe + word-align into an SRT
sub-align lecture.mp4 --language en -o lecture.asr.srt

Always pass --language (e.g. en, zh) or --detect-language.

See docs/usage.md for when to use --model, --margin, --offset, --fill-gaps, --trim-*, audio-only line limits, and a Whisper model size / VRAM cheat sheet.

Python API

1
2
3
4
5
6
7
8
9

from sub_align import align_file

align_file(
    media="a.mp4",
    subtitle="a.srt",   # omit for audio-only transcription
    output="a.aligned.srt",
    language="zh",
    device="auto",
)

How it works (short)

1. Load media as 16 kHz mono audio (via WhisperX / ffmpeg); optional --trim-start / --trim-end.
2. Resolve language (--language or tiny-model detection).
3. Build search windows by input type (ASR token match for .txt; offset + margin refine for .srt/.lrc; full ASR for media-only).
4. Run WhisperX forced alignment; remap word times onto original cues; trim overlaps; optional --fill-gaps; write .srt or .lrc.

Full diagram and tech notes: docs/pipeline.md.

License

MIT