Perturbation lab

Add noise, shift pitch, stretch time, or mask audio and watch metrics move.

The perturbation lab applies a controlled transform to a clip, re-runs the model on the result, and lets you compare the prediction before and after — a direct robustness check rather than an inferred one.

noise
time_masking
pitch_shift
time_stretch

What is perturbation?

Perturbation means changing a clip in a controlled way, such as adding noise or shifting pitch, and re-running the model on the result. Comparing the prediction before and after tells you directly whether the model's answer is robust to that change.

Four perturbation types

There are four perturbation types available today — noted explicitly here because marketing copy elsewhere may round this up; this page is the accurate count:

noise
time_masking
pitch_shift
time_stretch

Diagram: an original clip goes through one controlled change to become a perturbed clip, and the model is re-run on it. Four perturbation types, each shown as a small waveform: noise adds background noise, time masking silences a stretch of audio, pitch shift raises or lowers the pitch, and time stretch speeds up or slows down. Below, a before-and-after comparison: the original prediction 'the cat sat' next to the perturbed prediction 'the cap sat' with the changed word highlighted, and the metrics to read: word error rate, whether the label flipped, and confidence change. Up to 10 perturbations per request.

Request limits

Up to 10 perturbations per request. Each generates a new audio variant, which is then run back through the model so you can compare its prediction against the original.

Comparing results

For each perturbation, the panel shows:

Original vs. generated waveform, side by side, so you can see exactly what the transform did to the signal.
Re-run prediction for the perturbed clip, next to the original prediction, so you can see whether — and how much — the transform moved the model's output.

Metrics reported

For transcription models the perturbed transcript is scored against a reference after Whisper's own text normalisation, so punctuation and casing don't count as errors:

Metric	Meaning
`wer` / `cer`	Word and character error rate (capped at 1.0). Scored against the dataset ground truth, or against the unperturbed transcript to measure self-consistency.
`substitutions`, `deletions`, `insertions`	Edit counts behind WER, so you can see how the transcript degraded.
`insertion_ratio`	Insertions ÷ reference words. High insertions with few deletions is the hallucination signature.

For classification models the perturbed prediction is compared with the baseline prediction:

Metric	Meaning
`confidence_delta`	Change in top-class confidence versus baseline.
`label_flipped`	`1` if the predicted label changed.
`js_divergence`	Jensen-Shannon divergence between the perturbed and baseline probability distributions, in nats, bounded 0 to ln 2 (≈ 0.69). Symmetric and always finite.
`true_class_confidence_delta`	Change in confidence on the baseline class.

When several files are perturbed together, set-level accuracy, macro-F1, and macro-recall are also reported (they need at least 2 files).

This is the most direct robustness signal in AudioLens: instead of asking "what does the model attend to," perturbation asks "does the model's answer survive a small, controlled change to the input."

Was this page helpful?

AudioLens docs · MIT LicenseWaveform to logits