Loading samples…
Fine-grained nonverbal vocalization grounding
Nonverbal vocalizations (NVVs) convey affective and interactional cues, yet transcript-level tags leave their acoustic boundaries unspecified. We present NVV-Locator, built on Qwen3-ASR-0.6B, which uses non-autoregressive slot filling to jointly predict lexical timestamps, NVV categories, and event boundaries from audio and a lexical transcript in one forward pass. We harmonize public resources into a 26-category taxonomy and construct timestamp-supervised training data through dual-LLM verification, transcript-guided alignment, energy-based refinement, and decoupled augmentation. We also introduce NVV-TimeBench, an expert-refined benchmark comprising 667 utterances and 1,094 events. NVV-Locator achieves 71.0% micro F1 and 70.2% macro F1 at temporal IoU ≥ 0.5, outperforming the evaluated prompted large audio models. On matched events, it achieves 80.4% macro mIoU and 59.6 ms macro mMAE. External evaluation without target-corpus training further suggests cross-corpus NVV recognition transfer.
| Family | Canonical Category | Source Datasets and Original Labels |
|---|---|---|
| Physical Vocal Event | Sniff | NVS-38K [sniff]; MNV-17 [sniffle] |
| Snore | NVS-38K [snore] | |
| Whistle | MNV-17 [whistle] | |
| Hum | MNV-17 [hum] | |
| Sneeze | MNV-17 [sneeze] | |
| Yawn | NVS-38K [yawn] | |
| Moan | MNV-17 [moan] | |
| Cry | NVS-38K, SMIIP-NV [crying]; NVSpeech-170k [Crying] | |
| Cough | NVS-38K [coughing]; NVSpeech-170k [Cough]; MNV-17, SMIIP-NV [cough] | |
| Throat Clearing | NVS-38K [throatclearing]; MNV-17 [ahem] | |
| Sigh | NVS-38K, MNV-17 [sigh]; NVSpeech-170k [Sigh] | |
| Lip Smack | MNV-17 [smack] | |
| Hiss | MNV-17 [hiss] | |
| Gasp | NVS-38K [gasp] | |
| Laughter | NVS-38K [laughing]; NVSpeech-170k [Laughter]; MNV-17 [laugh], [chuckle]; SMIIP-NV [laughter] | |
| Breath | NVS-38K [breath]; NVSpeech-170k [Breathing]; MNV-17 [inhale/exhale/pant] | |
| Scream | Added in this work | |
| Roar | Added in this work | |
| Burp | Added in this work | |
| Functional Interjection | Shush | NVSpeech-170k [Shh] |
| Confirmation-X | NVSpeech-170k [Confirmation-en] | |
| Dissatisfaction-X | NVSpeech-170k [Dissatisfaction-hnn] | |
| Question-X | NVSpeech-170k [Question-en/ah/oh/ei/yi] | |
| Filled Pause | NVSpeech-170k [Uhm] | |
| Surprise-X | NVSpeech-170k [Surprise-ah/oh/wa/yo] | |
| Associated Non-Speech Event | Applause | MNV-17 [clap], [applaud] |
Interactive cases
26 NVV event types, IoU ≥ 0.5. Each case: NV-Locator correctly localizes the target; all four audio LLMs miss the target interval. Use the left panel or the ← → keys to switch samples.
Loading samples…