snsr-eval¶
This tool evaluates / runs TrulyNatural SDK snsr model files. It supports all task types, except wake word enrollment which is handled by spot-enroll and live-enroll.
Usage¶
Evaluates/runs TrulyNatural SDK .snsr model files.
usage: snsr-eval -t task [options] [wavefile ...]
options:
-a : Add tpl-vad-lvcsr to LVCSR and STT models
-d directory : VAD audio output directory
-f setting filename : load filename into task setting
-g setting value : load string into task setting
-i listFile : run evaluation on each filename in listFile
-l [-l [-l]] : reduce verbosity
-o out : output filename for VAD audio or listFile results
-p [-p] : Enable pipeline profiling (experimental)
-q setting : query a task setting
-s setting=value : override a task setting
-t task : specify task filename (required)
-u filename : remove unused settings and save model to filename
-v [-v [-v]] : increase verbosity
Use a filename of - to read headerless linear 16-bit PCM little-endian
audio from stdin. If you don't specify any wave files, snsr-eval uses
live audio captured from the default audio device.
The -d and -o options are mutually exclusive. The output directory
must be writable. Audio files created by VAD segmentation are named
<directory>/<start-time-in-ms>.wav
Settings are strings used as keys to query or change task behavior.
Most frequently used are operating-point for wake words and command sets,
leading-silence and trailing-silence for VAD templates,
partial-result-interval for LVCSR and STT, and stt-profile for STT models.
Refer to the TrulyNatural SDK documentation for a complete list and
descriptions of all supported settings.
Batch processing 7.8.0¶
If you specify the -i listFile option, snsr-eval will evaluate the model on the filenames in listFile. This loads the model once and re-uses the session instance for each evaluation, reducing overhead. It expects one filename per line.
In batch processing mode, snsr-eval produces output in tab-separated value format. Each audio file in listFile has a corresponding result line in the output, unless the processing the audio file results in an error. Such errors are treated as warnings and printed to stderr. If you don't specify an output file with -o output goes to stdout instead.
Output columns are, in order:
- File index, starting at
1 - Audio filename
- If there is a recognition result:
- If there is an nlu-intent:
- Intent name, nlu-intent-name
- Intent score, nlu-intent-score
- Intent value, nlu-intent-value
- For each NLU entity found:
- Entity name, nlu-entity-name
- Entity score, nlu-entity-score
- Entity value, nlu-entity-value
Examples¶
Fixed-phrase.
% snsr-eval -t ./model/spot-hbg-enUS-1.4.0-m.snsr hbg_2.wav hbg_7.wav
1200 1905 hello blue genie
3855 4575 hello blue genie
Fixed-phrase on default audio capture device.
% snsr-eval -v -t ./model/spot-hbg-enUS-1.4.0-m.snsr.snsr
Using live audio from default capture device. ^C to stop.
3180 3885 (1.00 sv) hello blue genie
9000 9720 (1.00 sv) hello blue genie
^C
Enrolled user-defined phrase.
% snsr-eval -v -t ./three-users.snsr -s sv-threshold=0\
./data/enrollments/armadillo-1-4-c.wav ./data/enrollments/armadillo-6-0.wav\
./data/enrollments/terminator-2-5.wav ./data/enrollments/jackalope-1-4-c.wav
435 990 (0.89 sv) armadillo-1
5940 6630 (0.99 sv) terminator-2
8100 8610 (0.32 sv) jackalope-1
Lower the speaker-verification threshold.
% snsr-eval -v -t ./three-users.snsr -s sv-threshold=0\
./data/enrollments/jackalope-1-5.wav ./data/enrollments/jackalope-1-5-c.wav
270 840 (0.56 sv) jackalope-1
2130 2610 (0.33 sv) jackalope-1
Recognize a list of audio files.