Grammar-based recognition tnl stt¶

Grammar-based recognition constrains a recognizer to a set of words and structures defined by a grammar. Focusing recognition on a limited set of phrases can improve speed and accuracy at the expense of recognizing arbitrary input.

LVCSR models support grammar-based recognition with build-capable models. STT models that support grammar decoding use the same grammar syntax. Some special symbols are model-specific; those cases are noted below.

Syntax¶

A context-free grammar is a set of rules that describes the sequences of words that a recognizer can match.

Definition¶

1. Grammars use UTF-8 encoding.
2. # marks the start of a comment, which extends to the end of the line.
3. 7.9.0 A grammar may optionally begin with a #SNSR identifier line (optionally with a version, for example #SNSR 2.0), mirroring the header that distinguishes BNF+EM and SRGS grammars. A grammar without the header is still recognized as native. Because a bare # line is a comment, the identifier is recognized only as the exact #SNSR token on the first line; any other #... first line remains a comment. When a version is present it must be written as MAJOR.MINOR (for example 2.0), optionally followed by whitespace. The compiler accepts a grammar whose declared version is less than or equal to the version it implements, comparing major and minor in turn: a 2.0 build accepts 1.9 and 2.0 but rejects 2.1 and 3.0. Declaring a newer version than the compiler supports is an error, so a grammar that relies on features from a later dialect fails clearly instead of misparsing. A malformed version (anything other than MAJOR.MINOR) is also an error. A bare #SNSR with no version is treated as the current version. Grammars emitted in native SNSR format (for example by reading back grammar-stream) are stamped with the current #SNSR 2.0 header.
4. A grammar is a series of rules representing variable definitions. The final rule in a grammar specifies the recognition vocabulary and typically references rules defined earlier. It should include the sentence start (<s>) and end (</s>) markers.
5. A rule is an assignment of the form name = expr ; where name is a symbol and expr is a sequence of symbols and operators. expr is a type of regular expression.
6. A symbol is a sequence of characters that does not include any whitespace or operators, optionally prefixed by sigils $ or ~. A symbol without a sigil is called a terminal and is part of the recognition vocabulary, for example temperature. Special symbols are predefined terminals that describe input characteristics such as pauses and the edges of an utterance.
7. The $ sigil does rule substitution at build time. The parser substitutes the value of the rule named name for $name. Substitutions include an implicit grouping operator: Grammar a = 1 | 2 | 3; b = <s> $a </s>; is equivalent to b = <s> (1 | 2 | 3) </s>;.
8. The ~ sigil substitutes a named recognition class at runtime.
   • Each class is a recognizer with its own grammar, separate from the main grammar.
   • All references to a class use instances of the same class recognizer.
   • You can update each class in isolation, without having to recompile the main grammar.
   • If you have a large rule that's referenced multiple times, converting it to a class can speed up build time significantly.
   • Use classes to augment a recognition vocabulary at runtime. In a voice dialing application, for example, you can define the entire recognition grammar at build time but use ~contacts instead of a predefined list of contact names. Once loaded, the application can scan the address book and build only the ~contacts class.
   • Specify class definitions with grammar-stream.classname or phrases-stream.classname, for example phrases-stream.contacts.
   • 7.9.0 A class may also be defined in the grammar itself. If a rule is referenced with the ~ sigil, the parser compiles that rule into the class recognizer for that name. The same rule referenced with $ is still substituted inline; a rule referenced with ~ becomes a class. With this grammar:

     number = 18 | 643 | 20 | 6;
     place = target | winco | susan's house;
     g = <s> directions to {place ~place} | start a timer for ~number minutes </s>;
     

     ~number and ~place are recognition classes built from the number and place rules, with no external stream required. This is a convenient default; it does not change the runtime nature of a class. A ~name that does not match a defined rule remains an undefined class to be supplied at runtime, exactly as before.
     • This changes an edge case of earlier behavior. Previously, if a grammar both defined a rule a and referenced ~a, the class ~a was left undefined until supplied at runtime. Now ~a defaults to the a rule body. For example, in a = 1 | 2 | 3; b = <s> say ~a </s>; the class ~a is now 1 | 2 | 3 rather than empty. A runtime stream still overrides it, so grammars that supply ~a externally are unaffected.
   • 7.9.0 When a class has both an in-grammar definition and an external definition, the external definition wins. A class is resolved in this order of precedence:
     1. An external grammar-stream.classname or phrases-stream.classname set at runtime.
     2. An in-grammar definition (the rule referenced with ~), including a rule pulled in from another module by import. This makes an in-grammar definition an overridable default: existing grammars that rely on external streams are unaffected, and an application can still override any class at runtime.
   • 7.9.0 Common and system classes (for example dates, times, numbers, and phone numbers, conventionally named with an s. prefix) are available as ordinary native grammar modules. The SDK ships them as a single grammar, system.grm, under data/grammars/<language>/. Reference the classes you need and import their definitions with import "system.grm"; and the compiler flattens them into your grammar, so a ~s.* reference resolves to an in-grammar definition like any other, and classes you do not reference are dropped. See modules and imports for the import / from / export directives and the aggregator pattern used to build such a library. Imported classes work in the same grammar on both STT and LVCSR.
     • Older LVCSR models can also load a pre-built binary class library (a separate .snsr repository of classes). This mechanism is deprecated: it works only with LVCSR models, and Sensory no longer publishes new ones. For new grammars, import class definitions instead, so the same grammar runs on both STT and LVCSR. When a binary class library is loaded, it is consulted only for a ~s.* class that is otherwise undefined, after the precedence above.
   • 7.9.0 A ~name class that is referenced but never defined -- not in the grammar, not through an external stream, and (on an older LVCSR model) not in a loaded binary class library -- now produces a warning when the grammar is built or run. Because every class can be filled at runtime this is only advisory, not an error, but it usually indicates an oversight. To keep a class intentionally empty (so it matches nothing until supplied at runtime) without the warning, define it as <void/>, for example city = <void/>;. A system class (~s.*) that is referenced but undefined remains a hard error, since it cannot be supplied by the application.
9. Operators include grouping parentheses, brackets, and braces, infix operators that indicate logical AND and OR between symbols, and postfix operators that change how the preceding symbol matches input. The operator precedence table lists the order and direction in which the parser applies operators.
10. Grouping
    • ( ) Parentheses enclose items that are grouped together.
    • [ ] Square brackets enclose optional items. [...] is equivalent to (...)?.
    • { } Braces implement slot-capturing lightweight NLU markup.
      • {slotName a b c} makes a b c available as the nlu-slot-value of nlu-slot-name slotName when the recognizer matches a b c to the input audio.
      • You can nest NLU slots to an arbitrary depth.
      • The outermost slots are defined as intents and all the nested slots in each intent as entities.
      • Each identified intent invokes handlers registered for ^nlu-intent and ^nlu-slot.
      • {rule} is shorthand for {rule $rule}.
      • With this grammar:

        seconds = 1 | 2 | 4 | 8 | half:0.5 a:? | a:? quarter:0.25 [of: a:];
        shutterSpeed = set shutter speed to {seconds} ( second | seconds );
        cmd = <s> {shutterSpeed} </s>;
        

        an utterance of "set shutter speed to a quarter of a second" will produce set shutter speed to 0.25 second as recognition output, with an additional ^nlu-intent callback for the top-level shutterSpeed slot:

        NLU intent: shutterSpeed (0.0000) = set shutter speed to 0.25 second
        NLU entity:   seconds (0.0000) = 0.25
        

11. Infix operators
    • These are valid between symbols and may be surrounded by whitespace.
    • ^ is the conjunction operator and is implied between adjacent terminals: Grammar g = one two three; will recognize only the sequence "one two three".
    • | is the disjunctive operator. It separates alternative items. Grammar g = one | two | three; will recognize "one", or "two", or "three".
12. Postfix operators
    • These directly follow a symbol without any intervening whitespace.
    • ? A question mark following a symbol makes that symbol optional: It requires zero or one repetitions of the symbol.
    • + A plus sign following a symbol or a group requires one or more repetitions of it.
    • * An asterisk following a symbol or a group requires zero or more repetitions.
    • {m,n} 7.9.0 A brace-enclosed count following a symbol or a group is a bounded repeat: it requires between m and n repetitions (inclusive). Related forms are {m} for exactly m, {m,} for m or more, and {,n} for up to n. Whitespace inside the braces is ignored, so x{2,3} and x{ 2 , 3 } are equivalent. Because a leading { instead begins slot-capturing markup (see below), the bounded-repeat form is recognized only in postfix position and only when its body is numeric: x{2} repeats x, whereas {x} is a slot.
    • : is the rewrite operator.
      • left:right recognizes symbol left but produces terminal right as a recognition result.
      • left: recognizes symbol left but rewrites that to an empty string, eliding left from the recognition result.
      • :right inserts right into the recognition result. If you say "one two three", grammar g = <s> one :mississippi two :mississippi three </s>; produces "one mississippi two mississippi three".
    • / A forward slash following a symbol followed by a floating point number defines a weight to be associated with that symbol. If there's a rewrite operator (:) the slash must follow the rewritten-to terminal, for example: one:een/0.123 Weights are in the logprob domain, convert from a \([0, 1]\) probability to a weight with \(w = -log_{10}(p)\). The default symbol weight is 0 for a probability of 1.0.
      • ( ... )/w 7.9.0 A weight suffix may also follow a group (...) or optional group [...], with no space before the slash. This attaches the weight to the whole group. Used on an alternative branch, for example g = (turn it on) | (turn it off)/2.3;, it biases the recognizer toward or away from that branch without changing which utterances match.
13. \ escape symbol. To include a literal special character in a grammar specification, escape it with a backslash. The list of characters that support this include: ^, |, *, +, ?, =, [ ], ( ), ;, #, and :.

grammar-stream, phrases-stream, nlu-grammar-stream, ^nlu-intent, ^nlu-slot

Modules and imports 7.9.0¶

A native grammar can be split across several files and pull rules from other modules with import directives. This keeps large grammars maintainable and lets you share common rule sets (for example a set of number, date, or place rules) between grammars.

1. Directives end with ; and may appear anywhere among the rules. A grammar that uses no directives compiles exactly as before, so this feature is fully backward compatible.
2. A module is another grammar file. Its default export is its final rule — the same rule that is the recognition vocabulary when that file is compiled on its own.
3. import "module"; is a verbatim import: every name the module exports — and its default export — is brought in unchanged, under those same names. There is no filename-derived namespace: a module that exports s.phone-number is referenced as $s.phone-number. Dots are ordinary name characters, not a namespace separator. import "module" as ns; is the same but prefixes every imported name with ns. (so digit becomes $ns.digit).
4. from "module" import a, b as c; selectively imports the exported rules a and b, binding b to the local name c. The imported names are then used as written ($a, $c).
5. export name, ...; marks local rules as visible to importing grammars, and can also re-export an imported name. export prefix*; is a glob that exports every visible name — local rule or imported name — that begins with prefix. Exports are resolved after the whole file is parsed, so a directive's position does not matter. A rule that is neither exported nor the default export is private to its module. Exporting an undefined name is an error; a glob that matches nothing is allowed.
6. Re-export is a passthrough: if a module imports a name and re-exports it, a consumer resolves that name straight to the module that actually defines it. This makes the aggregator pattern work — a module that only imports many sub-modules and re-exports them (export s.*;), with no rules of its own, is fully transparent. A module may consist of directives alone.
7. Imported rules behave exactly like local rules: a $ reference is inlined at build time, and a ~ reference becomes an in-grammar class (convert-on-reference), keyed by the reference as written (for example ~s.digit). Class resolution is set-wide: a ~name that does not resolve through the referencing module's own imports is matched against any visible rule of that name anywhere in the imported set, so one bundled module can use a class defined by a sibling (for example a phone-number module that references ~s.single-digit-integer) as long as an aggregator has pulled both in. This set-wide fallback applies to ~ classes only, not to $ inlining. All the class-resolution precedence rules above still apply, so an external stream can still override an imported class.
8. Module references resolve as follows: "/abs/path" is absolute; "./rel" and "../rel" are relative to the importing file's directory; and a bare "name" is searched for on the loader's import path. The reference is used verbatim as a filename — no extension is added or assumed, so name your modules and write the references however you like (the .grm suffix used in the examples below is only a convention). When you compile a grammar with the Sensory SDK tools, the import path is the current working directory, so an importing grammar can refer to a sibling module by name. Embedders that link the SDK can install their own loader (for example one backed by a model bundle) to resolve modules however they wish.
9. Import cycles (a module that transitively imports itself) are reported as errors, as is import nesting deeper than 32 levels. A module reached by more than one import path (a diamond) is not a cycle: it is fetched once and shared. If two modules contribute the same name resolving to different rules, that collision is reported as an error.

For example, with a shared numbers.grm module:

# numbers.grm
export digit;
digit = 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9;

a dialing grammar in the same directory can import it:

import "numbers.grm";
g = <s> dial ~digit ~digit ~digit ~digit </s>;

Here ~digit becomes a recognition class built from the imported digit rule, with no external stream required. To keep imported names grouped under a namespace, import as:

import "numbers.grm" as num;
g = <s> dial ~num.digit ~num.digit ~num.digit ~num.digit </s>;

An aggregator collects many modules and re-exports their rules, so a consumer imports a single file. Here export *; re-exports every name the aggregator imported (use export prefix*; to re-export only those sharing a prefix):

# slots.grm  (no rules of its own)
import "phone-number.grm";
import "color.grm";
export *;

import "slots.grm";
g = <s> call $phone-number </s>;

This aggregator pattern is how common and system classes are provided as a grammar rather than as a binary class library (legacy). The SDK ships the system classes as a single self-contained grammar, system.grm, under data/grammars/<language>/ (for example data/grammars/en-US/system.grm). Where an older LVCSR grammar relied on a loaded binary class library for its ~s.* classes:

# Legacy: ~s.phone-number resolved from a loaded binary class library.
g = <s> call {number ~s.phone-number} </s>;

import the system grammar instead, and the compiler flattens in the definitions of the ~s.* classes you reference (unused classes are dropped):

import "system.grm";
g = <s> call {number ~s.phone-number} </s>;

The imported version works on both STT and LVCSR, needs no separate model loaded into the session, and produces a self-contained grammar when read back through grammar-stream. Models included in the SDK distribution are configured so that a bare import "system.grm"; resolves against the shipped data/grammars/<language>/ directory; for other models, add that directory to grammar-import-path or import by absolute path.

Language declaration 7.9.0¶

A native grammar may declare its language with a language directive:

language en-US;
g = <s> hello </s>;

1. The tag is a subset of BCP 47: a two- or three-letter language code with an optional two-letter region, written xx, xxx, xx-YY, or xxx-YY (for example en, eng, or en-US). It is matched case-insensitively and stored canonically as a lowercase language and uppercase region, so EN is normalized to en, and en-us and EN-US are normalized to en-US. Longer languages or additional subtags (scripts, variants) are not accepted.
2. Like the other directives it ends with ; and may appear anywhere among the rules; unlike import/from/export it needs no module loader. The first declaration wins. A second declaration is fine if it names the same language, but a conflicting one is an error. A module's declaration is carried onto a flattened grammar, and modules that disagree on language are an error.
3. The declaration is advisory: it does not change the words a grammar accepts. A build may use it to sanity-check the grammar against the acoustic model's language. It is preserved across the native and SRGS formats; converting to BNF+ drops it (BNF+ has no meaningful language declaration).

Operator precedence¶

The following table lists the precedence and associativity of grammar operators. Operators are listed in descending precedence: level 0 is applied first and level 5 last.

This grammar:

a = one | two three four;
g = <s> ( $a | five six) </s>;

will recognize only these phrases:

one
two three four
five six

Special symbols¶

A grammar can include these special symbols:

• <s> - The silence at the start of a sentence.
• </s> - The silence at the end of a sentence.
• <wp> - Short pauses between words. The grammar compiler automatically adds these where needed, so there is no need to do so explicitly. Do not add <wp> to NLU grammars, use <pause/> instead.
• <pause/> - An explicit short pause.
• <no-match/> tnl - Matches when none of the alternatives are likely (i.e. "none of the above").
  • Recognition results at the phrase level can include <no-match/> even if this symbol was not explicitly used in the grammar. This is an indication that the result was rejected due to search.frame-nota, or that RAM or CPU constraints limited the recognizer's ability to produce a result.
• <void/> - Never matches; an unmatchable alternative that no input can traverse. Use it as a placeholder branch, for example g = ( <void/> | yes | no );, where a rule may later be filled in. It is the native equivalent of BNF+EM <VOID> and SRGS $VOID.
• <unknown/> tnl - Similar to <no-match/>. In some LVCSR models the threshold for determining whether this symbol matches better than any other is different from that of <no-match/>.
  • stt In STT grammars, <unknown/> matches an out-of-grammar word span at a specific point. Use this when the grammar should keep matching even if part of the utterance is not in the fixed vocabulary. In word-level STT models, <unknown/> expands to the primitive wildcard sequence <unknown-start/> <unknown-cont/>*. In character-level STT models, it expands to (<unknown-start/> | <unknown-cont/>).
  • stt By default, <unknown/> has an OOV penalty so in-grammar words win when they fit the audio. Add an explicit symbol weight, for example <unknown/>/2, to tune this penalty in a grammar.
  • stt In NLU output and in the final recognition result, the matched word is emitted unless the grammar uses a rewrite. Use <unknown/>: to match and drop it from these results.
• <unknown-start/> / <unknown-cont/> stt - Primitive OOV wildcard symbols used by <unknown/>. Use them directly only when you need explicit control over word-boundary matching or continuation weighting.
• <dictation/> stt - Hands off recognition to the STT model's statistical language component for free-form speech. This is a one-way hand-off; matching does not return to the grammar afterward.
• . - When used with lightweight NLU grammars, a single period matches any input word. If . is also used as the output label, the matched input word is echoed. Use .:* to match any input words and remove them from the NLU result.

Out-of-grammar words in STT grammars¶

Use <unknown/> to allow an STT grammar slot to absorb words outside the fixed grammar vocabulary. Add the empty rewrite operator (:) when the grammar should match the unknown word but omit it from the captured value:

digit = one | two | three | four | five | six | seven | eight | nine | zero | <unknown/>:;
g = <s> {number $digit+} </s>;

A non-zero weight overrides the default OOV penalty for that arc:

item = <unknown/>/2;

Formal syntax¶

The following ABNF (per RFC 5234) describes the surface syntax of the native grammar language: which sequences of tokens are well formed. It is a convenience for authors building editor support or validation tooling and is not a substitute for the descriptions above.

Two things it deliberately does not express:

• Operator precedence and associativity. ABNF cannot capture these, so the expr and concatenation rules below are intentionally permissive. The binding of operators is fixed by the operator precedence table, which is authoritative.
• Static validation. Constraints such as unique rule names, resolvable $name references, import resolution (export visibility, cycles, and the import-depth limit), and a non-empty start rule are enforced by the compiler, not by this grammar.

Grammars are UTF-8 encoded. The character classes below enumerate ASCII for readability; any non-ASCII UTF-8 code point is also an unreserved character.

; ---- top level ----
; An optional "#SNSR" identifier line may precede the rules. Rules and module
; import directives may be interleaved freely.
grammar         = [ snsr-header ] *ws *( ( rule / directive ) *ws )
; "#SNSR" (case-insensitive), optionally followed by a version token, on its
; own line at the very start (after an optional UTF-8 BOM). A version, when
; present, must be MAJOR.MINOR and must be <= the version the compiler
; implements (a newer version is an error). Any other "#..." first line is an
; ordinary comment.
snsr-header     = "#SNSR" [ 1*WSP snsr-version ] *WSP ( newline / end-of-input )
snsr-version    = 1*DIGIT "." 1*DIGIT
rule            = rule-name *ws "=" *ws expr *ws ";"
rule-name       = name

; ---- module imports (<span class="tag" title="Minimum required SDK version">:material-tag-outline: [7.9.0)][v7.9.0)]{ data-preview }</span> ----
; Directives are recognized contextually: a leading "import"/"from"/"export"
; token NOT immediately followed by "=" (which would make it a rule). A
; module's default export is its final rule; only exported rules and the
; default export are visible to importing grammars.
directive       = import-dir / from-dir / export-dir / language-dir
; "import "m";" imports every exported name (and the default export) VERBATIM;
; "import "m" as ns;" prefixes each imported name with "ns." ($ns.rule).
import-dir      = "import" 1*ws module-ref [ 1*ws "as" 1*ws name ] *ws ";"
; "from "m" import a, b as c;" selectively imports exported rules, optionally
; renaming each with "as".
from-dir        = "from" 1*ws module-ref 1*ws "import" 1*ws
                  import-item *( *ws "," *ws import-item ) *ws ";"
import-item     = name [ 1*ws "as" 1*ws name ]
; "export a, b;" exports local rules or re-exports imported names; "export s.*;"
; is a prefix glob over every visible name (resolved after the file is parsed).
export-dir      = "export" 1*ws export-item *( *ws "," *ws export-item ) *ws ";"
export-item     = name [ "*" ]
; "language en-US;" declares the grammar language (a BCP 47 subset: 2- or
; 3-letter language + optional 2-letter region), case-insensitive and normalized.
; It needs no loader; first wins, a conflicting redeclaration is an error.
language-dir    = "language" 1*ws lang-tag *ws ";"
lang-tag        = (2ALPHA / 3ALPHA) [ "-" 2ALPHA ]
; A module reference is a quoted string (recommended) or a bare token: "/abs"
; is absolute; "./rel" and "../rel" are relative to the importing file; a bare
; "name" is searched on the loader's import path. The reference is a filename
; used verbatim -- no extension is added or assumed.
module-ref      = quoted-ref / bare-ref
quoted-ref      = DQUOTE *( %x20-21 / %x23-FF ) DQUOTE   ; any char except '"'
bare-ref        = name

; ---- expressions ----
; One or more alternatives separated by "|".
expr            = concatenation *( *ws "|" *ws concatenation )
; A sequence of terms. Adjacent terms concatenate implicitly (whitespace);
; the explicit "^" conjunction is equivalent to juxtaposition.
concatenation   = term *( ( 1*ws / *ws "^" *ws ) term )
; A primary optionally followed by postfix repetition operators. Postfix
; operators immediately follow the operand with no intervening whitespace.
; A "{" in postfix position with a purely numeric body is a bounded repeat;
; a "{" that begins a term is instead slot-capturing markup (see "slot").
term            = primary *postfix-op
postfix-op      = "?" / "+" / "*" / bounded-repeat
; Whitespace inside the braces is ignored. "{m}" is exactly m; "{m,}" is m or
; more; "{,n}" is up to n; "{m,n}" is between m and n (inclusive).
bounded-repeat  = "{" *ws ( 1*DIGIT [ *ws "," *ws *DIGIT ]
                          / "," *ws 1*DIGIT ) *ws "}"

primary         = group / optional-group / slot / symbol
; A group (and optional group) may carry a trailing "/w" weight suffix, with no
; space before the slash; on an alternation branch this weights the branch.
group           = "(" *ws expr *ws ")" [ weight ]
; "[ ... ]" is equivalent to "( ... )?".
optional-group  = "[" *ws expr *ws "]" [ weight ]
; "{ slotName a b c }" captures "a b c" as slot "slotName";
; "{ rule }" is shorthand for "{ rule $rule }". Slots may nest.
slot            = "{" *ws [ slot-name 1*ws ] expr *ws "}"
slot-name       = name

; ---- symbols ----
; Any symbol may carry an optional rewrite and/or weight. A bare "/float"
; is an epsilon symbol carrying only a weight.
symbol          = recognized [ rewrite ] [ weight ]
                / rewrite [ weight ]
                / weight
recognized      = rule-ref / class-ref / terminal
; "$name" is a build-time rule substitution; "~name" is a runtime recognition
; class. Dots in "name" are ordinary characters (e.g. "$s.phone-number"). With
; module imports, "name" may be a verbatim imported name, an "as ns"-prefixed
; name, or a name bound by a selective "from ... import".
rule-ref        = "$" name          ; build-time rule substitution
class-ref       = "~" name          ; runtime recognition-class substitution
terminal        = name              ; a word in the recognition vocabulary
; "left:right" recognizes left, emits right; "left:" emits nothing;
; ":right" is a pure insertion (no recognized input).
rewrite         = ":" [ name ]
; Weight in the log-prob domain (w = -log10(p)); default 0.
weight          = "/" float

; ---- names and lexical primitives ----
; ":" and "/" are ordinary name characters; they take on their rewrite/weight
; meaning only when a symbol is decomposed. A literal ":" is written "\:".
name            = 1*name-char
name-char       = unreserved / escaped
; Any single character that is not reserved. Reserved characters are
; whitespace, ";", "#", and the operator/grouping characters
; ^ | * + ? = [ ] ( ) { }. Any non-ASCII UTF-8 code point is also unreserved.
unreserved      = %x21-22 / %x24-27 / %x2C-3A / %x3C / %x3E / %x40-5A
                / %x5C / %x5F-7A / %x7E / %x80-10FFFF
; A backslash-escaped reserved character. The backslash is removed.
escaped         = "\" escapable
escapable       = "^" / "|" / "*" / "+" / "?" / "=" / "[" / "]"
                / "(" / ")" / ";" / "#" / ":" / "{" / "}"

float           = [ sign ] ( 1*DIGIT [ "." *DIGIT ] / "." 1*DIGIT ) [ exponent ]
exponent        = ( "e" / "E" ) [ sign ] 1*DIGIT
sign            = "+" / "-"

; ---- whitespace and comments ----
; A "#" begins a comment that runs to end of line. Newlines are ordinary
; whitespace and do not terminate a rule; only ";" does.
ws              = WSP / newline / comment
comment         = "#" *( %x00-09 / %x0B-FF ) ( newline / end-of-input )
newline         = CR / LF / CRLF
end-of-input    = ""

; Core rules from RFC 5234 Appendix B: WSP, CR, LF, CRLF, DIGIT, DQUOTE.

The predefined special symbols (<s>, </s>, <pause/>, <unknown/>, and so on) lex as ordinary terminals.

BNF+EM v2.1 grammars 7.9.0¶

In addition to the native syntax described above, the SDK accepts grammars written in the BNF+EM v2.1 format used by other speech vendors. Support for this format eases migration of existing Cerence-style grammars.

Format detection¶

A grammar is parsed as BNF+EM when its first non-comment content is a #BNF+EM header, for example #BNF+EM V2.1;. Otherwise the native syntax is used. A single grammar is entirely one format or the other; you cannot mix the two.

#BNF+EM V2.1;
!grammar lights;
!start <command>;
<command>: ( turn ) ( on | off ) [ the ] ( <NULL> | living room ) lights ;

Supported subset¶

The SDK compiles the subset of BNF+EM most commonly used in command-and-control grammars. Each construct maps onto an equivalent in the native syntax, so its recognition behavior matches the corresponding native form:

Unsupported constructs¶

The following BNF+EM constructs are not yet supported. A grammar that uses one is rejected with an error that identifies the construct and its position, rather than compiling to something unexpected:

• !import and cross-grammar references (<grammar#rule>).
• !start with more than one entry-point rule.
• !slot runtime-attachable rules.
• !id(...) user identifiers.
• !pronounce, both the inline modifier and the grammar-wide statement.
• Quoted-string terminals ("...").
• Other inline ! modifiers, and !language.

Support for these constructs may be added in a future release.

SRGS 1.0 (ABNF Form) grammars 7.9.0¶

The SDK also accepts grammars written in the W3C SRGS 1.0 "ABNF Form", the speech recognition grammar format standardized for VoiceXML and other voice platforms. Support for this format eases migration of existing SRGS grammars.

Format detection¶

A grammar is parsed as SRGS when its first non-comment content is an #ABNF header, for example #ABNF 1.0;. Otherwise the native syntax (or BNF+EM, for a #BNF+EM header) is used. A single grammar is entirely one format; you cannot mix them.

#ABNF 1.0;
language en-US;
root $command;
public $command = turn ( on | off ) [ the ] ( $NULL | living room ) lights;

Supported subset¶

The full SRGS ABNF surface syntax is parsed. The subset most commonly used in command-and-control grammars compiles to a recognizer; each construct maps onto an equivalent in the native syntax, so its recognition behavior matches the corresponding native form:

A repeat operator may carry an optional repetition probability, written x<m-n /p/>, where p is the probability of taking another repetition. It does not change which utterances match (still between m and n copies); it biases the recognizer by charging a log-prob cost of \(-log(p)\) on each "repeat again" step and \(-log(1-p)\) on each "stop" step, the same way a branch weight biases an alternative.

The self-identifying header (#ABNF 1.0 [encoding];) and the header declarations (language, mode, base, tag-format, lexicon, meta, http-equiv) are accepted and preserved for round-tripping, but do not change recognition.

Unsupported constructs¶

The following SRGS constructs are parsed but cannot yet be compiled to a recognizer. A grammar that relies on one is rejected with an error that identifies the construct and its position, rather than compiling to something unexpected:

• Recursive rule references (a rule that refers to itself, directly or through a cycle).
• External rule references ($<uri> and $<uri>~<media-type>).

The *, +, and ? operators are reserved by SRGS and are not valid repetition operators; use the postfix <m-n> form instead. Semantic interpretation (executing tag scripts to build a result object) and the SRGS XML Form (grxml) are out of scope. Support for these may be added in a future release.