API Documentation


This endpoint generates Language Model completions that match a given Context-Free Grammar (CFG) rule. It allows fine-grained control over the completion process through various parameters.

Request: POST /v1/completion/cfg


  • Content-Type: application/json
  • Authorization: Bearer <your_api_key>


The request accepts a JSON body with the following properties:

promptstringThe prompt to generate from.Yes
grammarstringA single CFG grammar rule to match.Yes
max_new_tokensintegerThe maximum number of tokens to generate. Defaults to 5.No
stop_after_matchbooleanWhether to stop generating after a match is found. Defaults to true.No
top_kintegerThe number of highest probability vocabulary tokens to keep for top-k-filtering. Defaults to 50.No
top_pfloatIf set to float < 1, only the most probable tokens with probabilities that add up to top_p or higher are kept for generation. Defaults to 1.0.No
temperaturefloatThe value used to modulate the next token probabilities. Defaults to 1.0.No
repetition_penaltyfloatThe parameter for repetition penalty. 1.0 means no penalty. More details here (opens in a new tab). Defaults to 1.0.No


  "prompt": "Start your text here",
  "grammar": "<start> ::= <noun> <verb> | <verb> <noun>",
  "max_new_tokens": 10,
  "stop_after_match": false,
  "top_k": 30,
  "top_p": 0.9,
  "temperature": 1.2,
  "repetition_penalty": 0.8


The response from the server includes the following fields:

completionstringThe generated completion that matches the given CFG grammar rule.
tokens_generatednumberThe number of tokens generated in the completion.
stop_reasonstringThe reason the generation was stopped, if applicable. Either pattern_match or token_limit.


  "completion": "noun verb",
  "tokens_generated": 2,
  "stop_reason": "pattern_match"