Skip to main content

Send requests

Use aib req to check connectivity, inspect model output, and debug request bodies.

Text and authentication​

aib req \
--base-url https://your-inference-host.example \
--api-key "$AIB_API_KEY" \
--model your-model \
--prompt "What is prefix caching?"

--api-key reads OPENAI_API_KEY by default, falling back to JustKeepMe. --base-url accepts a trailing /v1; do not provide the full chat completions path.

Streaming is the default. Use --disable-stream to inspect a non-streaming JSON response, --raw to inspect raw SSE data, or --no-print to hide streamed text.

Use your own payload​

Use --prompt - (or --user-prompt -) to read plain text from a pipe or file redirection, preserving whitespace and newlines:

cat prompt.txt | aib req --prompt - --max-tokens 128
aib req --prompt - < prompt.txt

For a complete JSON request body, use --payload-stdin or --payload-path -.

Create request.json:

{
"messages": [{"role": "user", "content": "Explain batching."}],
"temperature": 0,
"max_tokens": 128
}
aib req --base-url http://127.0.0.1:30000 \
--model your-model --payload-path request.json

A payload containing prompt without messages is sent to /v1/completions.

Override fields​

aib req --prompt "What is a GPU?" --output-len 128 \
--override-payload '{"temperature":0}'

Overrides merge at the top level. Nested objects are replaced, not recursively merged. An overridden model takes precedence over --model. The tool then sets streaming and statistics fields; overrides cannot switch a benchmark to non-streaming mode.

--enable-thinking, --disable-thinking, --tools, --json-schema-response-format, and --ebnf require server support. The last three use built-in demonstration structures; use a payload file for custom structures.

Debug output

--verbose prints headers and payloads, including authentication details. Remove keys before sharing logs.