jknife

jllm

LLM tool family — a single binary with subcommands, based on langchain4j. Providers: openai, ollama. Mcp servers can be connected for tool use (see mcp servers).

Being one binary means the (large) langchain4j runtime is paid only once; new llm commands are added as subcommands.

Usage

jllm ask [options] [prompt]         # high level: complete response (no streaming)
jllm chat [options]                # interactive multi turn conversation (mcp tools allowed)
jllm stream [options] [prompt]      # high level: tokens live as they arrive
jllm request [options] [body]       # raw: provider json passthrough, curl for llms
jllm models [options] [model]       # discovery: list models or show model details
jllm mcp [options] [server]         # discovery: analyze mcp servers (tools, skills, docs)
jllm mcp login/logout SERVER        # oauth browser login (see mcp servers)
jllm mcp exec SERVER TOOL [args]    # execute one tool directly (json or yaml args)

ask vs stream vs request

	ask	stream	request	models
input	prompt (arg/stdin)	prompt (arg/stdin)	provider json (arg/stdin)	optional model name
output	response text when complete	tokens live	raw response bytes, as they arrive	model list / pretty json
langchain4j	ChatModel	StreamingChatModel	none (plain http)	none (plain http)
extras	-v metadata	--perf measurement	byte exact, sse passthrough	-v details, --raw passthrough

Streaming is a genuinely different process in langchain4j (separate interfaces, separate model classes, different wire protocol) — hence separate subcommands.

chat

jllm chat is a multi turn conversation in the terminal: every turn sends the full history (system prompt, all messages, tool calls) to the model. The model may use the mcp tools from the config — the same tool loop as ask runs per turn, no other tools. One failed turn does not end the session: the error is printed and the history stays at the state before the failed turn.

jllm chat --llm-config llm.yaml
# Chat with gpt-4o-mini (openai, 3 mcp tools). /exit or ctrl-d to quit, /reset clears the history.
> what files are in /tmp?
# ... answer ...
> /reset
History cleared
> /exit

Commands: /exit or /quit (or ctrl-d, end of pipe) end the session, /reset clears the history (the system prompt stays). The system prompt comes from --system or from the systemPrompt field of the llm config; -v logs tool discovery, calls and results on stderr. Piped input works too: each line is one turn, e.g. echo 'question' | jllm chat --llm-config llm.yaml.

Examples

jllm ask --llm '{provider: ollama, model: llama3.1}' "explain quantum computing"
jllm ask --llm-config llm.yaml --system "You are a pirate." "say hello"

# live tokens while the model writes
jllm stream --llm-config llm.yaml "write a poem"

# with performance measurement (block on stderr)
jllm stream --llm-config llm.yaml --perf "write a haiku"

# raw: send the openai chat completion json directly, response passthrough
jllm request --llm-config llm.yaml << 'EOF'
{"model": "gpt-4o-mini", "messages": [{"role": "user", "content": "hi"}]}
EOF

# raw with streaming: "stream": true makes the provider answer with a live
# sse/ndjson stream, which is passed through chunk by chunk
echo '{"model":"llama3.1","messages":[...],"stream":true}' | jllm request --llm-config ollam.yaml

# openai api key from env OPENAI_API_KEY or config
jllm ask --llm '{provider: openai, model: gpt-4o-mini}' "hi"

Model discovery

jllm models --llm-config ollam.yaml          # list models, one per line
jllm models --llm-config ollam.yaml -v       # with details
# llama3.1:latest | family=llama | params=8.0B | quant=Q4_K_M | size=4.6GB

jllm models --llm-config ollam.yaml llama3.1:latest   # details of one model (pretty json)
jllm models --llm-config openai.yaml gpt-4o-mini
jllm models --llm-config ollam.yaml --raw             # raw endpoint response

Endpoints used: openai GET <baseUrl>/models and GET <baseUrl>/models/{model}; ollama GET <baseUrl>/api/tags and POST <baseUrl>/api/show. Like request, the models command uses only provider, baseUrl, apiKey and timeoutSeconds from the config.

MCP server discovery

jllm mcp analyzes the mcp servers from the config — which tools, prompts (skills) and resources (docs) they provide. Only mcpServers and timeoutSeconds from the llm config are used, the model/provider fields are not required:

jllm mcp --llm-config llm.yaml            # overview: one block per server
# weather (http http://localhost:3001/mcp)
#   tools: get_weather, get_forecast
#   prompts: weather_report
#   resources: file:///info

jllm mcp --llm-config llm.yaml weather   # full details of one server
jllm mcp --llm-config llm.yaml weather --tool get_weather   # one tool only, with its schema
jllm mcp exec --llm-config llm.yaml weather get_weather '{"city": "Munich"}'   # execute a tool

The overview prints the transport endpoint and the names of all tools, prompts and resources ((none) if a server provides nothing of a kind; resource templates only when present). The detail view adds descriptions, the tools’ parameter schemas (pretty json), the arguments of the prompts (name (required)) and mime types of resources. With --tool NAME the detail view is reduced to a single tool — description and full parameter schema, without the prompts and resources sections; an unknown tool name is an error (exit code 2). For oauth servers the login status is part of the detail view. A server that cannot be reached is reported on stderr — the other servers are still analyzed, but the exit code is 2.

mcp exec executes one tool without a model: the arguments (a json or yaml object, default {} — or from stdin/-t file) go to the tool, the raw tool result is printed to stdout. Auth applies like everywhere else (oauth login, tokenCommand, headers) — the perfect end-to-end test for a fresh mcp configuration:

jllm mcp exec --llm-config llm.yaml weather get_weather '{"city": "Munich"}'
jllm mcp exec --llm-config llm.yaml weather ping        # no arguments -> {}

LLM configuration

All llm parameters come from the config — either inline (--llm, yaml or json) and/or from a file (--llm-config). If both are given, the inline config is merged over the file content, so a file can hold the base config and the inline value can override single fields (e.g. just the model).

# llm.yaml
provider: openai        # openai | ollama
model: gpt-4o-mini      # e.g. gpt-4o-mini, llama3.1
baseUrl: https://api.openai.com/v1   # optional, per provider default
apiKey: sk-...          # optional, openai also falls back to $OPENAI_API_KEY
temperature: 0.7        # optional
maxTokens: 1024         # optional (openai)
timeoutSeconds: 60      # optional
maxToolRounds: 10      # optional, bounds the mcp tool round trips of ask and chat (default 10)
systemPrompt: ...      # optional, system prompt for ask/chat/stream (a --system option overrides it)
mcpServers: {}         # optional, mcp servers for tools — see the section below

Defaults: baseUrl https://api.openai.com/v1 (openai) or http://localhost:11434 (ollama), timeoutSeconds 60.

request uses only provider, baseUrl, apiKey and timeoutSeconds from the config — the rest comes from the json body you send. Endpoints: openai <baseUrl>/chat/completions (with Authorization: Bearer <apiKey>), ollama <baseUrl>/api/chat.

MCP servers (tools)

ask can use tools provided by mcp servers (model context protocol, via the langchain4j-mcp client): each configured server is asked for its tools, the tool specifications are sent to the model, and tool calls of the model are executed and answered until the model responds with plain text (bounded by maxToolRounds).

Mcp servers are part of the llm config (file or inline, merge rules as usual):

# llm.yaml
provider: openai
model: gpt-4o-mini
apiKey: sk-...
mcpServers:
  # stdio: local subprocess (command + args), transport inferred from the command
  filesystem:
    command: npx
    args: [-y, "@modelcontextprotocol/server-filesystem", /tmp]
    env: {NODE_ENV: production}
  # http: remote server (streamable http), transport inferred from the url
  atlassian:
    transport: http
    url: http://localhost:3001/mcp
    headers: {Authorization: "Bearer ${ATLASSIAN_TOKEN}"}
  # http with a token from a command: the first stdout line becomes
  # the bearer token, refreshed every 60 seconds
  cloud:
    transport: http
    url: https://mcp.example.com/mcp
    tokenCommand: gcloud auth print-access-token

• transport: stdio (default if only command is given) or http (default if only url is given). stdio needs command (+ optional args, env), http needs url (+ optional headers).
• a command with arguments in one string is split on whitespace when no args are given (command: 'npx -y foo').
• ${NAME} in env, headers and tokenCommand values is replaced with the environment variable (empty string if unset) — secrets do not belong into the config file.

Mcp auth

• stdio servers: authentication is passed via env (api keys the subprocess needs).
• http servers with static tokens: headers, e.g. headers: {Authorization: "Bearer ${MY_TOKEN}"}. Good for personal access tokens and other long-lived credentials.

• http servers with expiring tokens: tokenCommand — a shell command whose first stdout line becomes the Authorization: Bearer header. The command runs through the system shell (/bin/sh -c, on windows cmd /c), its result is cached for 60 seconds so a tool loop does not spawn a shell for every request. Static headers are merged under the bearer header. ${NAME} expansion applies to the command itself.

  mcpServers:
    cloud:
      url: https://mcp.example.com/mcp
      tokenCommand: gcloud auth print-access-token   # or: op read ..., vault kv get ..., pass show ...
  

  This way the token infrastructure (sso cli, secret store) stays with the user — jllm never stores credentials. A failing or empty token command aborts the request with exit This way the token infrastructure (sso cli, secret store) stays with the user — jllm never stores credentials. A failing or empty token command aborts the request with exit code 2.

• http servers with oauth (browser login): an oauth section turns the server into an oauth client — jllm mcp login <server> opens the browser, the tokens are stored locally and refreshed automatically. See mcp oauth.

• if two servers expose a tool with the same name, the first one in config order wins.

Mcp oauth (browser login)

For mcp servers that require a real oauth login (e.g. the Atlassian remote mcp server):

mcpServers:
  atlassian:
    url: https://mcp.atlassian.com/mcp
    oauth:
      provider: atlassian      # atlassian (preset endpoints) | generic
      clientId: <your-client-id>
      scopes: [read:jira:work, write:jira:work]
  selfhosted:
    url: https://mcp.example.com/mcp
    oauth:
      provider: generic
      clientId: my-client
      clientSecret: ${MY_SECRET}     # optional (PKCE-only flows)
      authorizeUrl: https://sso.example.com/authorize
      tokenUrl: https://sso.example.com/token

Then log in once:

jllm mcp login atlassian       # browser flow (pkce): opens the browser, waits for the callback
jllm mcp login atlassian --no-browser   # headless: prints the url, you open it elsewhere
jllm mcp logout atlassian      # remove the stored tokens

The login uses the authorization code flow with PKCE (RFC 7636), a loopback callback on 127.0.0.1 (RFC 8252 style) and a csrf state — the Atlassian specifics (mandatory audience parameter, prompt=consent) are built in. The tokens (access + refresh) are stored in ~/.config/jllm/mcp-auth/<server>.json with owner-only permissions (0600) (JLLM_AUTH_DIR overrides the directory). Every request refreshes the token automatically when it is within 60 seconds of expiring; a broken refresh means the login is gone and you run jllm mcp login again. jllm mcp <server> shows the login status in its detail view.

For servers with dynamic client registration (RFC 7591, e.g. resource servers that publish a registration_endpoint) you can omit the clientId and give a registrationUrl instead — the login then registers its own client with the loopback redirect of this login (the callback port changes per login, so a fresh client is registered each time). The RFC 8707 resource parameter defaults to the mcp server url (some servers reject the authorize and token requests without it) and can be overridden with oauth.resource. Some oauth apps only allow a registered redirect uri with a concrete port — for those, oauth.callbackPort (or the login option --callback-port N) binds the callback server to that fixed port. Servers without the refresh grant (e.g. api.sipgate.com) simply expire — log in again when that happens.

The redirect uri of your oauth app must allow loopback redirects (http://localhost:<port>/callback); for Atlassian 3LO apps this is the default. Only one of oauth and tokenCommand can be configured per server; static headers are still merged under the bearer header.

jllm ask --llm-config llm.yaml "what files are in /tmp?"     # the model may use the mcp tools
jllm ask --llm-config llm.yaml -v "what files are in /tmp?"  # tool discovery, calls, results on stderr

-v shows the discovered tools per server (Mcp tools <name>: ...) and every tool call and result. The final answer goes to stdout as usual — script friendly.

Only ask uses mcp tools (streaming with tools is not supported yet); request and models ignore the mcpServers section. The model must support tool calling (e.g. gpt-4o, llama3.1). To inspect a server before using it, see mcp server discovery (jllm mcp).

Performance measurement (stream)

jllm stream --perf uses the streaming internals to measure and prints a block to stderr — the tokens still go to stdout, so the output stays script friendly:

--- performance ---
provider: openai
model: gpt-4o-mini
baseUrl: https://api.openai.com/v1
params: temperature=null maxTokens=null timeout=60s
latency: 1.234s
ttft: 0.412s
tokens: in=5 out=12 total=17
output tokens/sec: 14.1
finish reason: STOP
response model: gpt-4o-mini
response id: chatcmpl-abc123

ttft is the time to first token, tokens the token usage reported by the provider (when available) and output tokens/sec is derived from both.

ask -v prints token usage, finish reason and response model after the answer.

Options

• --llm CONFIG inline llm config (yaml or json), merged over --llm-config
• --llm-config FILE llm config file (yaml or json)
• ask/stream/chat: --system TEXT system prompt (overrides systemPrompt from the config)
• stream: --perf performance block (see above)
• -t, --text-file FILE, -c, --charset, -v, --verbose common input options (see jregex)
• mcp login: --no-browser print the authorize url only, --wait-seconds N callback timeout (default 300), --callback-port N fixed callback port (default: ephemeral or oauth.callbackPort)
• mcp: --tool NAME restrict the detail view to one tool (with its parameter schema)
• mcp exec SERVER TOOL [ARGS]: execute one tool directly (no model) — ARGS is a json or yaml object (default {}, alternatively from stdin via -t/--text-file), the raw tool result goes to stdout

Exit codes: 0 success, 2 error (missing/invalid config, empty prompt/body, request failure, http error status).

Install

brew tap mhus/jknife
brew install jllm

Or download the binary for your platform from the releases.

This site is open source. Improve this page.