# OpenLLM > OpenLLM carries the AI subscriptions, API keys and local models a user already has into any agent, app or machine, plus a hosted gateway with included inference, with a CLI and MCP server for agents. Use this guide to onboard an existing user on the computer running your agent. Discover current providers, login methods, models, and tools rather than assuming a fixed catalog. ## Quick setup - Install: `curl -fsSL https://www.openllm.sh/install | bash` (agents: `bash -s -- --non-interactive`, see below) - Sign in: `openllm login --non-interactive` (prints a link and code for the user to approve) - Hosted base URL: `https://www.openllm.sh/v1` (API keys and included inference) - Local base URL: `http://127.0.0.1:8787/v1` (subscriptions and local models, through the daemon on this machine) - Key: `OPENLLM_API_KEY` (an `sk-llm` key from sign-in; never print it) - Models: `GET /v1/models` with the key; use the IDs it returns - Formats: OpenAI Chat Completions, OpenAI Responses, Anthropic Messages - MCP: `openllm mcp` Install the CLI only from the `/install` path of the site that served this guide (production: `curl -fsSL https://www.openllm.sh/install | bash`); when this guide came from another host, use that host in place of `https://www.openllm.sh` in the install commands below. The PyPI / pip package named `openllm` is an unrelated BentoML project — do not install it. ## Before making changes - The user must authorize installation and provider login. Account creation and initial vault setup happen in the web onboarding; this guide does not bypass them. - Use the API key supplied by onboarding or the user's existing local configuration. Never request the recovery phrase. Local provider login does not need it. - API keys are sensitive credentials, not harmless identifiers. Do not print them, put them in source control, include them in URLs, or echo configuration files into a transcript. - These commands control only the daemon on this machine. They do not select another device, change the dashboard's “this machine” designation, or bypass remote device authorization. - Ask before logout, cancellation, reinstalling, restarting a working daemon, or performing billable inference. Do not continually refresh quotas or start background authentication. ## Install and start The installer installs `openllm` (alias `ollm`; the daemon is `openllm daemon`). Check `openllm --version` and `openllm status` first if already installed. Two modes (controlling TTY → interactive; otherwise headless). Flags override detection: - **Interactive:** Full/Minimal prompt (Enter → Full). Full starts the daemon after install. - **Headless:** Full only when no setup mode is saved (pass `--full` to force it); no prompts; does **not** start the daemon. Ends with a fixed summary (what was installed, daemon state); `Next:` lines appear only when a next step is needed. ```sh # Interactive (asks Full vs Minimal) curl -fsSL https://www.openllm.sh/install | bash # Headless / agents — always pass --non-interactive (a pseudo-TTY would otherwise hang on the prompt) curl -fsSL https://www.openllm.sh/install | bash -s -- --non-interactive # Minimal (no vendor CLIs, no daemon) curl -fsSL https://www.openllm.sh/install | bash -s -- --minimal ``` Flags (taken at face value; unknown or contradictory flags fail before install): `--help` `--dry-run` `--interactive` / `--non-interactive` (`-y`) `--full` / `--minimal` `--start` / `--no-start` (Full only) `--vendors=a,b` / `--no-vendors` `--origin=` `--port=` `--no-shell-setup`. Key only via `OPENLLM_API_KEY` (never a `--key=` flag). Auth is separate: `openllm login` (OAuth) or `openllm login --key …` / stdin; vendor sign-in stays in the UI or `openllm auth login`. **Agents:** pass `--non-interactive`; for hosted sandboxes use the Minimal path below. If the summary prints `export PATH=…`, run it in this session. With consent, run `openllm login --non-interactive` (JSON `authorize_url` / `code` / `resume`; user approves; then `openllm login --resume `). Start a daemon only when the user wants this long-running machine to serve. On Linux without a usable systemd user bus, `openllm start` runs detached (no restart at boot). Do not invent `nohup` start commands. - On a Mac, the user may instead install the desktop app from [Download for Mac](https://www.openllm.sh/download/mac); it connects the Mac with one click (`OPENLLM_SETUP_MODE=full` + `OPENLLM_START_DAEMON=1`). If the app is already installed, check `openllm status` before installing anything. - `openllm login` prints a sign-in link and a short code (`XXXX-XXXX`). On a terminal it opens the link on Enter; over SSH or with `--no-browser` / `OPENLLM_NO_BROWSER=1`, it prints a QR. `--force` skips the replace-key question. An `OPENLLM_API_KEY` in the environment is used as-is. - Provide an onboarding key through the installer's `OPENLLM_API_KEY` using your harness's secret handling, not a literal key in a command. Prefer Minimal when the user wants OpenLLM only. A vendor CLI skipped / already installing / failed / timed out is a warning naming `~/.openllm/cli-install.log`, not a failed install. - CLI and daemon share `~/.openllm/.env` (including `OPENLLM_SETUP_MODE`). Do not create a second config or overwrite an existing pairing. `OPENLLM_CLOUD_ORIGIN` selects the gateway; retain the origin supplied by onboarding. - A key is required for gateway pairing. Local management uses the daemon's owner-only, per-boot capability automatically; never copy or display that capability. - Missing/stopped daemon: start it with permission when the user wants this machine to serve. Unsupported auth commands on an older binary: update with permission rather than falling back to an unauthenticated HTTP route. - Under minimal setup, `openllm ` talks to the cloud origin and does not start a local daemon, unless `OPENLLM_GATEWAY=local` explicitly forces the local endpoint. Non-interactive runs never silently install a missing client CLI unless `OPENLLM_INSTALL_MISSING_CLIENT=1`. ## Agent sandbox / ephemeral host On hosted agent boxes that pause or recreate, with no TTY or user service manager, install cloud-only: `curl -fsSL https://www.openllm.sh/install | bash -s -- --non-interactive --minimal`. Save the key with `openllm login --non-interactive` and its printed resume command after approval; do not use `--start`. Never run `openllm start` or `openllm auth login` on that box. Run work through the cloud origin with `openllm ` or the gateway base URL and key. Minimal installs no client CLIs: install the client first, or set `OPENLLM_INSTALL_MISSING_CLIENT=1` so a non-interactive `openllm ` installs it. The origin tunnels subscription hops to the user's online devices. Ask the user to sign in to subscriptions on a long-running machine, using the Mac app or `openllm start` on their own computer. ## Discover provider authentication ```sh openllm auth providers --json openllm auth status --json openllm auth usage --json ``` On a sandbox using Minimal, skip local provider authentication and have the user manage subscriptions on their long-running device instead. Elsewhere, discovery reports the running daemon's providers and supported methods. Use the exact returned provider identifier. Status and usage reads do not initiate login or request fresh vendor quota data. Cached or unknown usage is not proof that quota is available. Start a login only when the user requests it. Choose a supported method explicitly: ```sh openllm auth login --method browser --json openllm auth login --method device --json ``` - Use only a method listed in that provider's `methods` from `openllm auth providers --json`. Its `default_method` is the primary method, selected automatically if you omit `--method`; it is not necessarily `browser`. - `browser` starts the vendor's browser-based login on this computer. `device` starts a headless flow that may use a verification URL/code or a paste-back code, not necessarily OAuth device authorization. Some providers are device-only; not every provider offers both methods. - The daemon reuses the vendor's existing native login machinery. The user may still need to open the vendor's site and approve access; “no OpenLLM dashboard required” does not mean “no human consent.” - Keep the returned `flow_id` private: it is the continuation handle for that local login, not a public status identifier. Login returns while native consent is still running. `command_status: "done"` or `pending: true` is not a completed login; confirm `connected: true`. Show the intended user the returned consent instructions, then check that flow: ```sh openllm auth status --flow-id --json ``` If the flow requests a paste-back code, feed the single-use code through stdin to: ```sh openllm auth submit-code --flow-id --json ``` Do not put the code in argv, shell history, logs, or a public document. For automation, use a subprocess stdin pipe. Check status again after submission; stop on an expired/failed flow and report the actual result rather than repeatedly restarting login. ```sh openllm auth cancel --flow-id --json openllm auth logout --json openllm auth refresh --json ``` Cancel ends the selected pending login. Logout signs out that provider on this daemon, not the OpenLLM web account. Refresh explicitly requests updated usage and may defer while authentication is busy. Use it on demand, not as a polling loop. Never cancel someone else's flow or change login methods mid-flow without permission. ## Run a client or connect MCP ```sh openllm --help openllm claude openllm codex openllm mcp ``` - Use the installed CLI's help for supported clients. Session clients launch with temporary configuration instead of overwriting their existing config; always-on integrations such as Raycast are an explicit, reversible exception. - The CLI's supported client overlays include OpenLLM's stdio MCP server. For another MCP client, configure the executable `openllm` with args `["mcp"]`, using the existing local pairing. The server must run on the machine whose daemon you want to control. - MCP exposes local auth tools, native gateway operations, and memory tools. Read their current schemas: do not guess tool names or required fields. - Local auth tools mirror the CLI lifecycle and require the same explicit user intent for mutations. They are not hosted cloud endpoints. - A client already running may need to reconnect/restart its MCP connection after installation or an update. Do not claim new tools are available until the client has listed them. ## Discover inference, then verify - Query authenticated `/v1/models` through the gateway's MCP tool or client SDK. Use exact returned IDs, `provider_type`, optional `provider_variant`, capabilities, formats, and limits. The catalog describes configured availability, not live readiness or remaining quota. - Subscription (oauth) inference and device-discovered local model servers require an online device daemon, either reached through the hosted gateway or used locally. The daemon exposes its own `/v1/` surface like the hosted gateway. Its default API base is `http://127.0.0.1:8787/v1`. The port is configurable through `OPENLLM_DAEMON_PORT` in `~/.openllm/.env`; honor that value instead of hardcoding 8787. The CLI's client launchers handle endpoint selection. - Local model servers the daemon finds on a device (Ollama, LM Studio, …) appear in `/v1/models` as `local_/` (for example `local_ollama/llama3.1:8b`) with `provider_type: "local"`, only while that device is online. Per-port API keys stay on the device; configure them under Providers → Local, never by inventing cloud credentials. - Included inference: every account comes with host-paid models (IDs under `openllm/`, e.g. `openllm/deepseek/...`) that work before any provider is connected, metered against a monthly per-plan allowance. They close the default `ultra` / `plus` / `lite` chains, answering after the user's own subscriptions, local servers and keys; once the allowance is spent they drop out. Use the IDs discovery returns rather than assuming one. - Hosted `https://www.openllm.sh/v1` serves API-key BYOK and included inference, and tunnels subscription and local-server hops to suitable online devices. The cloud does not itself run the user's subscription login or loopback server. - `ultra`, `plus`, and `lite` are configurable fallback aliases, not subscription-only guarantees. They may reach a paid API-key provider. For subscription-only intent, select a direct subscription model from discovery and verify its local auth state. - For a user-authorized smoke test, make one small request to a discovered model and supported endpoint. Preserve the requested wire format and report the actual response/error. Do not benchmark or exhaust quota during onboarding. - For media tools that support automatic model selection, omit `model`; do not send `"auto"` or an empty string. Preserve explicit user constraints and consult the current tool schema. ## Other capabilities and limits - Search: use the supported search tools/models returned by discovery. Grok has a native X/web-search path, but do not assume every model exposes it on every endpoint. Muse's presence alone does not establish Instagram/Facebook search access; do not promise unverified social-search coverage. - Memory: MCP `memory` / `recall` tools save, recall, and forget account-scoped memories across devices. Content is encrypted under the user's vault key; memory is included on every plan. Save only what the user has concluded, never secrets. Use existing pairing; never request the recovery phrase to enable an agent tool. - Privacy: do not describe embeddings as anonymization. Read the relevant tool's contract and the privacy policy before sending sensitive content. - **Agent sessions:** the browser and desktop app drive an on-device vendor CLI engine through a structured command/event protocol. One UI, one engine per session. Browser: device setup through Manage device (install-time trust anchor). Sessions on == the trust anchor is present; disable blanks it (Manage device, `openllm daemon device disable`, or the desktop loopback remove). There is no remote enable — re-enable is install / update / login / desktop with Device sessions on. Agent streams remain seed-gated over RTC/mux. Desktop app: talks to the local daemon on loopback, no grant. Engine and model are picked independently. Permissions: the engine's normal mode by default; `--dangerously-skip-permissions` is a per-session user option. Do not promise cross-device agent-session tunneling or promise an unsupported engine. - Quota estimates and API-equivalent cost estimates are not vendor billing records. Provider auth, entitlement, model capabilities, and current quota remain separate checks. ## References - [Sign in / onboarding](https://www.openllm.sh/sign-in): initial account, vault, and API-key setup. - [Documentation](https://docs.openllm.sh/): public guides and reference; `openllm api --spec` also prints the installed API spec. - [Daemon](https://github.com/openllmsh/daemon): public daemon distribution and documentation. - [CLI](https://github.com/openllmsh/cli): public CLI distribution and documentation. - [SDK](https://github.com/openllmsh/sdk): public `@openllmsh/sdk`, installed with `bun install github:openllmsh/sdk#`. Subpaths: `protocol` (typed contracts), `wire` (format transforms), `tunnel` (device mux), `vault` (client-side crypto), `openapi` (the HTTP API spec + typed calls), `mcp` (the MCP tools), `client` (one facade: vault session, `/v1` chat, device tunnel), `connect` (server-side "Connect with OpenLLM": OAuth, key pickup, token refresh, chat streaming and ready-made routes). The same code the OpenLLM browser app and CLI use. - [Developers](https://www.openllm.sh/developers): add a Connect with OpenLLM button so an app runs on its users' own models; integrate with `@openllmsh/sdk/connect`, guide at [docs.openllm.sh/connect](https://docs.openllm.sh/connect). - [Privacy](https://www.openllm.sh/privacy) (data handling and controls) · [Terms](https://www.openllm.sh/terms) · [Contact](https://www.openllm.sh/contact) (support). - [Public usage index](https://www.openllm.sh/usage-index): aggregate estimates, not a promise of available quota. - [Landing page as Markdown](https://www.openllm.sh/index.md): the home page for agents, with the same setup facts.