An OpenAI-compatible API on your own server
Last updated
An OpenAI-compatible API on your own server accepts the same requests as OpenAI's API, so apps built for OpenAI work with your local model once you change the base URL and key. vLLM and Ollama both provide one.
Olares One → Local LLM server → vLLM · Ollama · OpenAI-compatible API → Your models, served on your network
What is an OpenAI-compatible API on your own server?
An OpenAI-compatible API on your own server is an HTTP endpoint that accepts OpenAI's request and response format but answers with a model running on your hardware. Most AI tools, SDKs and editor plugins already speak that format. Point them at your server and they work after you change the base URL, the key and the model name.
The common routes are /v1/chat/completions for chat, /v1/completions for plain text, /v1/embeddings for embeddings and /v1/models to list what the server offers.
Which serving engines provide the API?
vLLM and Ollama both provide an OpenAI-compatible API. vLLM runs it as its main server mode, with streaming, batching across clients and an API key you set at startup. Ollama exposes it next to its own native API. Open WebUI can sit in front of either and give each user a key tied to their account.
We'd put vLLM behind the API whenever several apps or people call it at once, because it keeps per-request speed up under load. Ollama fits a single developer. The engines are compared in vLLM or Ollama.
How do you connect an app to it?
Connect an app by setting its base URL to your server's /v1 address and its API key to the key your server expects. In the OpenAI Python SDK that is one line, shown below. Coding tools, agent frameworks and document tools usually have a custom-endpoint field that takes the same two values.
client = OpenAI(base_url="https://llm.example.lan/v1", api_key="your-key")
Then call client.chat.completions.create with the model name your server reports. Expect some differences from the hosted API. Model names are your own. Tool calling and structured output depend on the engine and the model, and rate limits are whatever your hardware sustains.
How do you keep the API private?
Keep the API private by never exposing it straight to the internet. Anyone who reaches an open endpoint can use your GPU and read what it returns, and Ollama's API has no login of its own. Put a key or an authenticated proxy in front, keep the port on the local network, and reach it from outside through a VPN.
Olares One includes a built-in VPN for remote access. The rest of the checklist is in securing a local LLM server.
Questions & answers
Can I use the OpenAI SDK with a local model?
Yes. Set the base URL to your server's /v1 address and the API key to the one your server expects. Use the model name your server lists.
Does LM Studio work with a local LLM server?
LM Studio is a desktop app. It can run its own local server on your computer, and other tools can call that server through the same OpenAI-compatible format.
Sources for this entry
- Olares One FAQ (Olares docs) (Company-published)
Continue this reading path
How the related websites connect
Cite this entry
Local LLM Server. “An OpenAI-compatible API on your own server.” Content reviewed 13 November 2026.
https://localllmserver.com/openai-compatible-local-api/