Use DeepSeek Harness with llmrelay

Setup

1. 1 npx @deepseek-ai/dsh web — starts the Web UI on http://127.0.0.1:3080.
2. 2 Open Settings -> Models and click "Add a custom provider". ("Add provider" only offers the built-in catalog, which does not include us.)
3. 3 Provider ID: llmrelay — lowercase, and permanent, because requests, saved sessions and model defaults all reference it.
4. 4 API protocol: openai-completions. Base URL: https://api.llmrelay.dev/v1.
5. 5 Paste your llmrelay key as the API key. DSH stores it write-only in $DSH_HOME/.credentials.yaml and keeps only a reference in settings.
6. 6 Under Model catalog, click "Fetch available models" and select what you want, or type model ids by hand.
7. 7 Save, then pick an llmrelay model in the model picker. That also makes it the default for new sessions.

Config

# Option A - Web UI: Settings -> Models -> Add a custom provider
# Provider ID:   llmrelay
# Display name:  LLM Relay
# API protocol:  openai-completions
# Base URL:      https://api.llmrelay.dev/v1
# API key:       sk-llmrelay-...

# Option B - edit $DSH_HOME/settings.yaml directly
llm-pi-ai:
  providers:
    llmrelay:
      displayName: LLM Relay
      apiKeyEnv: LLMRELAY_API_KEY
      api: openai-completions
      baseURL: https://api.llmrelay.dev/v1
      models:
        - id: claude-opus-5
          name: Claude Opus 5 (relay)
          contextWindow: 200000
          maxTokens: 128000
        - id: claude-sonnet-5
          name: Claude Sonnet 5 (relay)
          contextWindow: 200000
          maxTokens: 128000
        - id: deepseek-v4-flash
          name: DeepSeek V4 Flash (relay)
          contextWindow: 128000
          maxTokens: 384000

# Anthropic-native route instead - note the bare host, no /v1
llm-pi-ai:
  providers:
    llmrelay-anthropic:
      displayName: LLM Relay (Anthropic)
      apiKeyEnv: LLMRELAY_API_KEY
      api: anthropic-messages
      baseURL: https://api.llmrelay.dev
      models:
        - id: claude-opus-5
          contextWindow: 200000
          maxTokens: 128000

Smoke test

Copy, paste, run. If you get JSON back with a message, you're good.

# 1. Confirm the endpoint DSH will call actually answers
curl https://api.llmrelay.dev/v1/chat/completions \
  -H "Authorization: Bearer $LLMRELAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"claude-opus-5","messages":[{"role":"user","content":"say hi"}]}'

# 2. Confirm tool calling works - a harness is useless without it
curl https://api.llmrelay.dev/v1/chat/completions \
  -H "Authorization: Bearer $LLMRELAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"claude-sonnet-5","max_tokens":200,
       "messages":[{"role":"user","content":"Weather in Paris? Use the tool."}],
       "tools":[{"type":"function","function":{"name":"get_weather",
         "parameters":{"type":"object","properties":{"city":{"type":"string"}}}}}]}'
# Expect finish_reason "tool_calls" and a populated tool_calls array.

# 3. In DSH: select an llmrelay model and send "say hi" in a new session.

Gotchas

• !claude-opus-5 is callable but is NOT returned by GET /v1/models, so "Fetch available models" will not list it. Tested 2026-08-23: the fetch returns 11 ids, while a chat request for claude-opus-5 returns 200. Add it by hand in the Model catalog field.
• !Use https://api.llmrelay.dev/v1 for api: openai-completions, but the bare host https://api.llmrelay.dev for api: anthropic-messages. Tested 2026-08-23: adding /v1 to the Anthropic route yields /v1/v1/messages and 404s.
• !You probably do NOT need the compat switches the DSH provider guide leads with. Tested 2026-08-23 against the relay: role "developer" returns 200 and max_completion_tokens returns 200, so supportsDeveloperRole: false and maxTokensField: max_tokens are unnecessary here. Add them only if you actually see a request refused.
• !A model you type by hand is treated as text-only until it says otherwise. Add input: [text, image] to that model in settings.yaml for vision — the Web UI form has no field for it.
• !The Provider ID cannot be renamed. Changing it means adding a new provider and deleting the old one, which orphans the model defaults saved in existing sessions.
• !DSH is in developer preview and its own README warns of compatibility-breaking changes. Pin @deepseek-ai/dsh to an exact version in anything you automate instead of tracking latest.
• !apiKeyEnv is a reference resolved per request, so no key enters settings.yaml, and one credential serves every model on its route. A reference that resolves to nothing fails with MISSING_CREDENTIAL rather than quietly authenticating with some other key in the environment.

Related integrations