For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.

Configure Nebius Token Factory as an LLM provider for agentgateway.

Configure Nebius Token Factory as an LLM provider in agentgateway. The Nebius preset uses https://api.tokenfactory.nebius.com/v1 as the default base URL. The preset serves Chat Completions and Embeddings requests.

Before you begin

1. Install and set up an agentgateway proxy.
2. Allow the AgentgatewayModel route kind on the http listener of the agentgateway-proxy Gateway. For the steps, see Enable LLM serving on a listener.
3. Get a Nebius API key and choose a Nebius model. The following examples use openai/gpt-oss-120b for Chat Completions and Qwen/Qwen3-Embedding-8B for embeddings.

Set up Nebius

Store your Nebius API key in a Secret. Then, create a model that uses the Nebius preset. The model attaches directly to the Gateway listener, so you do not create an AgentgatewayBackend or an HTTPRoute.

1. Save the API key in an environment variable.

   export NEBIUS_API_KEY='<your-api-key>'

2. Create a Kubernetes Secret to store your API key.

   kubectl apply -f- <<EOF
   apiVersion: v1
   kind: Secret
   metadata:
     name: nebius-secret
     namespace: agentgateway-system
   type: Opaque
   stringData:
     Authorization: $NEBIUS_API_KEY
   EOF

3. Create an AgentgatewayModel that uses the Nebius preset and references the Secret.

   kubectl apply -f- <<EOF
   apiVersion: agentgateway.dev/v1alpha1
   kind: AgentgatewayModel
   metadata:
     name: nebius-chat
     namespace: agentgateway-system
   spec:
     parentRefs:
     - group: gateway.networking.k8s.io
       kind: Gateway
       name: agentgateway-proxy
       sectionName: http
     match:
       model: openai/gpt-oss-120b
     provider: Nebius
     policies:
       auth:
         secretRef:
           name: nebius-secret
   EOF

   Review the following table to understand this configuration. For more information, see the API docs.

   Setting	Description
   spec.match.model	The model name that clients send in requests.
   spec.provider	The Nebius preset. The preset supplies the default URL, https://api.tokenfactory.nebius.com/v1, and the Chat Completions and Embeddings request formats.
   spec.policies.auth.secretRef	The Secret that holds your Nebius API key.

4. Send a Chat Completions request to the model.

   Cloud Provider LoadBalancerPort-forward for local testing

   curl "http://$INGRESS_GW_ADDRESS/v1/chat/completions" \
     -H 'Content-Type: application/json' \
     -d '{
       "model": "openai/gpt-oss-120b",
       "messages": [{"role": "user", "content": "Explain model routing in one sentence."}]
     }' | jq

   curl "localhost:8080/v1/chat/completions" \
     -H 'Content-Type: application/json' \
     -d '{
       "model": "openai/gpt-oss-120b",
       "messages": [{"role": "user", "content": "Explain model routing in one sentence."}]
     }' | jq

To serve embeddings, create another AgentgatewayModel that uses provider: Nebius and a Nebius embedding model, such as Qwen/Qwen3-Embedding-8B. Then, clients can send embeddings requests to /v1/embeddings with that model name.

Cleanup

kubectl delete AgentgatewayModel nebius-chat -n agentgateway-system
kubectl delete secret nebius-secret -n agentgateway-system