AI Gateway

View Markdown

Providers, models & consumers.mdDownload .md

# Operate providers, models and consumers

AI Gateway separates upstream supply from client access:

```mermaid
flowchart LR
  P[Provider] --> C[Encrypted credentials]
  C --> M[Enabled models]
  M --> J[Projects]
  J --> U[Consumers]
  U --> K[rk_ai_ keys]
  K --> X[IDEs and applications]
```

```text
provider credentials → enabled models → projects → consumers → rk_ai_ keys
```

Provider keys remain encrypted in the gateway. Clients authenticate with
revocable consumer keys.

## Providers and credentials

Create a provider:

```bash
runku-ai provider add --config config.ai.yaml \
  --name primary --kind openai --base-url https://api.openai.com/v1 \
  --api-key "$OPENAI_API_KEY"
```

Add a second credential for rotation or failover:

```bash
runku-ai provider credential add --config config.ai.yaml \
  --provider <provider-id> --api-key "$SECONDARY_KEY" \
  --label secondary --priority 20
```

List APIs and the portal never return raw provider keys.

## Model catalog

Enable an internal model ID and map it to the upstream provider model:

```bash
runku-ai model enable --config config.ai.yaml \
  --id coding-fast --display-name "Coding Fast" \
  --provider <provider-id> --upstream gpt-5-mini
```

Clients should prefer `runku/coding-fast`. Bare IDs and legacy Runku prefixes
are normalized by the gateway.

## Projects

Projects group consumers and may restrict which provider credentials are used:

```bash
runku-ai project create --config config.ai.yaml \
  --id engineering --name "Engineering" \
  --description "Engineering tools"

runku-ai project credential add --config config.ai.yaml \
  --project engineering --credential <credential-id>
```

## Consumers and keys

```bash
runku-ai consumer create --config config.ai.yaml \
  --id cursor-team --name "Cursor team" --kind human \
  --project engineering

runku-ai consumer quota set --config config.ai.yaml \
  --id cursor-team --max-tokens 2000000 --max-requests 10000

runku-ai consumer key create --config config.ai.yaml \
  --id cursor-team --label workstation
```

The raw `rk_ai_…` value is displayed once. Create separate keys for separate
people or workloads so they can be revoked and audited independently.

## Credential boundaries

| Credential | Prefix | Intended use |
|---|---|---|
| Consumer key | `rk_ai_` | `/v1/*`, IDEs, SDKs and applications |
| Admin key | `rk_admin_` | `/admin/v1/*` and portal administration |
| Provider key | Provider-specific | Stored only in the encrypted server vault |

Never place an admin key or provider key in an IDE.

## Cache operations

```bash
runku-ai cache stats --config config.ai.yaml
runku-ai cache purge --config config.ai.yaml
```

Purging cache does not delete model catalog, consumers or usage audit.

## Client installation lifecycle

```bash
runku-ai plugin list
runku-ai plugin install <id> --base https://ai.example.com/v1 \
  --model coding-fast --key rk_ai_...
runku-ai plugin uninstall <id>
```

Install and uninstall are symmetric. Cursor-specific repair must be run from a
normal terminal while Cursor is closed:

```bash
runku-ai plugin fix-cursor --relaunch
```

See [IDE and client integrations](/docs/v1/ai-clients) for client-specific limits.

Operate providers, models and consumers

AI Gateway separates upstream supply from client access:

flowchart LR
  P[Provider] --> C[Encrypted credentials]
  C --> M[Enabled models]
  M --> J[Projects]
  J --> U[Consumers]
  U --> K[rk_ai_ keys]
  K --> X[IDEs and applications]

provider credentials → enabled models → projects → consumers → rk_ai_ keys

Provider keys remain encrypted in the gateway. Clients authenticate with revocable consumer keys.

Providers and credentials

Create a provider:

runku-ai provider add --config config.ai.yaml \
  --name primary --kind openai --base-url https://api.openai.com/v1 \
  --api-key "$OPENAI_API_KEY"

Add a second credential for rotation or failover:

runku-ai provider credential add --config config.ai.yaml \
  --provider <provider-id> --api-key "$SECONDARY_KEY" \
  --label secondary --priority 20

List APIs and the portal never return raw provider keys.

Model catalog

Enable an internal model ID and map it to the upstream provider model:

runku-ai model enable --config config.ai.yaml \
  --id coding-fast --display-name "Coding Fast" \
  --provider <provider-id> --upstream gpt-5-mini

Clients should prefer runku/coding-fast. Bare IDs and legacy Runku prefixes are normalized by the gateway.

Projects

Projects group consumers and may restrict which provider credentials are used:

runku-ai project create --config config.ai.yaml \
  --id engineering --name "Engineering" \
  --description "Engineering tools"

runku-ai project credential add --config config.ai.yaml \
  --project engineering --credential <credential-id>

Consumers and keys

runku-ai consumer create --config config.ai.yaml \
  --id cursor-team --name "Cursor team" --kind human \
  --project engineering

runku-ai consumer quota set --config config.ai.yaml \
  --id cursor-team --max-tokens 2000000 --max-requests 10000

runku-ai consumer key create --config config.ai.yaml \
  --id cursor-team --label workstation

The raw rk_ai_… value is displayed once. Create separate keys for separate people or workloads so they can be revoked and audited independently.

Credential boundaries

Credential	Prefix	Intended use
Consumer key	`rk_ai_`	`/v1/*`, IDEs, SDKs and applications
Admin key	`rk_admin_`	`/admin/v1/*` and portal administration
Provider key	Provider-specific	Stored only in the encrypted server vault

Never place an admin key or provider key in an IDE.

Cache operations

runku-ai cache stats --config config.ai.yaml
runku-ai cache purge --config config.ai.yaml

Purging cache does not delete model catalog, consumers or usage audit.

Client installation lifecycle

runku-ai plugin list
runku-ai plugin install <id> --base https://ai.example.com/v1 \
  --model coding-fast --key rk_ai_...
runku-ai plugin uninstall <id>

Install and uninstall are symmetric. Cursor-specific repair must be run from a normal terminal while Cursor is closed:

runku-ai plugin fix-cursor --relaunch

See IDE and client integrations for client-specific limits.