> ## Documentation Index
> Fetch the complete documentation index at: https://docs.goodfire.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Chat Completions

> Documentation for the Goodfire Chat API

The Chat API provides methods for interacting with Goodfire's language models in a chat format.
The base chat interface is OpenAI-compatible. It supports both streaming and non-streaming completions, as well as logits computation.

Once you have a [model variant](/sdk-reference/variants), you can use it to create chat completions.

## Examples

### Basic Chat Completion

<CodeGroup name="Basic Chat Completion">
  ```python Code theme={null}
  # Initialize the client
  client = goodfire.Client(
      '{YOUR_API_KEY}',
  )

  # Create a non-streaming completion
  response =  client.chat.completions.create(
      messages=[
          {"role": "user", "content": "What is the capital of France?"}
      ],
      model="meta-llama/Llama-3.1-8B-Instruct"
  )

  print(response.choices[0].message["content"])
  ```

  ```output Output theme={null}
  The capital of France is Paris.
  ```
</CodeGroup>

### Streaming Chat Completion

<CodeGroup name="Streaming Chat Completion">
  ```python Streaming Chat Completion theme={null}
  # Stream the response tokens
  for chunk in client.chat.completions.create(
      messages=[
          {"role": "user", "content": "Write a short poem"}
      ],
      model="meta-llama/Llama-3.1-8B-Instruct",
      stream=True
  ):
      print(chunk.choices[0].delta.content, end="")
  ```

  ```output Output Poem theme={null}
  Here's a short poem:

  Golden sunsets paint the sky,
  Peaceful evening, twilight high.
  Stars appear, and night grows deep,
  A time for rest, a time to sleep.
  ```
</CodeGroup>

### Using with Model Variants

<CodeGroup name="Using with Model Variants">
  ```python Using with Model Variants theme={null}
  # Create a variant with feature modifications
  variant = goodfire.Variant("meta-llama/Llama-3.1-8B-Instruct")
  pirate_features = client.features.search(
      "talk like a pirate",
      model=variant,
      top_k=1
  )
  variant.set(pirate_features[0], 0.5)

  # Use the variant in chat completion
  for token in client.chat.completions.create(
      messages=[
          {"role": "user", "content": "Tell me about the ocean."}
      ],
      model=variant,
      stream=True,
      max_completion_tokens=100,
  ):
      print(token.choices[0].delta.content, end="")
  ```

  ```output Piratey Output theme={null}
  The ocean be a vast and wondrous place! Coverin' over 70% o' the Earth's surface, it's home to a treasure trove o' sea creatures, from tiny plankton to massive blue whales. The ocean's got a lot o' secrets, and it's up to us to explore 'em and learn 'bout the world below the waves.
  ```
</CodeGroup>

### Computing Token Probabilities

<CodeGroup name="Token Probabilities">
  ```python Token Probabilities theme={null}
  logits =  client.chat.logits(
      messages=[
          {"role": "user", "content": "The capital of France is"}
      ],
      model="meta-llama/Llama-3.1-8B-Instruct",
      filter_vocabulary=["Paris", "London", "Berlin"]
  )
  print(logits.logits)
  ```

  ```output Logit Output theme={null}
  {'Paris': 8.75, 'Berlin': 7.96875, 'London': 4.3125}
  ```
</CodeGroup>

## Methods

### create()

Create a chat completion with the model.

**Parameters:**

<ResponseField name="messages" type="list[ChatMessage]" required>
  List of messages in the conversation. Each message should have `role` ("user",
  "assistant", or "system") and `content` fields.
</ResponseField>

<ResponseField name="model" type="Union[str, VariantInterface]" required>
  Model identifier or variant to use for completion
</ResponseField>

<ResponseField name="stream" type="bool" default="False">
  Whether to stream the response tokens
</ResponseField>

<ResponseField name="max_completion_tokens" type="Optional[int]" default="2048">
  Maximum number of tokens to generate
</ResponseField>

<ResponseField name="top_p" type="float" default="0.9">
  Nucleus sampling parameter
</ResponseField>

<ResponseField name="temperature" type="float" default="0.6">
  Sampling temperature
</ResponseField>

<ResponseField name="stop" type="Optional[Union[str, list[str]]]">
  Sequences where the API will stop generating further tokens
</ResponseField>

<ResponseField name="seed" type="Optional[int]" default="42">
  Random seed for reproducible outputs
</ResponseField>

<ResponseField name="system_prompt" type="str">
  System prompt to prepend to the conversation
</ResponseField>

**Returns:**

* If `stream=False`: `ChatCompletion` object
* If `stream=True`: Iterator of `StreamingChatCompletionChunk` objects

**Examples:**

Non-streaming completion:

```python Basic Chat Completion theme={null}
response =  client.chat.completions.create(
    messages=[
        {"role": "user", "content": "What is the capital of France?"}
    ],
    model="meta-llama/Llama-3.1-8B-Instruct"
)
print(response.choices[0].message["content"])
```

Streaming completion:

```python Streaming Chat Completion theme={null}
for chunk in client.chat.completions.create(
    messages=[
        {"role": "user", "content": "Write a short poem"}
    ],
    model="meta-llama/Llama-3.1-8B-Instruct",
    stream=True
):
    print(chunk.choices[0].delta.content, end="")
```

### logits()

Compute token probabilities for the next token in a conversation.

**Parameters:**

<ResponseField name="messages" type="list[ChatMessage]" required>
  List of messages in the conversation
</ResponseField>

<ResponseField name="model" type="Union[str, VariantInterface]" required>
  Model identifier or variant to use
</ResponseField>

<ResponseField name="top_k" type="Optional[int]">
  Limit response to top K most likely tokens
</ResponseField>

<ResponseField name="filter_vocabulary" type="Optional[list[str]]">
  List of tokens to compute probabilities for
</ResponseField>

**Returns:** `LogitsResponse` containing token probabilities

**Example:**

```python Token Probabilities theme={null}
logits =  client.chat.logits(
    messages=[
        {"role": "user", "content": "The capital of France is"}
    ],
    model="meta-llama/Llama-3.1-8B-Instruct",
    filter_vocabulary=["Paris", "London", "Berlin"]
)
print(logits.logits)  
```

## Response Objects

### ChatCompletion

Response from a non-streaming chat completion.

**Properties:**

<ResponseField name="id" type="str">
  Unique identifier for the completion
</ResponseField>

<ResponseField name="object" type="str">
  Object type identifier
</ResponseField>

<ResponseField name="created" type="Optional[int]">
  Unix timestamp of when the completion was created
</ResponseField>

<ResponseField name="model" type="str">
  ID of the model used
</ResponseField>

<ResponseField name="system_fingerprint" type="str">
  System fingerprint for the completion
</ResponseField>

<ResponseField name="choices" type="list[ChatCompletionChoice]">
  List of completion choices
</ResponseField>

### StreamingChatCompletionChunk

Individual chunk from a streaming chat completion.

**Properties:**

<ResponseField name="id" type="str">
  Unique identifier for the completion
</ResponseField>

<ResponseField name="object" type="str">
  Object type identifier
</ResponseField>

<ResponseField name="created" type="Optional[int]">
  Unix timestamp of when the chunk was created
</ResponseField>

<ResponseField name="model" type="str">
  ID of the model used
</ResponseField>

<ResponseField name="system_fingerprint" type="str">
  System fingerprint for the completion
</ResponseField>

<ResponseField name="choices" type="list[StreamingChoice]">
  List of completion choices in this chunk
</ResponseField>

### LogitsResponse

Response from a logits computation request.

**Properties:**

<ResponseField name="logits" type="dict[str, float]">
  Dictionary mapping tokens to their probabilities
</ResponseField>