Variants

Documentation for working with model variants

Variants are edits to a model that allow you to modify model behavior by adjusting feature activations and defining conditional behaviors.

Creating Variants

Basic Usage

Create a variant by instantiating the Variant class with a base model:

Code

from goodfire import Variant

# Create a variant from a base model
variant = Variant("meta-llama/Llama-3.1-8B-Instruct")
print(variant)

Adding features to a variant

Adding features to a variant

from goodfire import Variant

# Create a variant from a base model
variant = Variant("meta-llama/Llama-3.1-8B-Instruct")

# Search for a feature to modify
feature = client.features.search("formal writing style", model=variant, top_k=1)[0]

# Set feature modifications
variant.set(feature, 0.5)  # Value typically between -1 and 1

Conditional Controls

You can create variants that respond dynamically to feature activations:

Conditional Controls

# First get some features to work with
whale_feature = client.features.search("whales", model=variant, top_k=1)[0]
pirate_feature = client.features.search("pirate speech", model=variant, top_k=1)[0]

# Activate pirate features when whale features are detected
variant.set_when(whale_feature > 0.75, {
    pirate_feature: 0.5
})

# Abort generation if certain features are too strong
variant.abort_when(whale_feature > 0.9)

# Custom handler when condition is met
def my_handler(context):
    print(f"Whale feature activated with strength: {context.activations[whale_feature]}")

Methods

set()

Set feature modifications. This method is overloaded to handle different input types.

Signatures:

def set(self, feature: Feature | FeatureGroup, value: float)
def set(self, edits: dict[Feature, float] | FeatureEdits)

Parameters:

feature · Feature | FeatureGroup

Single feature or feature group to modify. Required when using the first signature.

value · float

Modification value (typically between -1 and 1). Required when using the first signature.

edits · dict[Feature, float] | FeatureEdits

Dictionary of features and their values, or a FeatureEdits object. Required when using the second signature.

Examples:

# Single feature modification
formal_feature = client.features.search("formal writing style", model=variant, top_k=1)[0]
variant.set(formal_feature, 0.5)

# Multiple features at once using dict
casual_feature = client.features.search("casual writing style", model=variant, top_k=1)[0]
variant.set({
    formal_feature: 0.5,
    casual_feature: -0.3
})

# Using FeatureEdits object
edits = client.features.AutoSteer("be funny", model=variant)
variant.set(edits)

set_when()

Define conditional feature modifications.

Parameters:

condition · ConditionalGroup · required

Condition that triggers the modifications

values · dict[Feature, float] | FeatureEdits · required

Feature modifications to apply when condition is met

Example:

set_when example

whale_feature = client.features.search("whales", model=variant, top_k=1)[0]
pirate_feature = client.features.search("pirate speech", model=variant, top_k=1)[0]
# Activate pirate features when whale features are detected
variant.set_when(whale_feature > 0.3, {
    pirate_feature: 0.5
})

abort_when()

Abort generation when a condition is met.

Parameters:

condition · ConditionalGroup · required

Condition that triggers the abort

Example:

Abort example

inappropriate_feature = client.features.search("inappropriate content", model=variant, top_k=1)[0]
# Abort if inappropriate content is detected
variant.abort_when(inappropriate_feature > 0.8)

try:
    response = client.chat.completions.create(
        messages=[{"role": "user", "content": "..."}],
        model=variant,
        stream=False
    )
except goodfire.exceptions.InferenceAbortedException:
    print("Aborted because of inappropriate content")

print(variant)

reset()

Remove all feature modifications.

Example:

Reset variant

variant.reset()

clear()

Remove modifications for specific features.

Parameters:

feature · Feature | FeatureGroup · required

Feature(s) to clear modifications for

Example:

Clear particular feature

variant.clear(feature)

Serialization

Variants can be serialized to and from JSON:

Storing and loading variants

# Save variant to JSON
variant_json = variant.json()

# Load variant from JSON
loaded_variant = Variant.from_json(variant_json)

Using with OpenAI SDK

Variants are compatible with the OpenAI SDK:

Using with OpenAI SDK

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_GOODFIRE_API_KEY",
    base_url="https://api.goodfire.ai/api/inference/v1"
)

response = client.chat.completions.create(
    messages=[{"role": "user", "content": "Hello"}],
    model=variant.base_model,
    extra_body={"controller": variant.controller.json()}
)

Classes

VariantMetaData

Metadata about a model variant.

Properties:

name · str

Name of the variant

base_model · str

Base model identifier

id · str

Unique identifier for the variant