Examples

Examples are individual ground truth records within an example set. Each example contains an input, the expected output, and optional metadata explaining why this represents correct behavior.

Example Structure

An example consists of:

• Input - The input data that would be provided to a pipeline
• Expected Output - The correct output the pipeline should produce
• Name - Optional human-readable identifier
• Rationale - Optional explanation of why this is the correct output
• Status - active or archived

Creating an Example

Create an example

cURL

curl -X POST https://api.catalyzed.ai/example-sets/KjR8I6rHBms3W4Qfa2-FN/examples \
  -H "Authorization: Bearer $API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Quarterly Report Summary",
    "input": {
      "document": "Q4 2024 saw revenue growth of 15% year-over-year, driven primarily by expansion in the enterprise segment. Customer acquisition increased by 22%, while churn remained stable at 3.2%. The average contract value grew by $45,000 to reach $180,000."
    },
    "expectedOutput": {
      "summary": "Q4 2024 highlights: 15% YoY revenue growth led by enterprise expansion, 22% increase in customer acquisition, stable 3.2% churn, and ACV growth to $180K."
    },
    "rationale": "Summary captures all key metrics (revenue growth, customer acquisition, churn, ACV) in a concise format."
  }'

TypeScript

const response = await fetch(
  "https://api.catalyzed.ai/example-sets/KjR8I6rHBms3W4Qfa2-FN/examples",
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${apiToken}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      name: "Quarterly Report Summary",
      input: {
        document: "Q4 2024 saw revenue growth of 15% year-over-year...",
      },
      expectedOutput: {
        summary: "Q4 2024 highlights: 15% YoY revenue growth...",
      },
      rationale: "Summary captures all key metrics in a concise format.",
    }),
  }
);
const example = await response.json();

Python

response = requests.post(
    "https://api.catalyzed.ai/example-sets/KjR8I6rHBms3W4Qfa2-FN/examples",
    headers={"Authorization": f"Bearer {api_token}"},
    json={
        "name": "Quarterly Report Summary",
        "input": {
            "document": "Q4 2024 saw revenue growth of 15% year-over-year..."
        },
        "expectedOutput": {
            "summary": "Q4 2024 highlights: 15% YoY revenue growth..."
        },
        "rationale": "Summary captures all key metrics in a concise format."
    }
)
example = response.json()

Response:

{
  "exampleId": "ExR8I6rHBms3W4Qfa2-FN",
  "exampleSetId": "KjR8I6rHBms3W4Qfa2-FN",
  "exampleSetInputsSchema": { ... },
  "exampleSetOutputsSchema": { ... },
  "exampleSetConfiguration": { ... },
  "name": "Quarterly Report Summary",
  "input": {
    "document": "Q4 2024 saw revenue growth of 15% year-over-year..."
  },
  "expectedOutput": {
    "summary": "Q4 2024 highlights: 15% YoY revenue growth..."
  },
  "rationale": "Summary captures all key metrics in a concise format.",
  "status": "active",
  "createdAt": "2024-01-15T10:30:00Z",
  "updatedAt": "2024-01-15T10:30:00Z",
  "createdBy": "usr_abc123"
}

Note

When an example is created, it captures a snapshot of the parent example set’s schema. This ensures examples remain valid even if the example set schema is later updated.

Listing Examples

Within an Example Set

List examples in a set

cURL

curl "https://api.catalyzed.ai/example-sets/KjR8I6rHBms3W4Qfa2-FN/examples" \
  -H "Authorization: Bearer $API_TOKEN"

TypeScript

const response = await fetch(
  "https://api.catalyzed.ai/example-sets/KjR8I6rHBms3W4Qfa2-FN/examples",
  { headers: { Authorization: `Bearer ${apiToken}` } }
);
const { examples } = await response.json();

Python

response = requests.get(
    "https://api.catalyzed.ai/example-sets/KjR8I6rHBms3W4Qfa2-FN/examples",
    headers={"Authorization": f"Bearer {api_token}"}
)
examples = response.json()["examples"]

Across All Example Sets

Use the top-level endpoint to query examples across multiple sets:

curl "https://api.catalyzed.ai/examples?teamIds=ZkoDMyjZZsXo4VAO_nJLk" \
  -H "Authorization: Bearer $API_TOKEN"

Query Parameters

Parameter	Type	Description
exampleIds	string	Comma-separated list of IDs
exampleSetIds	string	Comma-separated example set IDs
teamIds	string	Comma-separated team IDs
name	string	Partial match filter
status	string	active or archived
page	number	Page number (1-indexed)
pageSize	number	Results per page (1-100)
orderBy	string	createdAt, name, updatedAt, status
orderDirection	string	asc or desc

Getting an Example

Get example details

cURL

curl https://api.catalyzed.ai/examples/ExR8I6rHBms3W4Qfa2-FN \
  -H "Authorization: Bearer $API_TOKEN"

TypeScript

const response = await fetch(
  "https://api.catalyzed.ai/examples/ExR8I6rHBms3W4Qfa2-FN",
  { headers: { Authorization: `Bearer ${apiToken}` } }
);
const example = await response.json();

Python

response = requests.get(
    "https://api.catalyzed.ai/examples/ExR8I6rHBms3W4Qfa2-FN",
    headers={"Authorization": f"Bearer {api_token}"}
)
example = response.json()

Updating an Example

Update example

cURL

curl -X PUT https://api.catalyzed.ai/examples/ExR8I6rHBms3W4Qfa2-FN \
  -H "Authorization: Bearer $API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Quarterly Report Summary (Updated)",
    "expectedOutput": {
      "summary": "Q4 2024: 15% revenue growth (enterprise-led), 22% more customers, 3.2% churn, $180K ACV."
    },
    "rationale": "Updated to use more concise format with parenthetical details."
  }'

TypeScript

await fetch("https://api.catalyzed.ai/examples/ExR8I6rHBms3W4Qfa2-FN", {
  method: "PUT",
  headers: {
    Authorization: `Bearer ${apiToken}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    name: "Quarterly Report Summary (Updated)",
    expectedOutput: {
      summary: "Q4 2024: 15% revenue growth (enterprise-led)...",
    },
    rationale: "Updated to use more concise format.",
  }),
});

Python

requests.put(
    "https://api.catalyzed.ai/examples/ExR8I6rHBms3W4Qfa2-FN",
    headers={"Authorization": f"Bearer {api_token}"},
    json={
        "name": "Quarterly Report Summary (Updated)",
        "expectedOutput": {
            "summary": "Q4 2024: 15% revenue growth (enterprise-led)..."
        },
        "rationale": "Updated to use more concise format."
    }
)

Creating Examples from Pipeline Executions

A common workflow is to promote successful pipeline executions to examples. This captures real-world inputs and verified outputs:

Promote execution to example

cURL

curl -X POST https://api.catalyzed.ai/pipeline-executions/GkR8I6rHBms3W4Qfa2-FN/promote-to-example \
  -H "Authorization: Bearer $API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "existingExampleSetId": "KjR8I6rHBms3W4Qfa2-FN",
    "exampleName": "Real customer query - Jan 2024",
    "rationale": "Verified correct by domain expert"
  }'

TypeScript

const response = await fetch(
  "https://api.catalyzed.ai/pipeline-executions/GkR8I6rHBms3W4Qfa2-FN/promote-to-example",
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${apiToken}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      existingExampleSetId: "KjR8I6rHBms3W4Qfa2-FN",
      exampleName: "Real customer query - Jan 2024",
      rationale: "Verified correct by domain expert",
    }),
  }
);
const { example, exampleSet } = await response.json();

Python

response = requests.post(
    "https://api.catalyzed.ai/pipeline-executions/GkR8I6rHBms3W4Qfa2-FN/promote-to-example",
    headers={"Authorization": f"Bearer {api_token}"},
    json={
        "existingExampleSetId": "KjR8I6rHBms3W4Qfa2-FN",
        "exampleName": "Real customer query - Jan 2024",
        "rationale": "Verified correct by domain expert"
    }
)
result = response.json()

You can also create a new example set when promoting:

{
  "newExampleSet": {
    "name": "Production Examples",
    "description": "Examples from verified production executions"
  },
  "exampleName": "First production example",
  "rationale": "Verified by QA team"
}

Note

The execution must have succeeded status to be promoted to an example.

Example Status

Status	Description
active	Example is used in evaluations
archived	Example is excluded from evaluations

Archiving an Example

Archive example

cURL

curl -X POST https://api.catalyzed.ai/examples/ExR8I6rHBms3W4Qfa2-FN/archive \
  -H "Authorization: Bearer $API_TOKEN"

TypeScript

await fetch("https://api.catalyzed.ai/examples/ExR8I6rHBms3W4Qfa2-FN/archive", {
  method: "POST",
  headers: { Authorization: `Bearer ${apiToken}` },
});

Python

requests.post(
    "https://api.catalyzed.ai/examples/ExR8I6rHBms3W4Qfa2-FN/archive",
    headers={"Authorization": f"Bearer {api_token}"}
)

Reactivating an Archived Example

curl -X POST https://api.catalyzed.ai/examples/ExR8I6rHBms3W4Qfa2-FN/reactivate \
  -H "Authorization: Bearer $API_TOKEN"

Deleting an Example

Delete example

cURL

curl -X DELETE https://api.catalyzed.ai/examples/ExR8I6rHBms3W4Qfa2-FN \
  -H "Authorization: Bearer $API_TOKEN"

TypeScript

await fetch("https://api.catalyzed.ai/examples/ExR8I6rHBms3W4Qfa2-FN", {
  method: "DELETE",
  headers: { Authorization: `Bearer ${apiToken}` },
});

Python

requests.delete(
    "https://api.catalyzed.ai/examples/ExR8I6rHBms3W4Qfa2-FN",
    headers={"Authorization": f"Bearer {api_token}"}
)

Example Properties

Field	Type	Description
exampleId	string	Unique identifier
exampleSetId	string	Parent example set
exampleSetInputsSchema	object	Schema snapshot from example set
exampleSetOutputsSchema	object	Schema snapshot from example set
exampleSetConfiguration	object	Configuration snapshot from example set
name	string	Optional name (max 255 chars)
input	object	Input data
expectedOutput	object	Expected output
rationale	string	Optional explanation
status	string	active or archived
createdAt	timestamp	Creation time
updatedAt	timestamp	Last modification time
createdBy	string	User who created this example

Best Practices

1. Use descriptive names - Make examples easy to identify in evaluation results
2. Include rationale - Document why the expected output is correct
3. Cover edge cases - Include examples that test boundary conditions
4. Promote real executions - Build your example set from verified production data
5. Archive instead of delete - Keep historical examples for reference

Next Steps

• Example Sets - Manage collections of examples
• Evaluations - Run evaluations using your examples
• Promoting Executions Guide - Build example sets from production data