Signals

Signals are feedback records that capture expert input on pipeline outputs. Use signals to record comments, ratings, and evaluation results that can inform pipeline improvements through synthesis.

Signal Types

Type	Description	Data
comment	Free-form text feedback	{ type: "comment", text: "..." }
rating	Numerical score	{ type: "rating", score: 0.0-1.0 }
evaluation	Linked to evaluation result	{ type: "evaluation", evaluationResultId: "...", score: 0.0-1.0 }

Creating Signals

Comment Signal

Create a comment signal

cURL

curl -X POST https://api.catalyzed.ai/signals \
  -H "Authorization: Bearer $API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "teamId": "ZkoDMyjZZsXo4VAO_nJLk",
    "data": {
      "type": "comment",
      "text": "This summary missed the key point about Q4 revenue growth. Should emphasize the 15% YoY increase."
    },
    "executionIds": ["GkR8I6rHBms3W4Qfa2-FN"]
  }'

TypeScript

const response = await fetch("https://api.catalyzed.ai/signals", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${apiToken}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    teamId: "ZkoDMyjZZsXo4VAO_nJLk",
    data: {
      type: "comment",
      text: "This summary missed the key point about Q4 revenue growth.",
    },
    executionIds: ["GkR8I6rHBms3W4Qfa2-FN"],
  }),
});
const signal = await response.json();

Python

response = requests.post(
    "https://api.catalyzed.ai/signals",
    headers={"Authorization": f"Bearer {api_token}"},
    json={
        "teamId": "ZkoDMyjZZsXo4VAO_nJLk",
        "data": {
            "type": "comment",
            "text": "This summary missed the key point about Q4 revenue growth."
        },
        "executionIds": ["GkR8I6rHBms3W4Qfa2-FN"]
    }
)
signal = response.json()

Response:

{
  "signalId": "SigR8I6rHBms3W4Qfa2-FN",
  "teamId": "ZkoDMyjZZsXo4VAO_nJLk",
  "dataType": "comment",
  "data": {
    "type": "comment",
    "text": "This summary missed the key point about Q4 revenue growth."
  },
  "executionIds": ["GkR8I6rHBms3W4Qfa2-FN"],
  "createdAt": "2024-01-15T10:30:00Z"
}

Rating Signal

Create a rating signal

cURL

curl -X POST https://api.catalyzed.ai/signals \
  -H "Authorization: Bearer $API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "teamId": "ZkoDMyjZZsXo4VAO_nJLk",
    "data": {
      "type": "rating",
      "score": 0.7
    },
    "executionIds": ["GkR8I6rHBms3W4Qfa2-FN"]
  }'

TypeScript

const response = await fetch("https://api.catalyzed.ai/signals", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${apiToken}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    teamId: "ZkoDMyjZZsXo4VAO_nJLk",
    data: {
      type: "rating",
      score: 0.7,
    },
    executionIds: ["GkR8I6rHBms3W4Qfa2-FN"],
  }),
});
const signal = await response.json();

Python

response = requests.post(
    "https://api.catalyzed.ai/signals",
    headers={"Authorization": f"Bearer {api_token}"},
    json={
        "teamId": "ZkoDMyjZZsXo4VAO_nJLk",
        "data": {
            "type": "rating",
            "score": 0.7
        },
        "executionIds": ["GkR8I6rHBms3W4Qfa2-FN"]
    }
)
signal = response.json()

Evaluation Signal

Evaluation signals are typically created automatically when an evaluation completes, linking the signal to an evaluation result:

{
  "teamId": "ZkoDMyjZZsXo4VAO_nJLk",
  "data": {
    "type": "evaluation",
    "evaluationResultId": "ResABC123...",
    "score": 0.85,
    "feedback": "Output matches expected with minor differences"
  },
  "executionIds": ["GkR8I6rHBms3W4Qfa2-FN"]
}

Listing Signals

List signals

cURL

curl "https://api.catalyzed.ai/signals?teamIds=ZkoDMyjZZsXo4VAO_nJLk" \
  -H "Authorization: Bearer $API_TOKEN"

TypeScript

const response = await fetch(
  "https://api.catalyzed.ai/signals?teamIds=ZkoDMyjZZsXo4VAO_nJLk",
  { headers: { Authorization: `Bearer ${apiToken}` } }
);
const { signals } = await response.json();

Python

response = requests.get(
    "https://api.catalyzed.ai/signals",
    params={"teamIds": "ZkoDMyjZZsXo4VAO_nJLk"},
    headers={"Authorization": f"Bearer {api_token}"}
)
signals = response.json()["signals"]

Query Parameters

Parameter	Type	Description
signalIds	string	Comma-separated list of IDs
teamIds	string	Comma-separated team IDs
executionIds	string	Comma-separated execution IDs
dataTypes	string	Comma-separated: comment, rating, evaluation
page	number	Page number (1-indexed)
pageSize	number	Results per page (1-100, default: 50)
orderDirection	string	asc or desc

Filter by Execution

Get all signals for a specific pipeline execution:

curl "https://api.catalyzed.ai/signals?executionIds=GkR8I6rHBms3W4Qfa2-FN" \
  -H "Authorization: Bearer $API_TOKEN"

Filter by Type

Get only rating signals:

curl "https://api.catalyzed.ai/signals?teamIds=ZkoDMyjZZsXo4VAO_nJLk&dataTypes=rating" \
  -H "Authorization: Bearer $API_TOKEN"

Getting a Signal

Get signal details

cURL

curl https://api.catalyzed.ai/signals/SigR8I6rHBms3W4Qfa2-FN \
  -H "Authorization: Bearer $API_TOKEN"

TypeScript

const response = await fetch(
  "https://api.catalyzed.ai/signals/SigR8I6rHBms3W4Qfa2-FN",
  { headers: { Authorization: `Bearer ${apiToken}` } }
);
const signal = await response.json();

Python

response = requests.get(
    "https://api.catalyzed.ai/signals/SigR8I6rHBms3W4Qfa2-FN",
    headers={"Authorization": f"Bearer {api_token}"}
)
signal = response.json()

Linked Resources

Signals can be linked to pipeline executions. View what resources a signal is linked to:

Get linked resources

cURL

curl "https://api.catalyzed.ai/signals/SigR8I6rHBms3W4Qfa2-FN/linked-resources" \
  -H "Authorization: Bearer $API_TOKEN"

TypeScript

const response = await fetch(
  "https://api.catalyzed.ai/signals/SigR8I6rHBms3W4Qfa2-FN/linked-resources",
  { headers: { Authorization: `Bearer ${apiToken}` } }
);
const { resources } = await response.json();

Python

response = requests.get(
    "https://api.catalyzed.ai/signals/SigR8I6rHBms3W4Qfa2-FN/linked-resources",
    headers={"Authorization": f"Bearer {api_token}"}
)
resources = response.json()["resources"]

Response:

{
  "resources": [
    {
      "resourceType": "execution",
      "resourceId": "GkR8I6rHBms3W4Qfa2-FN",
      "resourceLabel": "Document Summarizer #GkR8I",
      "createdAt": "2024-01-15T10:30:00Z",
      "metadata": {
        "pipelineId": "EMbMEFLyUWEgvnhMWXVVa",
        "pipelineName": "Document Summarizer",
        "status": "succeeded",
        "startedAt": "2024-01-15T10:30:02Z",
        "completedAt": "2024-01-15T10:32:15Z"
      }
    }
  ],
  "total": 1,
  "page": 1,
  "pageSize": 20
}

Synthesis Runs Using a Signal

Signals can be used as input to synthesis runs that generate pipeline improvements. View which synthesis runs used a signal:

Get synthesis runs for signal

cURL

curl "https://api.catalyzed.ai/signals/SigR8I6rHBms3W4Qfa2-FN/synthesis-runs" \
  -H "Authorization: Bearer $API_TOKEN"

TypeScript

const response = await fetch(
  "https://api.catalyzed.ai/signals/SigR8I6rHBms3W4Qfa2-FN/synthesis-runs",
  { headers: { Authorization: `Bearer ${apiToken}` } }
);
const { synthesisRuns } = await response.json();

Python

response = requests.get(
    "https://api.catalyzed.ai/signals/SigR8I6rHBms3W4Qfa2-FN/synthesis-runs",
    headers={"Authorization": f"Bearer {api_token}"}
)
synthesis_runs = response.json()["synthesisRuns"]

Response:

{
  "synthesisRuns": [
    {
      "synthesisRunId": "SynR8I6rHBms3W4Qfa2-FN",
      "pipelineId": "EMbMEFLyUWEgvnhMWXVVa",
      "pipelineName": "Document Summarizer",
      "status": "generated",
      "createdAt": "2024-01-15T11:00:00Z",
      "completedAt": "2024-01-15T11:05:00Z"
    }
  ],
  "total": 1,
  "page": 1,
  "pageSize": 20
}

Deleting a Signal

Delete signal

cURL

curl -X DELETE https://api.catalyzed.ai/signals/SigR8I6rHBms3W4Qfa2-FN \
  -H "Authorization: Bearer $API_TOKEN"

TypeScript

await fetch("https://api.catalyzed.ai/signals/SigR8I6rHBms3W4Qfa2-FN", {
  method: "DELETE",
  headers: { Authorization: `Bearer ${apiToken}` },
});

Python

requests.delete(
    "https://api.catalyzed.ai/signals/SigR8I6rHBms3W4Qfa2-FN",
    headers={"Authorization": f"Bearer {api_token}"}
)

Signal Properties

Field	Type	Description
signalId	string	Unique identifier
teamId	string	Team that owns this signal
dataType	string	comment, rating, or evaluation
data	object	Signal data (varies by type)
executionIds	array	Linked pipeline execution IDs
createdAt	timestamp	Creation time

Best Practices

1. Link to executions - Always link signals to the relevant execution(s) for context
2. Be specific in comments - Include details about what was wrong and what would be correct
3. Use ratings consistently - Establish team conventions for rating scales
4. Capture signals promptly - Record feedback while context is fresh
5. Review signals before synthesis - Ensure signals reflect correct expert judgment

Signal Resources

Signals can be linked to various resources (currently pipeline executions, with future support for datasets and files). View linked resources and synthesis runs that used a signal via the sub-resource endpoints:

• Linked Resources - /signals/:signalId/linked-resources - See what resources a signal is linked to
• Synthesis Runs - /signals/:signalId/synthesis-runs - See which synthesis runs used this signal

See the Signal Resources API for complete endpoint documentation.

Next Steps

• Synthesis Runs - Use signals to generate pipeline improvements
• Evaluations - Generate evaluation signals automatically
• Feedback Loops Guide - End-to-end workflow for capturing and using feedback