Classifiers

INT-Output

Response moderation before an assistant message reaches the user.

INT-Output moderates an assistant response before it reaches the user. Give it the assistant text plus the conversation context so it can judge the response in light of what was asked. It is exposed as output.check in the SDKs and POST /v1/output-guard over REST.

What it detects#

Responses that are unsafe to deliver: harmful instructions, leaked secrets or system prompts, or content that violates your safety bar. It also reports whether the model refused. Providing context lets it catch a response that is only unsafe relative to the request.

An output verdict is content moderation, not static or dynamic code analysis. Code, commands, packages, binaries, and build artifacts in an assistant response are not scanned for malware, vulnerabilities, provenance, or execution safety.

Inputs#

Field	Required	Description
`assistant_text`	Yes	The response to moderate.
`user_text`	Recommended	The user message that produced it.
`messages`	Optional	Full conversation as {role, content}. Takes precedence over user_text for context.

Result fields#

Field	Type	Description
`label`	string	One of `Safe`, `Controversial`, `Unsafe`, or `Unknown`.
`severity_score`	number \| null	Severity: 0.0 safe, 0.75 controversial, 1.0 unsafe.
`categories`	string[]	Violated content categories, if any.
`refusal`	string \| null	`"Yes"` or `"No"`. Whether the assistant refused.
`raw_output`	string \| null	REST only: raw moderation-model output for label-parsing diagnostics. SDK callers can inspect it through `raw`.
`latency_ms`	number	Server-side inference time.
`is_safe`	boolean	Convenience: true only when `label` is `Safe`.

Labels & severity#

Label	severity_score	is_safe
`Safe`	0.0	true
`Controversial`	0.75	false
`Unsafe`	1.0	false
`Unknown`	null	false

The SDKs also expose is_refusal (Python) and isRefusal (TypeScript) so you can distinguish a safe refusal from a safe substantive answer.

Example#

out = triage_sdk.output.check(
    assistant_text=model_response,
    user_text=user_message,
    session_id="sess_abc123",
)

if out.severity_score is None or out.severity_score >= 1.0:
    model_response = "I can't help with that."
elif out.severity_score >= 0.75:
    print("response flagged for review")
elif out.is_refusal:
    print("model returned a refusal")

Note. In the gateway, the output guard is disabled by default and enabled per project on the policy screen.