Skip to content
Docs
Comic API
On this page

Developer guides

Moderation

Screen user-provided text with the content moderation API before generation.

Add content moderation to your generation workflow with a single API call. Submit text from your server, receive a moderation result, and use it to decide whether the request can continue.

Why moderation matters

Checking user input before generation helps keep rejected content out of your workflow and avoids spending generation resources on it. A server-side check also gives every entry point the same rule: generation starts only after moderation succeeds.

Content moderation, built into your workflow

Use the API as a server-side checkpoint before content reaches your model. Your application handles the result, communicates the next step to the user, and applies any additional safeguards your product requires.

How it works

Your server submits the user-controlled content and uses the result to decide whether the generation request can continue.

  1. A user submits content

    Collect the prompt and dialogue your generation job will use.

  2. Your server calls the moderation endpoint

    Send the input to POST /api/safety/content before queueing a job, charging for generation, or invoking a model.

  3. The API returns a result

    HTTP 200 with code: 200 and safe: true confirms that the submitted input passed this check.

  4. Your application continues or stops

    Generate only on that verified success. For any other result, stop the request and show an appropriate message.

The endpoint

Submit text in a JSON body. No API key required.

POSThttps://llamagen.ai/api/safety/contentContent-Type: application/json

Use llamagen.ai for moderation, not the Comic generation API host, api.llamagen.ai. Authentication for generation remains separate.

Request

Moderation request fields
FieldTypeRequiredDescription
textsstring[]Yes¹Text inputs to screen. Up to 20 items and 200,000 characters in total.

¹ For the text-only requests in this guide, provide at least one nonblank string in texts. Do not send an empty array or null.

  • Text is trimmed before validation. Blank strings are rejected. Character counts use JavaScript string length (UTF-16 code units); some emoji count as two.
  • Extra fields outside the endpoint's schema are rejected.

Response

Moderation response fields
FieldTypeDescription
codenumberThe application status, matching the HTTP status for endpoint responses: 200, 400, or 423.
safebooleanPresent as true on a successful response. It is not returned as false for rejected requests.
messagestringPresent on error responses. Describes a validation issue, rejection, or failed check. Do not use this free-form text for routing.

One result covers the whole request. Any rejected item or failed check blocks the request; no per-item decisions are returned.

200 · Successful response
{
  "code": 200,
  "safe": true
}

Check the HTTP status together with the response body. A successful result applies to the submitted request as a whole.

Status & next steps

Moderation status and recommended action
HTTP statusMeaningRecommended action
200Check passedContinue only when the body also contains code === 200 and safe === true.
400Invalid requestFix malformed JSON, unsupported fields, invalid inputs, or exceeded limits. Do not generate or retry unchanged input.
423Not clearedStop generation. Content was rejected, or the check could not complete. Do not assume this proves a policy violation.
Other / no responseCheck unavailableStop on timeouts, network errors, 5xx, invalid JSON, or an unexpected body. Offer a later retry.
423 is deliberately fail-closed

A 423 response means the request has not been cleared, whether moderation rejected it or the check could not complete. Never use a failed check as permission to generate. Error messages may include submitted text; do not display or log the raw response.

Test with an AI assistant

Copy this self-contained SKILL.md into your coding assistant and ask it to test the content moderation API. It provides the request contract, a small synthetic-text probe, and instructions for checking failure handling without calling a generation model.

llamagen-moderation / SKILL.md
View SKILL.md
---
name: llamagen-moderation
description: Test LlamaGen's text moderation API and verify fail-closed handling in a developer's generation workflow. Use when asked to test this endpoint or review its integration.
---

# LlamaGen content moderation

Validate the content moderation integration without invoking generation.
Reading or copying this skill does not authorize a live request: run the probe
only when the developer asks to test the endpoint. Do not change application
code, deploy, or add credentials unless that work is separately requested.

## Request and response contract

- POST https://llamagen.ai/api/safety/content
- Content-Type: application/json. No API key is required for this endpoint.
- Text-only body: {"texts":["A watercolor lighthouse at sunset."]}.
- Send 1–20 nonblank strings, at most 200,000 UTF-16 code units in total after
  trimming. An empty array or null is not a valid text-only request.
- Continue only for HTTP 200 with numeric code: 200 and boolean safe: true.
- HTTP 400 means an invalid request. HTTP 423 means not cleared; it may also
  mean the check could not complete. Neither permits generation.
- Stop on all other statuses, timeouts, network failures, redirects, malformed
  JSON, or unexpected response bodies. Never route on free-form message text.

## Bounded live probe

Use Node.js 20+ and run the following as a JavaScript module. It sends at most
two sequential requests to the production moderation service, with no retries.
It uses only fixed synthetic text and stops if the first check fails.
Never substitute customer data, private prompts, or credentials. Do not probe
moderation rules or try to reverse-engineer decisions; use mocks for failures.

```javascript
const endpoint = 'https://llamagen.ai/api/safety/content';

async function probe(label, payload, matches) {
  try {
    const response = await fetch(endpoint, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify(payload),
      signal: AbortSignal.timeout(10_000),
      redirect: 'error',
    });
    const result = await response.json();
    const passed = matches(response.status, result);
    console.log(label + ': ' + (passed ? 'PASS' : 'FAIL') +
      ' (HTTP ' + response.status + ')');
    return passed;
  } catch {
    console.log(label + ': FAIL (request unavailable)');
    return false;
  }
}

const passed = await probe(
  'Text request',
  { texts: ['A watercolor lighthouse at sunset.'] },
  (status, result) =>
    status === 200 && result?.code === 200 && result?.safe === true,
);

if (!passed) {
  process.exitCode = 1;
} else {
  const invalid = await probe(
    'Validation request',
    { texts: [] },
    (status, result) => status === 400 && result?.code === 400,
  );
  if (!invalid) process.exitCode = 1;
}
```

## Verify the application separately

A passing probe does not prove that an application's generation gate is correct.
If integration testing is requested, use mocked responses to verify that only
HTTP 200 with code: 200 and safe: true can reach the generation adapter.
Cover 400, 423, 5xx, timeout, network failure, invalid JSON, missing safe,
safe: false, and safe: "true"; every case must stop before generation.
Do not invoke a real model or infer internal moderation rules from these tests.

## Report

Report pass/fail, HTTP status, the endpoint tested, and any blocked checks.
Keep raw responses and submitted content out of logs and user-facing errors.
Distinguish live endpoint checks from mocked application checks, and never
claim a live check ran if network access or authorization was unavailable.

Documentation: https://llamagen.ai/comic-api/docs/moderation

Copying does not run a test. The live probe requires Node.js 20+ and sends at most two requests after you ask your assistant to run it.

Integration guide

1. Add moderation to your generation handler

We recommend checking user-controlled input in your server-side handler, before creating a generation job or calling a model. The following examples show how your application can stop generation when the check does not pass.

The helpers below return only when the check passes. If they throw, catch the error at your request boundary and return without calling your generation function.

type ModerationInput = {
  texts: string[];
};

async function assertSafe(input: ModerationInput): Promise<void> {
  let response: Response;
  let result: unknown;

  try {
    response = await fetch('https://llamagen.ai/api/safety/content', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify(input),
      signal: AbortSignal.timeout(10_000),
      redirect: 'error',
    });
    result = await response.json();
  } catch {
    // Network errors, timeouts, and invalid JSON never permit generation.
    throw new Error('moderation_unavailable');
  }

  if (response.status === 400) {
    throw new Error('moderation_invalid_request');
  }
  if (response.status === 423) {
    // May mean risky content OR a failed check. Do not guess which.
    throw new Error('moderation_not_cleared');
  }
  if (
    response.status !== 200 ||
    typeof result !== 'object' ||
    result === null ||
    !('code' in result) ||
    result.code !== 200 ||
    !('safe' in result) ||
    result.safe !== true
  ) {
    throw new Error('moderation_unavailable');
  }
}

// Call from your server before queueing, charging, or generating.
// If assertSafe throws, return a friendly error and STOP this request.
// await assertSafe({ texts: [userPrompt] });
// Only after it resolves: await yourExistingGenerationFunction(...);

TypeScript uses built-in fetch in Node.js 20+. Python uses the standard library. The cURL tab is a request probe; it does not implement your application's generation gate.

2. Use the correct endpoint

Send live checks to https://llamagen.ai/api/safety/content. There is no API key to configure for this endpoint.

Use mocked responses to test routing during development. If you run the existing endpoint locally, point your test scripts at that deployment. Verify the production URL from your deployed server before going live.

3. Treat 423 as a block

A 423 result means the input has not been cleared. Do not forward it to your model, automatically retry every 423, or infer a category from the error message.

Keep the user-facing message neutral: “We couldn't clear this content for generation. Please revise your input or try again later.”

4. Fail closed, not open

Network errors, timeouts, invalid JSON, and unexpected responses must also stop generation. Only an explicit successful HTTP status and response body permit the request to continue.

The examples use a 10-second timeout. This is an application choice, not a service latency guarantee. Adjust it for your workload without introducing a fallback that skips moderation.

5. Screen the input before generation

Submit the exact user-controlled text that will reach your model. Screen updated content again, and apply the same rule to edits, retries, and background jobs.

Output checks can be an additional safeguard; they do not replace screening before a generation job starts. Browser-only checks are not sufficient because clients can bypass them.

End-to-end example

A complete, prompt-only Next.js route showing input validation, the moderation call, blocked and unavailable responses, and the final handoff to your generation adapter.

app/api/generate/route.ts
import { NextRequest, NextResponse } from 'next/server';
// Replace this import with your application's existing generation adapter.
import { generateImage } from '@/lib/generate-image';

export async function POST(req: NextRequest) {
  // 1. Validate input before contacting any external service.
  let input: unknown;
  try {
    input = await req.json();
  } catch {
    return NextResponse.json({ error: 'invalid_json' }, { status: 400 });
  }

  if (
    typeof input !== 'object' ||
    input === null ||
    !('prompt' in input) ||
    typeof input.prompt !== 'string' ||
    !input.prompt.trim() ||
    input.prompt.length > 200_000
  ) {
    return NextResponse.json({ error: 'invalid_prompt' }, { status: 400 });
  }
  const prompt = input.prompt;

  // 2. Screen the exact prompt before queueing, billing, or generating.
  try {
    const response = await fetch('https://llamagen.ai/api/safety/content', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ texts: [prompt] }),
      signal: AbortSignal.timeout(10_000),
      redirect: 'error',
    });

    if (response.status === 400) {
      return NextResponse.json(
        { error: 'moderation_invalid_request' },
        { status: 400 },
      );
    }
    if (response.status === 423) {
      // A rejection OR a failed check. Never expose the raw API message.
      return NextResponse.json(
        {
          error: 'moderation_not_cleared',
          message: 'We could not clear this content. Revise it or try later.',
        },
        { status: 423 },
      );
    }

    const result: unknown = await response.json();
    if (
      response.status !== 200 ||
      typeof result !== 'object' ||
      result === null ||
      !('code' in result) ||
      result.code !== 200 ||
      !('safe' in result) ||
      result.safe !== true
    ) {
      throw new Error('Unexpected moderation result');
    }
  } catch {
    // Fail closed on a timeout, network error, or malformed response.
    return NextResponse.json(
      { error: 'moderation_unavailable', message: 'Please try again later.' },
      { status: 503 },
    );
  }

  // 3. Only a verified success can reach your existing generation code.
  try {
    const image = await generateImage(prompt);
    return NextResponse.json({ image });
  } catch {
    return NextResponse.json({ error: 'generation_failed' }, { status: 502 });
  }
}

This is code for your application, not an additional LlamaGen endpoint. Replace the generateImage import with your existing adapter and preserve your own authentication, rate limits, and billing controls. The request to your route uses prompt; the moderation request translates it to texts: [prompt].

Verify your integration is live

Run a harmless request from the environment where your generation handler runs. Confirm that you receive HTTP 200 and the exact positive fields, then verify a validation failure.

Terminal · Successful request
curl --include --max-time 30 \
  'https://llamagen.ai/api/safety/content' \
  -H 'Content-Type: application/json' \
  -d '{"texts":["A watercolor lighthouse at sunset."]}'
Terminal · Invalid request
curl --include --max-time 30 \
  'https://llamagen.ai/api/safety/content' \
  -H 'Content-Type: application/json' \
  -d '{"texts":[]}'

# Expected: HTTP 400
# {"code":400,"message":"Invalid moderation request"}

Finally, submit a request through your own application. Check that moderation completes before your generation adapter is called. A successful standalone probe does not verify your application's call order.

Checklist before going live

Use mocked responses to test failure paths without depending on a live classifier's decision.

  • Every route to your generation model performs a server-side moderation check.
  • HTTP 200 with code: 200 and safe: true reaches your generation adapter exactly once.
  • HTTP 400 and 423 never reach generation, and raw error messages are not exposed.
  • Timeouts, network failures, 5xx, and malformed JSON all block generation.
  • Missing safe, safe: false, and safe: "true" never permit generation.
  • Text requests work from your deployed server.

This guide does not establish pricing, rate limits, an availability SLA, or compliance with a third-party provider's moderation requirements.

Search this guide