Document Sets & Schema Fill

Every endpoint you've seen so far answers a question about a single document: "what's the delivery term in this document?" is scoped to one document_id (see Extraction, Ask (RAG)). In the real world, though, the question is often scoped to a transaction, not a document: "what's the delivery term for this SHIPMENT?" might be answered by the commercial invoice, verified by the freight invoice, but neither one alone is "this transaction" — the question's real subject is a group of documents (a set).

This section covers two primitives: a set (a lightweight resource that groups several documents as one transaction/shipment) and schema fill (given a schema — a list of fields — return the values filled in from the set's documents, plus any conflicts and gaps). Typical use: paste a customs declaration template (30+ fields) and say "fill this schema from this shipment's documents" — see the Use case section below.

What a set is

A DocumentSet is a lightweight resource that groups one or more of your tenant's documents — it has no processing or representation of its own, only a reference list of member documents. The same document can belong to multiple sets (a set is not a batch — unlike POST /v1/batches, which groups documents at upload time, a set groups existing, already-processed documents however you like, after the fact).

Set CRUD

Create a set — POST /v1/sets

curl -X POST https://api.docsfra.com/v1/sets \
  -H "Authorization: Bearer dip_live_a1b2c3d4e5f6..." \
  -H "Content-Type: application/json" \
  -d '{
    "name": "ACME GmbH - INV-2026-0001 shipment",
    "document_ids": ["d_a1b2c3", "d_d4e5f6", "d_a7b8c9"]
  }'

201 Created:

{
  "id": "set_8f3a1b2c",
  "name": "ACME GmbH - INV-2026-0001 shipment",
  "documents": [
    { "id": "d_a1b2c3", "status": "completed", "doc_type": "commercial_invoice" },
    { "id": "d_d4e5f6", "status": "completed", "doc_type": "packing_list" },
    { "id": "d_a7b8c9", "status": "completed", "doc_type": "bill_of_lading" }
  ],
  "created_at": "2026-08-05T09:00:00Z"
}

documents[].doc_type comes from the document's memory layer (memory.doc_type) when it has been generated — null if it hasn't yet. Schema fill doesn't wait for it: candidate gathering runs off the document's canonical markdown even without memory (see the Resolution engine section below).

No existence leakage: if any id in document_ids doesn't exist or belongs to another tenant, the request returns 404 not_found — which case it is isn't distinguishable (see Getting Started → Bearer authentication, same pattern).

Get a set — GET /v1/sets/{id}

curl https://api.docsfra.com/v1/sets/set_8f3a1b2c \
  -H "Authorization: Bearer dip_live_a1b2c3d4e5f6..."

Returns the same shape as the POST response above.

Add documents to a set — POST /v1/sets/{id}/documents

curl -X POST https://api.docsfra.com/v1/sets/set_8f3a1b2c/documents \
  -H "Authorization: Bearer dip_live_a1b2c3d4e5f6..." \
  -H "Content-Type: application/json" \
  -d '{"document_ids": ["d_f1e2d3"]}'

200 OK, returns the updated set body. Idempotent: re-sending a document_id that's already a member doesn't error, it's just a no-op.

Remove a document from a set — DELETE /v1/sets/{id}/documents/{doc_id}

curl -X DELETE https://api.docsfra.com/v1/sets/set_8f3a1b2c/documents/d_f1e2d3 \
  -H "Authorization: Bearer dip_live_a1b2c3d4e5f6..."

204 No Content. The document itself isn't deleted, only its membership in this set — it stays reachable in other sets, or standalone via GET /v1/documents/{id}.

List sets — GET /v1/sets

curl "https://api.docsfra.com/v1/sets?limit=20&offset=0" \
  -H "Authorization: Bearer dip_live_a1b2c3d4e5f6..."

{
  "results": [
    {
      "id": "set_8f3a1b2c",
      "name": "ACME GmbH - INV-2026-0001 shipment",
      "documents": [
        { "id": "d_a1b2c3", "status": "completed", "doc_type": "commercial_invoice" }
      ],
      "created_at": "2026-08-05T09:00:00Z"
    }
  ],
  "limit": 20,
  "offset": 0,
  "total": 1
}

limit (default 20, capped at 100) and offset (default 0) are optional query parameters. Results are returned newest-first by creation date.

Set CRUD errors

The set CRUD endpoints (/v1/sets, /v1/sets/{id}, /v1/sets/{id}/documents) do not require any module entitlement — same pattern as GET /v1/jobs/{id} and /v1/batches: a valid API key is enough (see Polling). Entitlement checks only kick in for the schema-fill endpoints below.

Schema fill (asynchronous)

Schema fill runs against every document in a set and involves (potentially) one LLM micro-call per document — with large schemas (30+ fields) and multiple documents, total time can exceed the 120-second request wall. Like other heavy jobs (upload, cross-check), it therefore runs as a background job: POST queues it and returns right away; you poll the result with a GET.

1. Start a schema-fill run — POST /v1/sets/{id}/extract

curl -X POST https://api.docsfra.com/v1/sets/set_8f3a1b2c/extract \
  -H "Authorization: Bearer dip_live_a1b2c3d4e5f6..." \
  -H "Content-Type: application/json" \
  -d '{
    "schema": {
      "fields": [
        {
          "key": "delivery_term",
          "label": "20. Delivery Term",
          "hint": "Incoterms (EXW, FOB, CIF, etc.)",
          "primary_docs": ["commercial_invoice"],
          "verify_docs": ["freight_invoice", "insurance_certificate"],
          "type": "text"
        }
      ]
    },
    "language": "en"
  }'

Validation errors (see Errors below) are returned immediately on this call. Otherwise the response is 202 Accepted:

{ "run_id": "sxr_4d9e2f1a" }

language is optional (ISO 639-1 code) and only affects the language of the field-filling instructions given to the LLM — values/quotes taken from the source documents always stay in the document's own language; if omitted, it falls back to the tenant's Console → Models default (see Extraction → Output language, same pattern).

2. Poll for the result — GET /v1/sets/{id}/extract/{run_id}

status moves through queued → running → completed (or failed). Poll this endpoint (e.g. every 2-3 seconds) until it reaches a terminal state — GET calls are never billed.

curl https://api.docsfra.com/v1/sets/set_8f3a1b2c/extract/sxr_4d9e2f1a \
  -H "Authorization: Bearer dip_live_a1b2c3d4e5f6..."

While in progress:

{ "status": "running", "result": null, "error": null }

Once completed, result is filled in (see the Result format section below):

{
  "status": "completed",
  "result": {
    "filled": { "delivery_term": { "value": "CIF", "confidence": 0.95, "sources": [] } },
    "conflicts": [],
    "missing": [],
    "stats": { "fields_total": 1, "filled": 1, "conflicts": 0, "missing": 0,
               "documents_used": 3, "llm_calls": 3, "duration_ms": 5200 }
  },
  "error": null
}

If failed, result stays null and error is filled in ({"code", "message"}) — an isolated issue on a single field/document (e.g. one document's LLM call times out) doesn't fail the whole run; candidates from that document simply stay missing for that field, and the run still resolves to completed. failed only happens when the run couldn't be started at all (e.g. the LLM was unreachable for every document).

3. List past runs — GET /v1/sets/{id}/extract

curl https://api.docsfra.com/v1/sets/set_8f3a1b2c/extract \
  -H "Authorization: Bearer dip_live_a1b2c3d4e5f6..."

{
  "results": [
    { "run_id": "sxr_4d9e2f1a", "status": "completed", "created_at": "2026-08-05T09:05:00Z" }
  ]
}

Newest-first list of run summaries (call the GET .../extract/{run_id} endpoint above with the matching run_id for the full result).

Schema-fill errors

A failure that happens after the run has started doesn't produce an HTTP error on POST — it surfaces as status: "failed" with an error object on a later GET (see Ask (RAG) → Cross-check, same async pattern).

Schema format

The schema is accepted in two shapes: canonical (used as-is) and customs format (automatically converted to canonical on input, no work required on your part). Both are sent in the same schema field — the shape is auto-detected (a fields key means canonical, a baslik_alanlari key means customs format).

Canonical format

{
  "fields": [
    {
      "key": "delivery_term",
      "label": "20. Delivery Term",
      "hint": "Incoterms (EXW, FOB, CIF, etc.)",
      "primary_docs": ["commercial_invoice"],
      "verify_docs": ["freight_invoice", "insurance_certificate"],
      "type": "text"
    }
  ]
}

When primary_docs/verify_docs are left empty, that field can gather candidates from every document in the set (no doc-type restriction) — for how the comparison actually plays out, see the Resolution engine section below.

Customs format (auto-converted)

Grouped, Turkish-keyed customs declaration templates can also be sent as-is — the conversion happens server-side, no work required on your part:

{
  "baslik_alanlari": {
    "teslim_odeme": {
      "alanlar": {
        "teslim_sekli": {
          "etiket": "20. Delivery Term",
          "aciklama": "Incoterms (EXW, FOB, CIF, etc.)",
          "belge_birincil": ["commercial_invoice"],
          "belge_dogrulama": ["freight_invoice", "insurance_certificate"],
          "kaynak_tanimi": "belge"
        },
        "doviz_kuru": {
          "etiket": "23. Exchange Rate",
          "aciklama": "Official rate on the customs declaration date",
          "belge_birincil": [],
          "belge_dogrulama": [],
          "kaynak_tanimi": "operator input"
        }
      }
    }
  },
  "mikro_cagri": true
}

Path: baslik_alanlari.<group>.alanlar.<key>. This example produces one field once converted to canonical:

{
  "fields": [
    {
      "key": "teslim_sekli",
      "label": "20. Delivery Term",
      "hint": "Incoterms (EXW, FOB, CIF, etc.)",
      "primary_docs": ["commercial_invoice"],
      "verify_docs": ["freight_invoice", "insurance_certificate"],
      "type": "text"
    }
  ]
}

Mapping: etiket → label, aciklama → hint, belge_birincil → primary_docs, belge_dogrulama → verify_docs. The <group> level doesn't survive into the result — the result body always uses a flat key -> ... map (see Result format below); grouping is purely part of your template's layout.

doviz_kuru, on the other hand, never makes it into fields — see the next section for why.

Operator fields never reach the LLM

A field whose kaynak_tanimi contains the words "operator" or "automatic" (operator/otomatik) is treated as a field that isn't expected to be extractable from a document at all (filled in by hand by an operator, or populated automatically by another system). These fields:

• are never sent to the LLM (not during candidate gathering, or anywhere else) — no micro-call is wasted on them,
• are listed directly in the result body under missing as {"key": "doviz_kuru", "reason": "operator_input"}.

This is deliberately a distinct reason from "not found in the documents" (not_found) — you can handle the two differently client-side (one means "go check the documents again", the other means "an operator will fill this in").

Unknown keys

Any key not recognized in either the canonical or customs shape (like mikro_cagri in the example above) is silently ignored — it never produces an error. You can safely carry your own template metadata (version numbers, internal notes, etc.) inside the schema.

Result format

{
  "filled": {
    "delivery_term": {
      "value": "CIF",
      "confidence": 0.95,
      "sources": [
        { "document_id": "d_a1b2c3", "doc_type": "commercial_invoice", "page": 1, "quote": "Delivery term: CIF Rotterdam" }
      ]
    }
  },
  "conflicts": [
    {
      "key": "package_count",
      "values": [
        { "value": "12", "sources": [{ "document_id": "d_a1b2c3", "doc_type": "commercial_invoice", "page": 1, "quote": "12 packages" }] },
        { "value": "14", "sources": [{ "document_id": "d_d4e5f6", "doc_type": "packing_list", "page": 1, "quote": "14 packages" }] }
      ]
    }
  ],
  "missing": [
    { "key": "exchange_rate", "reason": "operator_input" },
    { "key": "payment_terms", "reason": "not_found" }
  ],
  "stats": {
    "fields_total": 30,
    "filled": 21,
    "conflicts": 2,
    "missing": 7,
    "documents_used": 9,
    "llm_calls": 9,
    "duration_ms": 48000
  }
}

filled

A key -> {value, confidence, sources[]} map. Only fields for which a single, consistent value was found land here — see the What confidence scores mean section below for exactly which situation produces which score. sources[] carries, for every document that supports the value, its document_id, doc_type, page, and the quote where the value actually appears — quote is a verified (never fabricated) exact excerpt from the source document (see Anti-hallucination shield below).

conflicts

If a field has two or more values that are still different after normalization, the field does NOT land in filled — instead it's listed under conflicts with its key and each distinct value found for it, along with that value's own sources[]. This is deliberate: schema fill never decides on its own which value is "correct" — it surfaces the conflict as-is (see package_count in the example above: the invoice says 12, the packing list says 14 — both stay, neither is silently picked).

missing

A list of {key, reason} pairs. reason is one of two values:

stats

What confidence scores mean

confidence, on every value under filled, is a fixed scale reflecting how cross-verified that value is — it is not a self-reported "how sure am I" score from the model, it's a decision computed server-side by plain code (no LLM involved):

If two or more different values are found (still not equal after normalization), the field doesn't get a confidence at all — it falls into conflicts instead (see above). confidence only measures the number/kind of sources behind a value, it does not guarantee the value is correct — flagging low-confidence (0.5) fields for review client-side is recommended.

Resolution engine (deterministic-first)

Schema fill is engineered to keep the LLM to a minimum and let plain code make every decision — the same philosophy proven in line-item extraction (see Extraction → Line items).

Candidate gathering — one LLM call per document

Schema fill calls the LLM per document, not per field: instead of a 9-documents x 30-fields = 270-call disaster, it makes exactly one call per document — the document's canonical markdown (within length limits) plus the relevant subset of fields for that document's doc_type (fields where this document appears in primary_docs/ verify_docs, or fields with no doc-type restriction) are given to the model together; the model returns {value, quote, page?} for each relevant field. This is why stats.llm_calls scales with documents_used, not the number of fields.

Anti-hallucination shield

Every candidate the model returns is searched for in the source document's normalized text before being accepted: if the candidate's quote doesn't actually appear in that document, the candidate is dropped — no value reaches filled/conflicts without being verified against the source text. The page number is also derived deterministically from wherever the quote was actually found, not from the model's claim.

Normalization

Comparisons run over normalized values: whitespace/case-insensitive comparison; numbers reconcile European (1.234,56) vs US (1,234.56) formatting; dates are converted to ISO 8601 (dates that can't be converted are compared as-is). v1 has no country code/name mapping ("TR" and "Turkey" count as different values, same as line items) — it's a raw comparison.

Decision rules

Once normalized candidates are gathered, the decision is made entirely in plain code (the LLM is never called a second time):

1. A value from primary_docs + the same (normalized) value from verify_docs → filled, confidence: 0.95.
2. A value from primary_docs only → filled, confidence: 0.7.
3. A value from verify_docs/other documents only → filled, confidence: 0.5.
4. 2+ different values after normalization → conflicts (the field doesn't land in filled).
5. No candidates at all → missing, reason: "not_found" (or directly reason: "operator_input", without ever calling the LLM, if kaynak_tanimi was operator/automatic).

LLM calls go through resolve_lane(tenant, "structuring"), the same model-selection/BYOK infrastructure as everywhere else (see Getting Started → Model selection & BYOK) — no separate model lane is invented for schema fill.

Auth and entitlement

The set CRUD endpoints only require a valid Authorization: Bearer <api_key> header (see above). The schema-fill endpoints (POST/GET /v1/sets/{id}/extract*) are additionally gated by the extract module entitlement — a key not authorized for extract gets 403 service_not_enabled (see Getting Started → Entitlement).

Billing

Schema fill is billed at SET_EXTRACT_RATE, defaulting to 0.05 credits x the number of documents in the set:

For example, on a 9-document set a run costs 9 x 0.05 = 0.45 credits — this is independent of the number of fields in the schema (a 30-field schema and a 3-field schema cost the same on the same document count). Balance is checked before the request (402 insufficient_credit on POST); the charge is taken once, when the run completes successfully — polling with GET is always free, and a run that ends in status: "failed" is never charged (the same "no charge for a failed synthesis" pattern as /v1/ask, see Ask (RAG) → Credits and billing). dip_test_... keys are still measured for this call, but not deducted from balance.

Use case: customs declaration preparation

Scenario: 6 documents for an ACME GmbH shipment (commercial invoice, packing list, bill of lading, freight invoice, insurance certificate, certificate of origin) have already been uploaded and processed. Goal: fill a customs declaration template's fields from these documents to produce a "declaration-ready" output for a human operator to review.

1. Group the shipment's documents into a set:

curl -X POST https://api.docsfra.com/v1/sets \
  -H "Authorization: Bearer dip_live_a1b2c3d4e5f6..." \
  -H "Content-Type: application/json" \
  -d '{
    "name": "ACME GmbH - INV-2026-0001",
    "document_ids": ["d_a1b2c3", "d_d4e5f6", "d_a7b8c9", "d_c3d4e5", "d_e5f6a7", "d_f6a7b8"]
  }'

→ set_8f3a1b2c.

2. Paste the customs declaration template as-is (grouped, customs format with kaynak_tanimi, 32 fields) and start schema fill:

curl -X POST https://api.docsfra.com/v1/sets/set_8f3a1b2c/extract \
  -H "Authorization: Bearer dip_live_a1b2c3d4e5f6..." \
  -H "Content-Type: application/json" \
  -d '{"schema": { "baslik_alanlari": { "...": "..." } }, "language": "en"}'

→ 202 Accepted, {"run_id": "sxr_4d9e2f1a"}.

3. Poll every 2-3 seconds; once completed, fetch the result:

curl https://api.docsfra.com/v1/sets/set_8f3a1b2c/extract/sxr_4d9e2f1a \
  -H "Authorization: Bearer dip_live_a1b2c3d4e5f6..."

4. Declaration-ready output:

{
  "status": "completed",
  "result": {
    "filled": {
      "delivery_term": { "value": "CIF", "confidence": 0.95, "sources": ["..."] },
      "origin_country": { "value": "DE", "confidence": 0.7, "sources": ["..."] }
    },
    "conflicts": [
      {
        "key": "package_count",
        "values": [
          { "value": "12", "sources": [{ "document_id": "d_a1b2c3", "doc_type": "commercial_invoice", "page": 1, "quote": "12 packages" }] },
          { "value": "14", "sources": [{ "document_id": "d_d4e5f6", "doc_type": "packing_list", "page": 1, "quote": "14 packages" }] }
        ]
      }
    ],
    "missing": [
      { "key": "exchange_rate", "reason": "operator_input" },
      { "key": "hs_code_note", "reason": "not_found" }
    ],
    "stats": { "fields_total": 32, "filled": 24, "conflicts": 2, "missing": 6,
               "documents_used": 6, "llm_calls": 6, "duration_ms": 34500 }
  }
}

The operator's workflow becomes clear: 24 fields can be transferred straight into the declaration form (with confidence scores flagging which ones deserve a second look), the package_count conflict (invoice says 12, packing list says 14) is routed to human review, exchange_rate is already flagged as a field the operator will fill in (never searched for in any document), and hs_code_note sits in its own bucket because it genuinely couldn't be found in the documents.

Notes

Schema fill is a composition layer built on top of the existing extraction layers (memory, structured objects, entities, reasoning — see Extraction); it doesn't produce its own separate document representation. If one of a set's documents hasn't reached completed yet, or has no canonical markdown, that document simply contributes no candidates for that run — it doesn't fail the run as a whole.