Ingest and reconcile - NDI Platform API

Ingestion is the only call that makes uploaded files searchable. Outcomes are per file: one corrupt document in a folder does not fail the whole run.

Python
TypeScript
curl

from ndi_sdk import NdiClient

with NdiClient() as client:
    workspace = client.workspaces.create(name="ingest-demo")
    client.jobs.wait(
        client.files.upload(
            workspace.workspace_id,
            "report.pdf",
            path="reports/report.pdf",
        ).job_id
    )
    job = client.jobs.wait(
        client.ingestion.ingest(
            workspace.workspace_id,
            path_prefix="reports/",
        ).job_id,
        timeout=600,
    )
    print(job.result)

import { NdiClient } from "ndi-sdk";

const client = new NdiClient();
const workspace = await client.workspaces.create({ name: "ingest-demo" });
await client.jobs.wait(
  (
    await client.files.upload(workspace.workspace_id, "report.pdf", {
      path: "reports/report.pdf",
    })
  ).job_id,
);
const job = await client.jobs.wait(
  (await client.ingestion.ingest(workspace.workspace_id, { path_prefix: "reports/" })).job_id,
  { timeout: 600 },
);
console.log(job.result);

curl -s -X POST "$NDI_BASE_URL/v1/workspaces/$WS/ingestions" \
  -H "X-API-Key: $NDI_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: ingest-reports-v1" \
  -d '{"selection":{"type":"path_prefix","path_prefix":"reports/","stale_only":false}}'

Selection

Pass one of these selectors, or omit both to ingest the whole workspace:

file_ids — named files
path_prefix — a folder (empty prefix = the whole workspace)

stale_only=True narrows a folder run to files whose bytes changed since the last ingest. reingest_unchanged=True rebuilds derivatives even when the hash matches.

Reconcile

Reconcile compares the storage tree to the file ledger. Detection only: discovered, stale, and orphan rows. auto_ingest=True chains an ingestion and returns result.ingestion_job_id.

from ndi_sdk import NdiClient

with NdiClient() as client:
    workspace_id = "550e8400-e29b-41d4-a716-446655440001"
    job = client.jobs.wait(
        client.ingestion.reconcile(workspace_id, auto_ingest=True).job_id
    )
    print(job.result)

Errors

file_not_ingested — a tool ran before ingest finished
invalid_request — both file_ids and path_prefix on one call

See File lifecycle.