Skip to main content
Customer storage is not generally available. This guide applies to deployments where the feature has been enabled after qualification. Disabled deployments refuse new connections and external workspaces with 501 not_implemented; incomplete enabled deployments return 503 storage_unavailable. A storage connection authorizes NDI to read and write an existing destination. Select a validated connection when creating a workspace. Omitting it keeps the deployment’s default storage, and existing workspaces do not move. The initial providers are AWS S3 with an IAM role and Azure Blob Storage with a Microsoft Entra application secret. Custom endpoints, S3-compatible services, account keys and SAS credentials are not accepted by this connection API. Only the AWS commercial partition (arn:aws: role ARNs) and Azure public cloud are supported.

​
Choose the destination

Create the bucket or container in your cloud account first. NDI creates neither resource. Use a dedicated destination or base prefix that permits read, write, list and delete operations. The prefix is relative, such as customer-ndi, with no leading or trailing slash. NDI allocates a fresh namespace for each workspace; it does not adopt or index objects already in the bucket. For base prefix customer-ndi, the reserved layout is:
Use NDI’s upload/import APIs to register files. Keep other applications and lifecycle rules from changing NDI’s ownership marker, workspace objects or active scratch objects. Workspace contents include derived and internal files, not just the uploads visible in the console.

​
AWS S3

  1. Create a connection with a name and a destination of type: "s3" with bucket, region and optional prefix. The draft response supplies aws_trusted_principal_arn and a connection-specific external_id.
  2. Create an IAM role in your account. Set its trust policy using those exact response values:
The external ID identifies this connection; use the value issued by NDI rather than an account name or a reused value. Both API and worker validation reject a role that can be assumed without that ID or with near misses of it (one character changed or appended). Use the exact StringEquals condition shown above; these probes do not prove every possible trust policy safe. Expect denied AssumeRole attempts from NDI in CloudTrail during validation. The role must belong to your AWS account, outside NDI’s account; NDI-owned buckets cannot be connected as customer storage. AWS recommends an external-ID condition for third-party access. See AWS’s cross-account role guidance.
  1. Attach a permissions policy to that role. This example grants access to NDI’s allocations beneath customer-ndi in YOUR_BUCKET; replace both values to match the connection:
For an empty connection prefix, use arn:aws:s3:::YOUR_BUCKET/ndi/* for objects. Grant s3:ListBucket without an s3:prefix condition: NDI treats a missing object as normal, and S3 reports a missing key as 404 rather than 403 only when the role can list the bucket. Validation checks a deleted probe object’s metadata and refuses a grant that reports that missing object as an authorization failure. Bucket-wide listing reveals object names outside NDI’s prefix, so use a dedicated bucket when that matters. Bucket policies and organization policies must also permit the role. Encryption with a customer-managed KMS key may require additional key permissions. See S3’s operation-to-permission mapping and HeadObject’s 404/403 rule. Automatic cleanup lists incomplete multipart uploads under the job’s scratch and abandoned, unselected publication attempts, aborts them, and checks their remaining parts after writers settle. Selected publication attempts are preserved. Workspace deletion covers the complete NDI-owned allocation. These operations require s3:ListBucketMultipartUploads, s3:AbortMultipartUpload, and s3:ListMultipartUploadParts; see AWS multipart cleanup. Validation checks unfinished-upload listing; it does not prove abort or part-listing permission. Missing permissions can prevent cleanup. If you configured an AWS role from an earlier policy, add the multipart permissions and revalidate the connection before creating customer workspaces. Historical versions follow your storage policy.
  1. Set the connection credential to type: "aws_role" with your new role’s role_arn. NDI assumes it using temporary credentials; do not supply IAM access keys.
  2. Validate the connection and wait for its validation job to succeed. Confirm that the connection reports ready before creating a workspace.
The trust policy and permissions policy serve different purposes: the first allows NDI to assume the role for this connection; the second limits what that role can do in your bucket.

​
Azure Blob Storage

  1. Create a Blob container and register an application in your Microsoft Entra tenant. Record its tenant ID, application client ID and client secret value.
  2. Assign the application’s service principal Storage Blob Data Contributor, scoped to the chosen container. This grants blob read, write and delete access. Azure’s management-plane Contributor role alone does not grant blob data access. See Azure blob authorization.
  3. Create an NDI connection with a name and a destination of type: "azure_blob" with account (the storage account name), container and optional relative prefix. Stage type: "azure_service_principal" credentials containing tenant_id, client_id and client_secret.
  4. Validate and wait for the validation job to succeed, then confirm the connection is ready.
Container-scoped RBAC covers every blob in that container. The connection prefix limits NDI’s destination; it does not narrow the application’s Azure role assignment. Use a dedicated container when that is the required cloud access boundary. Role assignment changes can take time to propagate; see Azure’s container-scope instructions. Cleanup includes uncommitted blobs and uses the normal blob delete permission, included in Storage Blob Data Contributor. Historical versions and snapshots follow your storage policy. Supply the secret through the authenticated console credential form or an SDK using your secret manager or a hidden prompt. It is a write-only input. Do not include it in command-line arguments, chat prompts or logs.

​
Validate and create the workspace

Validation writes and removes temporary probe objects and checks that both the API and background worker can access the destination. The API probes before it starts the validation job, so a misconfigured destination can return storage_auth_required or storage_unavailable from the validate request itself, with no job. A failed validation may leave probe objects under ndi/probes/. A successful credential upload alone does not make a connection ready. The destination must be reachable from both service components. If the bucket or storage account restricts network access, get the addresses to allow from the deployment operator before validating. In the console, use Connect storage in the workspace creation dialog, or manage connections in Settings. Select the ready connection under Workspace storage, then create the workspace. Connections that are still draft, degraded or revoked cannot be selected. Storage selection is fixed at creation. Connections can be attached only within their creator’s scope. A connection created by a console member belongs to that member’s console session; one created for an external account belongs to callers for that account. Connections created by console owners, admins or tenant API keys are shared by those callers, but not by members’ console sessions. Administrators can inspect and revoke connections they cannot attach. Members can mint tenant API keys, so use a separate tenant if its destinations must be inaccessible to them. Create the connection with the identity that will create the workspaces. For programmatic setup, follow the Python storage connection example or TypeScript example. Create the workspace with storage: { type: "external", connection_id: "<connection-id>" }. Keep one idempotency key per creation request when recovering a lost response across process restarts.

​
Reuse Parse results

A job that reuses a Parse result copies its artifacts into the selected workspace’s scratch before processing. Its published copies remain available after the producing job expires. Reuse requires an available result; a customer-storage producer without a publication receipt returns storage_conflict. Reuse does not reopen an original HTTP URL. Split, Extract and Ground need retained document Markdown or a readable original file. A URL-backed spreadsheet without that input returns result_expired, even while its Parse result remains available. Another Parse can still copy the available result.

​
Rotate, revoke and delete

Stage replacement credentials and validate them before retiring the old cloud identity. A failed rotation leaves the previous active identity selected. For a revoked connection, validation must set reactivate: true. After administrative revocation or member offboarding, the connection reports reactivation_requires_approval and a console owner or admin must call allow-reactivation first; approval itself does not restore access. Revoking a connection stops subsequent workspace access; it does not delete customer objects or cancel cloud requests already in flight. Published customer objects remain after job payload expiry, whether the connection is ready or revoked. NDI needs restored access to finish pending scratch or abandoned-attempt cleanup. Reconnecting restores access to retained content but does not renew expired job payloads or artifact links. Check the job’s result_error when operational job status remains available but content is withheld. Signed downloads for customer-workspace files and document artifacts recheck storage access when opened and stop serving after revocation or deletion. Document artifact links also stop serving after job cancellation or expiry, even if their URL has not expired. A transfer already in progress cannot be recalled. Document cleanup runs after the job’s payload expires, or its zero-retention delivery window ends, and after dependent jobs release their holds. It removes temporary processing scratch once in-flight writes settle. A separate cleanup pass removes abandoned, unselected publication attempts; published customer objects and their integrity receipts survive job payload expiry. Deleting objects directly in your cloud account does not close NDI’s pending cleanup record: restore access so NDI can verify the deletion, or complete verified workspace deletion. Deleting a workspace removes its NDI-owned workspace and scratch namespaces. Cleanup uses the connection, so delete workspaces before revoking it, and keep valid cloud authorization available until cleanup completes. If the connection is revoked, the workspace_delete job fails with storage_auth_required and the workspace stays deleting; reactivate the connection and send DELETE again. Cleanup is complete only when the workspace_delete job succeeds with result.verified: true. A failed cleanup needs revalidation and another DELETE; changing credentials does not migrate the workspace to a new destination. Deleting a connection is refused while any workspace binding or pending creation attempt remains. Verified workspace cleanup releases its binding, so delete the connection after all its workspace deletions complete. Cleanup waits for in-flight writes to settle. A failed or interrupted write whose outcome is uncertain can delay cleanup until its upload deadline and provider retry window have passed. There is no endpoint to release a pending attempt. Complete it by retrying the creation with its original idempotency key; otherwise keep the connection ready and wait for automatic cleanup of attempts abandoned for more than a day. A failed creation sent without an idempotency key can only wait for that cleanup. NDI keeps the staged credential while the connection exists, and a revoked connection can be reactivated from NDI. To end NDI’s access permanently, delete its workspaces and wait for verified cleanup, revoke the connection, then remove access in your cloud account: delete the IAM role or its trust statement, or delete the Entra client secret or role assignment. Removing the trust statement or the client secret stops new credentials but does not invalidate sessions or tokens already issued, which stay valid until they expire. For a faster cutoff, remove the role’s permissions policy or revoke its active sessions in AWS, or remove the Azure role assignment, allowing for propagation.

​
Retention scope

Customer storage selects where durable workspace bytes live. Temporary processing, database metadata, search indexes, job history and service logs retain their existing placement. This is not a service-wide zero-retention guarantee. External workspaces reject both source and derived TTLs. NDI automatic retention preserves durable source files, derivatives and published results in customer storage; you control their cloud lifecycle. Workspace deletion removes current NDI-owned objects, but cloud versioning, soft deletion, backups and retention policies can keep historical bytes in your account. Configure those policies according to your retention requirements; NDI workspace deletion does not override them.