API reference · Data

Data

Connections and datasets are one group under /v2/data. A connection is where data comes from; a dataset is an ingested or registered table, auto-classified and profiled. Each ingest or sync mints an immutable snapshot (dsv_...), which is what endpoints pin and held-out reports stamp.

Four families of source: upload (CSV, Excel, JSON), databases (PostgreSQL, MySQL, Supabase, MongoDB, Databricks, Snowflake, Pinecone, Chroma), cloud storage (Google Drive, GCP Storage, AWS S3), and APIs (REST, GraphQL). Small files upload directly through the platform; everything else is a registered connection referenced at run or creation time.

Routes

objectThe data objects
POST/v2/data/connect
GET/v2/data
GET/v2/data/:id
POST/v2/data/:id/sync
POST/v2/data/:id/pin
POST/v2/data/:id/unpin
POST/v2/data/generate
DELETE/v2/data/:id

The data objects

GET /v2/data returns both kinds. A dataset carries its current snapshot and pin state; a connection carries its scheme ref. Neither carries analytical outputs (sector, profile): those come from a run and live on results and reports.

The data objectsjson

{
  "id": "ds_crm01",
  "kind": "dataset",
  "name": "crm_contacts",
  "source": "snowflake://example/sales/public/live_accounts",
  "snapshot": "dsv_91f2",
  "snapshot_at": "2026-08-12T09:31:00Z",
  "rows": 22400,
  "cols": 12,
  "cells": 268800,
  "size_bytes": 3145728,
  "pinned": true,
  "pinned_by": ["{endpoint_id}"]
}

Attributes

idstring
ds_... for a dataset, conn_... for a connection. Immutable.
kindstring
Object kind.
datasetconnection
namestring
Display name.
sourcestring
Scheme ref for connections (snowflake://sales/live, s3://bucket/exports/, postgres://..., rest+https://...); upload for direct uploads.
snapshotstring | null
Datasets: the current dsv_... snapshot id.
snapshot_atstring | null
Timestamp of the current snapshot.
rowsinteger
Datasets: row count of the current snapshot.
colsinteger
Datasets: column count.
cellsinteger
Datasets: rows x cols, the exact metering size of a fresh pass over it.
size_bytesinteger
Stored size, for the storage line.
pinnedboolean
Datasets: pinned datasets bill the cached rate on repeat calls indefinitely and count toward plan storage.
pinned_byarray of strings
Endpoint ids currently serving this dataset. A dataset pinned by a live endpoint stays pinned until that endpoint is deleted or refreshed onto other data.

Connect a source

POST/v2/data/connect

scope manage

Registers a connection to a database, cloud bucket, or API, and ingests an initial snapshot.

Credentials are stored encrypted at rest and are write-only; rotation is by re-entering them. Every connection creation and every sync is recorded in the audit log.

POST/v2/data/connect

curl -X POST 'https://api.schemalabs.ai/v2/data/connect' \
  -H "Authorization: Bearer $SCHEMA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "source": "snowflake://acme/sales/public/live_accounts",
  "name": "sales_live",
  "credentials": {
    "user": "schema_reader",
    "password": "••••••••",
    "role": "READER",
    "warehouse": "WH_XS"
  }
}'

import os
import requests

SCHEMA_API_KEY = os.environ["SCHEMA_API_KEY"]

r = requests.post(
    "https://api.schemalabs.ai/v2/data/connect",
    headers={"Authorization": f"Bearer {SCHEMA_API_KEY}"},
    json={
        "source": "snowflake://acme/sales/public/live_accounts",
        "name": "sales_live",
        "credentials": {
            "user": "schema_reader",
            "password": "••••••••",
            "role": "READER",
            "warehouse": "WH_XS",
        },
    },
)
r.raise_for_status()
result = r.json()

const res = await fetch('https://api.schemalabs.ai/v2/data/connect', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.SCHEMA_API_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    source: "snowflake://acme/sales/public/live_accounts",
    name: "sales_live",
    credentials: {
      user: "schema_reader",
      password: "••••••••",
      role: "READER",
      warehouse: "WH_XS",
    },
  }),
});
if (!res.ok) throw new Error(`Schema API error ${res.status}`);
const result = await res.json();

{
  "connection": {
    "id": "conn_3f9a",
    "kind": "connection",
    "name": "sales_live",
    "source": "snowflake://acme/sales/public/live_accounts"
  },
  "dataset": {
    "id": "ds_sales01",
    "kind": "dataset",
    "name": "live_accounts",
    "source": "snowflake://acme/sales/public/live_accounts",
    "snapshot": "dsv_0c44",
    "rows": 51200,
    "cols": 18,
    "cells": 921600,
    "pinned": false,
    "pinned_by": []
  }
}

Body application/json

sourcestringrequired
Scheme ref: the provider scheme followed by the provider’s own path to the table or location, for example snowflake://..., postgres://..., s3://bucket/prefix/, gdrive://folder-id, rest+https://host/path.
namestring
Display name. Defaults to the last path segment.
credentialsobject
Provider-specific credentials (for example user, password, role, warehouse for Snowflake; access_key_id, secret_access_key for S3). Never returned.
optionsobject
Provider options such as query (a SQL statement to materialize as the dataset) or sheet.

Returns

The connection object and the dataset it produced, with the first snapshot.

List data

GET/v2/data

scope read

Lists connections and datasets with their snapshot and pin state.

GET/v2/data

curl 'https://api.schemalabs.ai/v2/data?limit=20' \
  -H "Authorization: Bearer $SCHEMA_API_KEY"

import os
import requests

SCHEMA_API_KEY = os.environ["SCHEMA_API_KEY"]

r = requests.get(
    "https://api.schemalabs.ai/v2/data?limit=20",
    headers={"Authorization": f"Bearer {SCHEMA_API_KEY}"},
)
r.raise_for_status()
result = r.json()

const res = await fetch('https://api.schemalabs.ai/v2/data?limit=20', {
  headers: {
    Authorization: `Bearer ${process.env.SCHEMA_API_KEY}`,
  },
});
if (!res.ok) throw new Error(`Schema API error ${res.status}`);
const result = await res.json();

{
  "datasets": [
    {
      "id": "ds_crm01",
      "kind": "dataset",
      "name": "crm_contacts",
      "source": "snowflake://example/sales/public/live_accounts",
      "snapshot": "dsv_91f2",
      "rows": 22400,
      "cols": 12,
      "cells": 268800,
      "pinned": true,
      "pinned_by": ["{endpoint_id}"]
    },
    {
      "id": "ds_bill01",
      "kind": "dataset",
      "name": "billing_db",
      "source": "postgres://billing/live",
      "snapshot": "dsv_c4e7",
      "rows": 22400,
      "cols": 6,
      "cells": 134400,
      "pinned": true,
      "pinned_by": ["{endpoint_id}"]
    },
    {
      "id": "ds_claims",
      "kind": "dataset",
      "name": "claims_2025",
      "source": "upload",
      "snapshot": "dsv_77e0",
      "rows": 500000,
      "cols": 30,
      "cells": 15000000,
      "pinned": true,
      "pinned_by": ["{other_endpoint_id}"]
    },
    {
      "id": "conn_3f9a",
      "kind": "connection",
      "name": "sales_live",
      "source": "snowflake://acme/sales/public/live_accounts",
      "dataset": "ds_sales01",
      "snapshot": "dsv_1d09"
    }
  ],
  "next_cursor": null,
  "total": 4
}

Query parameters

limitintegerdefault 20
Page size.
cursorstring
Cursor from a previous page.

Returns

A page of data objects and a next_cursor.

Retrieve data

GET/v2/data/:id

scope read

Returns a dataset’s profile (rows, columns, current snapshot, pin state) or a connection’s status.

GET/v2/data/:id

curl 'https://api.schemalabs.ai/v2/data/ds_crm01' \
  -H "Authorization: Bearer $SCHEMA_API_KEY"

import os
import requests

SCHEMA_API_KEY = os.environ["SCHEMA_API_KEY"]

r = requests.get(
    "https://api.schemalabs.ai/v2/data/ds_crm01",
    headers={"Authorization": f"Bearer {SCHEMA_API_KEY}"},
)
r.raise_for_status()
result = r.json()

const res = await fetch('https://api.schemalabs.ai/v2/data/ds_crm01', {
  headers: {
    Authorization: `Bearer ${process.env.SCHEMA_API_KEY}`,
  },
});
if (!res.ok) throw new Error(`Schema API error ${res.status}`);
const result = await res.json();

{
  "id": "ds_crm01",
  "kind": "dataset",
  "name": "crm_contacts",
  "source": "snowflake://example/sales/public/live_accounts",
  "snapshot": "dsv_91f2",
  "snapshot_at": "2026-08-12T09:31:00Z",
  "rows": 22400,
  "cols": 12,
  "cells": 268800,
  "size_bytes": 3145728,
  "pinned": true,
  "pinned_by": ["{endpoint_id}"]
}

Path parameters

idstringrequired
ds_... or conn_....

Returns

The data object.

Sync a connection

POST/v2/data/:id/sync

scope manageasync · 202

Re-pulls a connection and mints a new immutable snapshot (dsv_...) on its dataset. Endpoints pinned to the dataset keep serving the previous snapshot until you refresh them.

POST/v2/data/:id/sync

curl -X POST 'https://api.schemalabs.ai/v2/data/conn_3f9a/sync' \
  -H "Authorization: Bearer $SCHEMA_API_KEY"

import os
import requests

SCHEMA_API_KEY = os.environ["SCHEMA_API_KEY"]

r = requests.post(
    "https://api.schemalabs.ai/v2/data/conn_3f9a/sync",
    headers={"Authorization": f"Bearer {SCHEMA_API_KEY}"},
)
r.raise_for_status()
result = r.json()

const res = await fetch('https://api.schemalabs.ai/v2/data/conn_3f9a/sync', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.SCHEMA_API_KEY}`,
  },
});
if (!res.ok) throw new Error(`Schema API error ${res.status}`);
const result = await res.json();

{
  "id": "conn_3f9a",
  "dataset": "ds_sales01",
  "snapshot": "dsv_1d09",
  "previous_snapshot": "dsv_0c44",
  "diff": {
    "rows_added": 310,
    "rows_changed": 12,
    "rows_removed": 4,
    "rows_unchanged": 51184
  },
  "synced_at": "2026-08-16T09:00:03Z"
}

Path parameters

idstringrequired
conn_... or the dataset id it feeds.

Returns

202 with a job for large sources, or 200 with the new snapshot for small ones.

A sync itself is not metered. What a subsequent pass pays follows the diff: added and changed rows bill fresh, unchanged rows bill the cached rate, removed rows bill nothing.

Pin a dataset

POST/v2/data/:id/pin

scope manage

Pins a dataset: repeat calls over it bill the cached rate indefinitely, and it counts toward plan storage. Endpoint creation pins its datasets automatically.

POST/v2/data/:id/pin

curl -X POST 'https://api.schemalabs.ai/v2/data/ds_sales01/pin' \
  -H "Authorization: Bearer $SCHEMA_API_KEY"

import os
import requests

SCHEMA_API_KEY = os.environ["SCHEMA_API_KEY"]

r = requests.post(
    "https://api.schemalabs.ai/v2/data/ds_sales01/pin",
    headers={"Authorization": f"Bearer {SCHEMA_API_KEY}"},
)
r.raise_for_status()
result = r.json()

const res = await fetch('https://api.schemalabs.ai/v2/data/ds_sales01/pin', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.SCHEMA_API_KEY}`,
  },
});
if (!res.ok) throw new Error(`Schema API error ${res.status}`);
const result = await res.json();

{ "id": "ds_sales01", "kind": "dataset", "pinned": true, "pinned_by": [] }

Path parameters

idstringrequired
ds_....

Returns

The dataset object with pinned: true.

Unpin a dataset

POST/v2/data/:id/unpin

scope manage

Unpins a dataset. The cached rate expires shortly after last use; the next pass bills fresh. A dataset pinned by a live endpoint stays pinned (409 names the endpoint).

POST/v2/data/:id/unpin

curl -X POST 'https://api.schemalabs.ai/v2/data/ds_crm01/unpin' \
  -H "Authorization: Bearer $SCHEMA_API_KEY"

import os
import requests

SCHEMA_API_KEY = os.environ["SCHEMA_API_KEY"]

r = requests.post(
    "https://api.schemalabs.ai/v2/data/ds_crm01/unpin",
    headers={"Authorization": f"Bearer {SCHEMA_API_KEY}"},
)
r.raise_for_status()
result = r.json()

const res = await fetch('https://api.schemalabs.ai/v2/data/ds_crm01/unpin', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.SCHEMA_API_KEY}`,
  },
});
if (!res.ok) throw new Error(`Schema API error ${res.status}`);
const result = await res.json();

{ "id": "ds_crm01", "kind": "dataset", "pinned": false }

{
  "error": {
    "type": "conflict",
    "message": "dataset ds_crm01 is pinned by live endpoint churn ({endpoint_id}); delete the endpoint or refresh it onto other data first",
    "request_id": "req_2c8f1b"
  }
}

Path parameters

idstringrequired
ds_....

Returns

The dataset object with pinned: false, or 409 conflict naming the dependent endpoint.

Generate synthetic data

POST/v2/data/generate

scope manageasync · 202idempotency-keybatchable

Produces a synthetic tabular dataset for a sector with realistic per-column ranges. The result is an ordinary dataset (ds_...), useful for cold-start when representative data is thin.

POST/v2/data/generate

curl -X POST 'https://api.schemalabs.ai/v2/data/generate' \
  -H "Authorization: Bearer $SCHEMA_API_KEY" \
  -H "Idempotency-Key: <unique key>" \
  -H "Content-Type: application/json" \
  -d '{ "sector": "hospital operations", "rows": 5000, "name": "hospital_ops_synth" }'

import os
import requests

SCHEMA_API_KEY = os.environ["SCHEMA_API_KEY"]

r = requests.post(
    "https://api.schemalabs.ai/v2/data/generate",
    headers={"Authorization": f"Bearer {SCHEMA_API_KEY}", "Idempotency-Key": "<unique key>"},
    json={
        "sector": "hospital operations",
        "rows": 5000,
        "name": "hospital_ops_synth",
    },
)
r.raise_for_status()
result = r.json()

const res = await fetch('https://api.schemalabs.ai/v2/data/generate', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.SCHEMA_API_KEY}`,
    'Content-Type': 'application/json',
    'Idempotency-Key': '<unique key>',
  },
  body: JSON.stringify({
    sector: "hospital operations",
    rows: 5000,
    name: "hospital_ops_synth",
  }),
});
if (!res.ok) throw new Error(`Schema API error ${res.status}`);
const result = await res.json();

{
  "dataset": "ds_synth03",
  "name": "hospital_ops_synth.csv",
  "sector": "hospital operations",
  "rows": 5000,
  "columns": ["patient_id", "age", "sex", "diagnosis_code", "lvef_pct", "admission_date"],
  "processing": { "mode": "realtime" }
}

Headers

Idempotency-Keystring
Optional. A unique key for this request. Retrying a POST with the same key and body never creates a duplicate job; the same key with a different body returns 409 conflict.

Body application/json

sectorstringrequired
Sector name, in the same vocabulary the sector output uses, for example "hospital operations" or "consumer financial services".
rowsintegerrequired
Rows to generate.
namestring
Dataset display name.
optionsobject
processing: "batch" runs the generation as a batch job at the batch rate.

Returns

The new dataset: id, name, columns, rows, and the processing block. Large generations run as batch jobs and return 202 with a job.

Metering: C = cells generated at the synthetic rate; the output grid is the meter.

Delete data

DELETE/v2/data/:id

scope delete

Removes a dataset or connection immediately and irreversibly.

A dataset pinned by a live endpoint cannot be deleted: the 409 conflict names the dependency, so delete the endpoint or refresh it onto other data first.

DELETE/v2/data/:id

curl -X DELETE 'https://api.schemalabs.ai/v2/data/ds_old01' \
  -H "Authorization: Bearer $SCHEMA_API_KEY"

import os
import requests

SCHEMA_API_KEY = os.environ["SCHEMA_API_KEY"]

r = requests.delete(
    "https://api.schemalabs.ai/v2/data/ds_old01",
    headers={"Authorization": f"Bearer {SCHEMA_API_KEY}"},
)
r.raise_for_status()
print(r.json())

const res = await fetch('https://api.schemalabs.ai/v2/data/ds_old01', {
  method: 'DELETE',
  headers: {
    Authorization: `Bearer ${process.env.SCHEMA_API_KEY}`,
  },
});
if (!res.ok) throw new Error(`Schema API error ${res.status}`);
const result = await res.json();

{ "id": "ds_old01", "status": "deleted", "deleted_at": "2026-08-16T10:11:02Z" }

Path parameters

idstringrequired
ds_... or conn_....

Returns

The deleted id and status, or 409 conflict naming dependent endpoints.