bt datasets manages remote Braintrust datasets directly from the CLI.
Subcommands
| Subcommand | Description |
|---|
bt datasets list | List all datasets in the current project |
bt datasets create <name> | Create a dataset, optionally seeding it with records |
bt datasets update <name> | Upsert records into a dataset (aliases: add, refresh) |
bt datasets view <name> | Display dataset metadata and preview records |
bt datasets delete <name> | Delete a dataset and all its records |
bt datasets pipeline <stage> | Transform project logs into dataset rows (run, or staged pull, transform, push) |
bt datasets list
Lists all datasets in the current project.
bt datasets create
Create a dataset, optionally seeding it with records from a file, stdin, or inline JSON.
# Create an empty dataset
bt datasets create my-dataset
# Create with a description
bt datasets create my-dataset --description "Dataset for smoke tests"
# Seed from a JSONL file
bt datasets create my-dataset --file records.jsonl
# Seed from stdin
cat records.jsonl | bt datasets create my-dataset
# Seed with inline JSON rows
bt datasets create my-dataset --rows '[{"id":"case-1","input":{"text":"hi"},"expected":"hello"}]'
# Rows without id fields get auto-generated stable IDs
bt datasets create my-dataset --rows '[{"input":{"text":"hi"},"expected":"hello"}]'
Flags
| Flag | Description |
|---|
--description <TEXT> | Dataset description |
--file <PATH> | Seed from a JSONL file |
--rows <JSON> | Seed with an inline JSON array of rows |
When rows omit an id field, bt datasets auto-generates stable record IDs.
bt datasets update
Upsert records into an existing dataset. Also available as bt datasets add and bt datasets refresh.
bt datasets update my-dataset --file records.jsonl
bt datasets add my-dataset --rows '[{"id":"case-2","input":{"text":"bye"},"expected":"goodbye"}]'
bt datasets refresh my-dataset --file records.jsonl --id-field metadata.case_id
Flags
| Flag | Description |
|---|
--file <PATH> | Input JSONL file |
--rows <JSON> | Inline JSON array of rows |
--id-field <PATH> | Dot-separated path to use as the record ID instead of the id field |
Each row must have a stable ID via the id field or --id-field. Rows without IDs are rejected.update, add, and refresh upsert rows directly — rows not in the input are not deleted. refresh fails if the dataset does not exist.--id-field uses dot-separated paths (e.g., metadata.case_id). Escape literal dots as \. and literal backslashes as \\.Input may also be a JSON object with a top-level rows array (matching bt datasets view --json output). Each row in rows is validated against the accepted fields: id, input, expected, metadata, tags, and origin.
bt datasets view
Display dataset metadata and preview records in the terminal.
bt datasets view my-dataset # up to 200 rows by default
bt datasets view my-dataset --limit 50
bt datasets view my-dataset --all-rows
bt datasets view my-dataset --full # exact values, no truncation
bt datasets view my-dataset --json # machine-readable JSON output
Flags
| Flag | Description |
|---|
--limit <N> | Maximum rows to show (default: 200) |
--all-rows | Show all rows regardless of dataset size |
--full | Show exact values without truncation |
--json | Output as JSON (compatible as input to bt datasets update) |
bt datasets delete
Permanently delete a dataset and all its records.
bt datasets delete my-dataset
This operation is irreversible. All records in the dataset are permanently deleted.
bt datasets pipeline
Transform project logs into dataset rows using a pipeline declared with DatasetPipeline(...) in a TypeScript or Python file. See Dataset pipelines for how to write a pipeline definition.
Beta — This feature is subject to change.
Dataset pipelines require bt CLI v0.10.0 or later, plus the braintrust SDK for the language you write the pipeline in: TypeScript SDK v3.16.0 or later, or Python SDK v0.23.0 or later.
pull, transform, and push steps:
# One-shot: discover source refs, transform, and insert new rows
bt datasets pipeline run ./pipeline.ts --limit 100
# Staged: inspect or edit artifacts between steps
bt datasets pipeline pull ./pipeline.ts --limit 500
bt datasets pipeline transform ./pipeline.ts
bt datasets pipeline push ./pipeline.ts
# Python pipelines are supported too
bt datasets pipeline run ./pipeline.py --project "My Project" --limit 100
pulled.jsonl and transformed.jsonl to the bt-sync/ directory by default. Inspect or edit transformed.jsonl before running push.
Flags
| Flag | Description |
|---|
--limit <N> | Number of source spans or traces to discover |
--window <DURATION> | Constrain source discovery by created time (default: 1d) |
--root-span-id <ID> | Restrict pulling to one or more specific root spans |
--root <PATH> | Directory for staged artifacts (default: bt-sync) |
--out <PATH> | Override the managed output path for pull and transform |
--in <PATH> | Override the input artifact for transform or push |
--fresh | Restart an already completed push spec (push) |
--project <NAME> | Source project when the pipeline source omits a project |
--source-project <NAME> | Override the source project (pull, transform, run) |
--source-project-id <ID> | Override the source project ID (pull, transform, run) |
--source-org <NAME> | Override the source organization (pull, transform, run) |
--source-filter <FILTER> | Override the source SQL filter (pull, transform, run) |
--target-project <NAME> | Override the target project (run, push) |
--target-project-id <ID> | Override the target project ID (run, push) |
--target-org <NAME> | Override the target organization (run, push) |
--target-dataset <NAME> | Override the target dataset (run, push) |
--max-concurrency <N> | Maximum concurrent transforms |
--name <NAME> | Select a pipeline when the file defines more than one |
