A Source is a project-scoped connection identity. Each driver resolves which metadata, preview, query, transfer, write, delete or quality operations are safe in the current environment.
Data operations
Connect, inspect, transform and execute with explicit scope and run context.
Move from heterogeneous Sources and controlled files to managed tables, quality checks and persisted execution history without flattening all connectors into the same query or write model.
Server-owned source and project context, read-only policy and bounded results support discovery before a query becomes SavedQuery, Dataset or visualization input. Numbered tabs are durable workspaces with rename, autosave and optimistic revision handling, so leaving the browser does not silently discard the current draft.
Upload and secure URL import first create an immutable Drive version with SHA-256. Preflight resolves content, delimiter, quote, header and schema before any transform is queued.
Validated CSV, delimited UTF-8 text, JSON object records and content from the first XLSX worksheet load into a new physical table version. The stable alias changes only after result validation succeeds.
The public catalog exposes ExecuteQ, QueryTransfer, DirectStreamTransfer, SchemaTransfer, TransformSchema, DeleteSchema, NoSqlDelete, DataQualityRule and Master. Each dispatch creates a persisted PipelineRun; hidden legacy definitions do not become supported merely because an old record still exists.
Queued, running, failed, partial, canceled and recovered work must remain distinguishable. A monitoring interruption is not evidence that a state-changing operation never reached its target. Pipeline currently has no universal cancel, retry, rollback or exactly-once contract, and least-privilege database roles remain mandatory.
Sources expose capabilities, not assumptions.
A Source is a project-scoped connection identity. Each driver resolves which metadata, preview, query, transfer, write, delete or quality operations are safe in the current environment.
SQL Console is exploration, not database administration.
Server-owned source and project context, read-only policy and bounded results support discovery before a query becomes SavedQuery, Dataset or visualization input. Numbered tabs are durable workspaces with rename, autosave and optimistic revision handling, so leaving the browser does not silently discard the current draft.
A successful or failed service execution is retained as QueryExecution when a valid Source and SQL request enters execution.
SavedQuery keeps editable SQL; Dataset promotion creates a separate governed data asset and lineage reference.
Source dialect comment syntax and safe read-only handling are normalized through shared execution boundaries rather than one PostgreSQL-only wrapper.
Files keep their original identity.
Upload and secure URL import first create an immutable Drive version with SHA-256. Preflight resolves content, delimiter, quote, header and schema before any transform is queued.
CSV, TXT, TSV, PSV, DSV, DAT, DATA, LOG, JSON and XLSX are recognized through format-aware inspection.
A text file with another extension can follow the delimited-text path when bounded content classification verifies it.
Delimiter, control byte, quote/escape, header decision, preview rows and warnings remain visible for user confirmation.
Filename and stable-alias collisions are resolved explicitly and recorded rather than silently overwriting an existing object.
Publication is atomic at the stable alias.
Validated CSV, delimited UTF-8 text, JSON object records and content from the first XLSX worksheet load into a new physical table version. The stable alias changes only after result validation succeeds.
-
Validate the immutable input
File identity, size, checksum, parser decision and row structure are checked before load.
-
Create a shadow table version
The new physical version is isolated from the currently visible stable alias.
-
Load, profile and validate
Row/column counts, schema and bounded samples are recorded; a partial load never becomes the current table.
-
Promote or clean up
Only a validated result moves the alias; failure keeps the previous version available and records the attempted run.
The System Sandbox Source exposes managed aliases to SQL Console under the current Project schema.
Rename and restore create auditable lifecycle changes; restore can deliberately re-expose an earlier schema contract.
Pipeline exposes implemented execution honestly.
The public catalog exposes ExecuteQ, QueryTransfer, DirectStreamTransfer, SchemaTransfer, TransformSchema, DeleteSchema, NoSqlDelete, DataQualityRule and Master. Each dispatch creates a persisted PipelineRun; hidden legacy definitions do not become supported merely because an old record still exists.
Master supports ordered jobs, sleeps and quality gates plus serial or parallel batches.
Continuation creates a new run from prior child status; it does not rewrite the original run.
Job definitions are mutable and currently have no immutable JobVersion.
Interval/cron Schedule intent is stored, but there is no active schedule dispatcher or reliable next-run calculation.
Failure state is part of the result.
Queued, running, failed, partial, canceled and recovered work must remain distinguishable. A monitoring interruption is not evidence that a state-changing operation never reached its target. Pipeline currently has no universal cancel, retry, rollback or exactly-once contract, and least-privilege database roles remain mandatory.
Next step
Start with the flow that cannot afford ambiguity.
Bring the systems, owners, rules, delivery obligations and evidence requirements that matter.
Discuss a critical flow