Delta Lake

Install

See the Install guide for the full setup, including Windows PowerShell.

curl -fsSL https://install.skippr.io/install.sh | shClick to copy

Reads from a Delta Lake table URI (S3, ADLS, or local).

Configuration

source:
  kind: delta_lake
  table_uri: "s3://my-bucket/path/to/table"
  storage_options:
    AWS_REGION: us-east-1
  version: 5

Field	Default	Description
table_uri	(required)	Path to the Delta table (e.g. s3://, abfss://, file path)
storage_options		Key/value options for the object store (credentials, region, etc.)
version		Optional table version to read
filter		Optional predicate filter expression

CLI

skippr connect source delta-lake \
  --table-uri s3://bucket/path/to/table \
  --storage-option AWS_REGION=us-east-1 \
  --version 5

Flag	Description
--table-uri	Table location URI (e.g. s3://bucket/path)
--storage-option KEY=VALUE	Storage backend option; repeat for multiple values
--version	Optional Delta table version to read
--filter	Optional filter expression

Authentication

Authentication depends on the storage backend referenced by table_uri. For security best practices, we strongly advise against storing storage credentials in skippr.yaml. Use environment variable interpolation instead: replace the relevant storage_options value with your own ${ENV_VAR} reference.

The relevant part of skippr.yaml looks like this:

source:
  kind: delta_lake
  table_uri: "s3://bucket/path/to/table"
  storage_options:
    AWS_ACCESS_KEY_ID: ${DELTA_AWS_ACCESS_KEY_ID}
    AWS_SECRET_ACCESS_KEY: ${DELTA_AWS_SECRET_ACCESS_KEY}

Set the env vars before running skippr:

macOS / Linux

export DELTA_AWS_ACCESS_KEY_ID="AKIA..."
export DELTA_AWS_SECRET_ACCESS_KEY="secret"

Windows PowerShell

$env:DELTA_AWS_ACCESS_KEY_ID = "AKIA..."
$env:DELTA_AWS_SECRET_ACCESS_KEY = "secret"

Windows Command Prompt

set DELTA_AWS_ACCESS_KEY_ID=AKIA...
set DELTA_AWS_SECRET_ACCESS_KEY=secret

Permissions or Network Requirements

The runner needs read access to the Delta transaction log and data files behind table_uri. Make sure the URI is reachable from the machine running skippr and that the underlying storage credentials allow reads.

Troubleshooting

Symptom	Fix
table cannot be opened	Verify table_uri, backend credentials, and that the Delta log is present at that path.
schema inference looks wrong	Check the selected table version and any optional filter expression.

Next steps

• Pair this source with a destination from Destination Connectors.
• See How It Works for the end-to-end pipeline flow.

Install

See the Install guide for the full setup, including Windows PowerShell.

curl -fsSL https://install.skippr.io/install.sh | shClick to copy