Smuggler

MIT

SQLite ↔ Cloudflare D1

Smuggle data between SQLite and Cloudflare D1. Fast. Stateless. Questionable life choices. Content hashing catches actual changes. Delta sync only moves rows that differ. No state files, no stale data, no drama.

curl -fsSL .../install.sh | bash

Source Releases

Sync Flow

Local SQLite → SHA256 Hashing → Diff Engine → D1 API → Cloudflare D1

1. Quick Start

Install (recommended)

# Detects platform, downloads binary, verifies checksum
curl -fsSL https://raw.githubusercontent.com/ezmode-games/smuggler/main/install.sh | bash

Detects your platform, downloads the right binary, verifies the SHA256 checksum, and installs to ~/.local/bin/. Linux x64, macOS x64, macOS ARM64.

From source

# From source (requires Rust toolchain)
cargo install --git https://github.com/ezmode-games/smuggler

Get running

# 1. Copy the example config
cp config.example.toml config.toml

# 2. Add your credentials (don't commit this file, genius)
#    Edit config.toml with your Cloudflare details

# 3. Check if you can reach D1
smuggler status

# 4. See what's different
smuggler diff

# 5. Push your local changes (point of no return)
smuggler push

2. Commands

smuggler status Can we phone home?

smuggler diff What's different?

smuggler push Local → D1 (YOLO)

smuggler pull D1 → Local (safer YOLO)

Options

-c, --config <FILE> Config file. Default: config.toml

-v, --verbose See what's happening under the hood.

--dry-run Coward mode. (Just kidding, it's smart.)

--table <NAME> Sync one table only. Validated against schema.

3. How It Works

The Sync Algorithm

For each table, Smuggler:

Grabs all primary keys from both databases
SHA256 hashes each row's content (excluding timestamp columns)
Compares timestamps when content differs to determine which side is newer
Sorts rows into buckets for action

Bucket Breakdown

local_only You added it locally. Push inserts to D1.

remote_only Someone else added it. Pull inserts locally.

local_newer Your timestamp wins. Push updates D1.

remote_newer Their timestamp wins. Pull updates local.

content_differs Same timestamp, different data. Configurable.

identical Exactly the same. Skip.

Why content hashing?

Timestamps lie. Clocks drift. Bulk imports set everything to "now." Content hashing catches actual changes regardless of what the timestamps say. Never tell me the odds -- show me the SHA256.

4. Configuration

config.toml

cloudflare_account_id = "abc123"
cloudflare_api_token  = "your-token-with-d1-permissions"
database_id           = "your-d1-uuid"
local_db              = "/path/to/local.sqlite"

# --- Sync Settings ---
[sync]
# Empty = sync all tables except excluded
tables = []

# Things you definitely don't want to sync
exclude_tables = [
    "sqlite_sequence",
    "_cf_KV",
    "__drizzle_migrations",
]

# Optional column for timestamp ordering when content differs
timestamp_column = "updated_at"

# When both sides changed: "local_wins", "remote_wins", "newer_wins"
conflict_resolution = "local_wins"

Finding your local D1 database

Wrangler hides it at:

.wrangler/state/v3/d1/miniflare-D1DatabaseObject/<hash>.sqlite

The hash is derived from your binding name. If you have multiple databases, may the Force be with you.

API Token

Get one from Cloudflare Dashboard:

D1:Read -- for diff, pull, status
D1:Write -- for push

Pro tip: Create one token with both permissions. Fewer tokens to lose.

5. Downloads

Pre-built binaries. No Rust toolchain required. Making the Kessel Run in under twelve parsecs.

Linux x64 smuggler-linux-x64.tar.gz

macOS x64 smuggler-macos-x64.tar.gz

macOS ARM64 smuggler-macos-arm64.tar.gz

Or use the install script above. It does the same thing but with more style.

6. Limitations

Things we don't do. We're data movers, not miracle workers.

No schema sync -- Run your migrations separately. We move data, not DDL.

No full-sync transactions -- Each batch is atomic, but the whole sync isn't. Re-run if interrupted.

BLOB = hex string comparison -- Binary data compared as hex strings. It works but it's not pretty.

Tables need primary keys -- We need something to compare. Add a PK.

Stack

RustSQLiteCloudflare D1SHA256

MIT Licensed. View source on GitHub. "Never tell me the odds."