On-the-Fly Object Storage Migration at Scale
Godwit Sync is a predictable, plan-first migration tool for moving data between S3-compatible object stores. Built for control, observability, and production safety.
Rate Limited
Control read/write throughput
Resumable
Continue from exact checkpoint
No Local Staging
True stream-to-stream copy
Source Safe
Never overload source systems
Plan Before You Move
Know exactly what will happen before any data moves. Generate migration plans, inspect them, modify as needed, and execute with confidence.
- Deterministic migration plan generation (--plan-only)
- Migration plans stored in SQLite
- Inspect, modify, and reuse plans
- Predictable execution every time
$ godwit sync \
--source s3://production-bucket \
--dest s3://backup-bucket \
--plan-only --brief
────────────────────────────────────────
PLAN CREATED
────────────────────────────────────────
✓ Run-ID: run-1770316589731184186
✓ Objects analyzed: 685,234
✓ Objects to transfer: 620,205
✓ Objects skipped: 65,029
✓ Data to transfer: 40.03 GB
✓ Data skipped: 4.7 GB
────────────────────────────────────────
End-to-End Checksums
Validate every object with real MD5 hashes, not just ETAGs.
Deterministic Runs
Identical inputs produce identical results—no hidden randomness.
Parallel, Controlled Transfers
High throughput without sacrificing stability or safety.
Metadata Fidelity
Preserve object metadata exactly as stored at the source.
Exact Object Ordering
Stable, predictable processing order across runs.
Chunk-Level Recovery
Recover from partial uploads without corrupting large files.
Production-Safe by Design
Built for environments where downtime and data loss are not options. Rate-limit operations, resume from any point, and keep your source systems healthy.
- Rate-limited reads and writes
- True stream-to-stream copy (no local staging)
- Resume from exact file or chunk
- Large file stability
- No overload of source systems
Observable & Verifiable
Know exactly what's happening at every moment. Integrate with your existing monitoring stack and verify every byte after it lands.
- Verbose CLI UI with real-time progress
- REST status endpoint
- Prometheus-compatible metrics
- Predictable duration and progress estimates
- Built-in MD5 verify after every transfer
# Reusable run IDs resume interrupted syncs
$ godwit sync \
--resume --brief --run-id run-daily-backup \
--source s3://prod \
--dest s3://backup
────────────────────────────────────────
RESUME READY
────────────────────────────────────────
✓ Run-ID: run-daily-backup
✓ Total objects: 20
✓ Uploaded objects: 15
✓ Objects left: 5
✓ Data to transfer: 2.00 GB
────────────────────────────────────────
# Automation-friendly exit codes
$ echo $?
0Built for Repetition and Scale
Design once, run many times. Perfect for scheduled syncs, multi-bucket operations, and CI/CD integration.
- Reusable run IDs
- Incremental syncs
- Multiple buckets and paths
- Automation-friendly CLI workflows
Data Movement, Done Right
Godwit Sync turns risky storage migrations into planned, observable, production-safe workflows for modern infrastructure teams.