On-the-Fly Object Storage Migration at Scale
Godwit Sync is a predictable, plan-first migration tool for moving data between S3-compatible object stores. Built for control, observability, and production safety.
Version History
Migrate all object versions
Rate Limited
Control read/write throughput
Resumable
Continue from exact checkpoint
Source Safe
Never overload source systems
Plan Before You Move
Know exactly what will happen before any data moves. Generate migration plans, inspect them, modify as needed, and execute with confidence.
- Deterministic migration plan generation (--plan-only)
- Migration plans stored in SQLite
- Inspect, modify, and reuse plans
- Predictable execution every time
$ godwit sync \
--source s3://production-bucket \
--dest s3://backup-bucket \
--plan-only --brief
────────────────────────────────────────
PLAN CREATED
────────────────────────────────────────
✓ Run-ID: run-1770316589731184186
✓ Objects analyzed: 685,234
✓ Objects to transfer: 620,205
✓ Objects skipped: 65,029
✓ Data to transfer: 40.03 GB
✓ Data skipped: 4.7 GB
────────────────────────────────────────
End-to-End Checksums
Validate every object with real MD5 hashes, not just ETAGs.
Deterministic Runs
Identical inputs produce identical results—no hidden randomness.
Parallel, Controlled Transfers
High throughput without sacrificing stability or safety.
Metadata Fidelity
Preserve object metadata exactly as stored at the source.
Chunk-Level Recovery
Recover from partial uploads without corrupting large files.
Object Lock Preservation
Replicate GOVERNANCE/COMPLIANCE retention and legal hold settings per version.
Production-Safe by Design
Built for environments where downtime and data loss are not options. Rate-limit operations, resume from any point, and keep your source systems healthy.
- Rate-limited reads and writes
- True stream-to-stream copy (no local staging)
- Resume from exact file or chunk
- Large file stability
- No overload of source systems
Observable & Verifiable
Know exactly what's happening at every moment. Integrate with your existing monitoring stack and verify every byte after it lands.
- Verbose CLI UI with real-time progress
- REST status endpoint
- Prometheus-compatible metrics
- Predictable duration and progress estimates
- Built-in MD5 verify after every transfer
Migrate Every Version, Not Just the Latest
Preserve the complete history of your S3 objects. Godwit Sync copies all versions - or just the ones you need - with full checksum verification, Glacier-aware skipping, and Object Lock preservation.
- Full or partial version history migration
- Per-version checksum verification
- Glacier-class objects detected and skipped
- Object Lock retention and legal hold preserved
- Detailed per-version transfer reporting
# Plan a full version history migration
$ godwit sync \
--source s3://prod-bucket \
--dest s3://archive-bucket \
--version-mode all --plan-only
Planning...
⚠ Warning: 9 GLACIER objects detected.
Restore required before migration.
────────────────────────────────────────
PLAN READY
────────────────────────────────────────
✓ Run-ID: version-history-prod
✓ Objects analyzed: 23
✓ Objects to transfer: 14
✓ Objects skipped: 9
✓ Data to transfer: 48.20 GB
────────────────────────────────────────# Reusable run IDs resume interrupted syncs
$ godwit sync \
--resume --brief --run-id run-daily-backup \
--source s3://prod \
--dest s3://backup
────────────────────────────────────────
RESUME READY
────────────────────────────────────────
✓ Run-ID: run-daily-backup
✓ Total objects: 20
✓ Uploaded objects: 15
✓ Objects left: 5
✓ Data to transfer: 2.00 GB
────────────────────────────────────────
# Automation-friendly exit codes
$ echo $?
0Built for Repetition and Scale
Design once, run many times. Perfect for scheduled syncs, multi-bucket operations, and CI/CD integration.
- Reusable run IDs
- Incremental syncs
- Multiple buckets and paths
- Automation-friendly CLI workflows
Data Movement, Done Right
Godwit Sync turns risky storage migrations into planned, observable, production-safe workflows for modern infrastructure teams.