mapbox / dynamodb-replicator

module for dynamodb multi-region replication
ISC License
129 stars 48 forks source link

Don't allow empty S3-key parts #98

Open Dunedan opened 7 years ago

Dunedan commented 7 years ago

dynamodb-replicator uses / as delimiter to hierarchically structure the data in S3. Up to now it did a naive join of all path parts when constructing paths and key names for S3. That produced different paths in S3 whether or not a trailing slash was provided when specifying the target. For example the following two commands produced different output files in S3:

incremental-backfill us-east-1/table s3://s3-bucket/foo
incremental-backfill us-east-1/table s3://s3-bucket/foo/

As that's different from other tools which interact with S3, like the official awscli this commit changes that behavior to work like path.join does, but by using an external package to ensure that also under Windows paths are generated with forward slashes.