php-opencloud / openstack

PHP SDK for OpenStack clouds
Apache License 2.0
221 stars 148 forks source link

Swift Large Object Segments Sort Incorrectly #318

Closed jbeales closed 9 months ago

jbeales commented 3 years ago

If a Large Object uploaded to Swift has more than 10 segments they do not sort correctly, so when they are downloaded the file is re-assembled in the wrong order. See demo.

According to the Swift docs:

[segments] ...sort in the order in which they should be concatenated. Object names are sorted lexicographically as UTF-8 byte strings.

We are naming the segments with integers, so segments get the names 1, 2, 3.... 9, 10, 11, 12, etc. When Swift re-assembles the large object to be downloaded the segments are ordered 1, 10, 11, 12... 2, 3, 4, etc.

The python-based command-line tools get around this problem by left-padding the segment name with zeros so they sort numerically. They always use a string length of 8 characters.

I'm working on a PR to fix this.

k0ka commented 9 months ago

duplicate of https://github.com/php-opencloud/openstack/issues/365