search

ncw / swift

Go language interface to Swift / Openstack Object Storage / Rackspace cloud files (golang)
MIT License
310 stars 107 forks source link

Add Object Options to keep Marker when fetching all objects #144

Closed jeremy-clerc closed 5 years ago

jeremy-clerc commented 5 years ago

Hi,

Linked to #113, I use ObjectNamesAll, but I need the Marker to be kept, the idea is basically I need to delete files between certain markers, so I fetch all the name and then delete them and I may have more than 10_000.

I started creating a method with a new name, but I had to copy a lot, just to not reset Marker, in order to not break the API, as ObjectWalk is public.

This solution does not break the API, while allowing to keep the Marker when doing *All methods.

I would understand if you think it is not clean, and it is preferable to copy all the things.

In which case I would use ObjectNamesRangeAll, ObjectsRangeWalk as names and override objectsAllOpts to reset marker depending of an extra parameter (as it is only used by ObjectsWalk).

Thanks

jeremy-clerc commented 5 years ago

Note that related to https://github.com/ncw/swift/issues/113#issuecomment-389830370

The All versions are supposed to page through and return everything hence the overwriting of Marker and Limit.

EndMarker should have been overridden too. As today it can be set, and works as expected.

ncw commented 5 years ago

I think that looks like a neat solution - well done :-)

I'll merge it now.