Use continuation marker properly

[rluetzner]

Description

Previously the continuation marker was not evaluated correctly.

Motivation and Context

This slightly speeds up subsequent list requests.

How to test this PR?

The easiest way to verify this is to run a performance comparison between iternity-rb and these changes.

Types of changes

• [x] Bug fix (non-breaking change which fixes an issue)
• [ ] New feature (non-breaking change which adds functionality)
• [x] Optimization (provides speedup with no functional changes)
• [ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• [ ] Fixes a regression (If yes, please add commit-id or PR # here)
• [ ] Documentation updated
• [x] Unit tests added/updated

[rluetzner]

That's interesting. What makes these tests fail are the changes from 0091093 which only set the Marker on the WalkDirOptions struct. Removing the commit makes the tests pass again. The tests still expect the right files to be returned when a ContinuationMarker is set. This is taken care of by the metaR.filter() function at the very end of fsListObjects(). I haven't quite figured out why the Marker isn't working properly in WalkDir. It looks like the returned results are off by one, so it seems like it should be easy to find, especially as usages of the Marker property are sparse.

[rluetzner]

Ok, I found it. As soon as I figured out that it affected fs-storage backends and not xl-storage backends I pretty much saw right away what the problem was. It's the same thing Klaus Post fixed for xl-storages. Whenever we don't use the marker the final filter operation would get rid of all the superfluous entries, which is why this worked before we added the continuation marker to WalkDir.