Open lunixbochs opened 4 days ago
@lunixbochs Thanks for taking a look at this. I think you are probably right, not much thought was put into using os.PathSeparator
here.
One optimization that I was trying to figure out if it would be possible is somehow storing the directory and readdir position in the continuation marker. Ideally we would like to continue the walk without having to visit all the previous entries. The current walk really restricts the bucket scalability to be able to support large bucket list-objects.
Describe the bug backend.Walk constructs paths using
os.PathSeparator
This has two problems:
os.PathSeparator
is incorrect for the io/fs abstraction./
or delimiter, notos.PathSeparator
I'm looking at speeding up
backend.Walk
. I'll see about improving this while I'm in there.