search

martymac / fpart

Sort files and pack them into partitions
https://www.fpart.org/
BSD 2-Clause "Simplified" License
230 stars 39 forks source link

fpart/fsync is slowly when reading files names from S3 or similiar object file system #45

Closed jidongzhang closed 1 year ago

jidongzhang commented 1 year ago

Is it possible to use rclone output file system list to fpart, and procssed by fpart

martymac commented 1 year ago

Hello,

Fpart is designed to fast-crawl local filesystems only (through the VFS layer, it does not support any kind of remote filesystem if it is not mounted locally).

Also, if you want to use another program's output, it supports a '-a' option that will read arbitrary values (following the “size(blank)path” scheme) to produce partitions, but that option cannot be used in live mode, which is the mode used by fpsync. You will not be able to trigger fpsync using rclone's output.

Best regards,

Ganael.