aaronleopold
commented
5 months ago Stump: Reads full file on SCAN and partial file on OPEN
The main reason Stump reads the full file on scan is to determine the actual page count. That operation involves iterating through each file e.g. in an archive to determine whether it is a valid page (re: image file). The validity check uses actual byte content, and falls back onto the extension, as a way of attempting to be more accurate in knowing what is truly a valid page. The only feasible way to allow for partial reads would be:
- Drop the accuracy and operate only on file extensions, trusting the extension is always correct
- Don't count the pages and rely on metadata. This has problems in that missing or inaccurate metadata would cause problems.
This could be a configuration. I'll have to think on implementation details, though.
Strontium
Is your feature request related to a problem? Please describe. When working with a remote file system to the stump host, like keeping files on a cloud storage provider, there may be both bandwidth and egress limits when accessing data. Currently when doing an initial scan for a book, stump appears to request to read the entire file. When using cloud storage via an rclone mount, this scan causes the file to be downloaded in its entirety, which is inefficient and likely slow for large files. Opening the book for reading, stump appears to only request the pages it requires, hence rclone will only download those parts of the file, reducing traffic and loading times.
Describe the solution you'd like Allow an option for reducing file access during the initial scan. Scan should be limited to:
Describe alternatives you've considered Stump being remote storage aware and will limit itself appropriately automatically.
Additional context In comparison to other applications in my testing with an rclone mount: