newhinton / Round-Sync

An android cloud file manager, powered by rclone. Visit https://roundsync.com for more information!
https://roundsync.com
GNU General Public License v3.0
1.21k stars 48 forks source link

Default file listing mode with s3 is very slow #229

Open sedlund opened 7 months ago

sedlund commented 7 months ago

Pre-Submission checklist

What version of Round Sync are you using (About -> App version)?

2.2.3

What problem are you trying to solve?

When listing a larger (180 files) directory - round-sync by default attempts to get the modification time of each file which takes 67 seconds on a minio s3 bucket

What should Round Sync be able to do differently to help with this problem?

Options:

  1. allow to call rclone with RCLONE_USE_SERVER_MODTIME=true, which reduces the directory listing from 67 seconds to 4.
  2. allow to choose listing method for names and sizes without mod times.
sedlund commented 7 months ago

After thinking about this it should be resolved upstream with a better default configuration for all users that want to get a file listing with size and time.

But there will be other cases where it will be useful to add arbitrary options

walterwick commented 6 months ago

Any news? pls fix this.

sedlund commented 6 months ago

the app can set the environment variable to fix this until better default config for s3 remotes in rclone is set

export RCLONE_USE_SERVER_MODTIME=true

walterwick commented 6 months ago

how and where to do it.

sedlund commented 6 months ago

after this line:

https://github.com/newhinton/Round-Sync/blob/2b1bfb5eb1aa20d59db2103b35312ff1edc4251e/app/src/main/java/ca/pkay/rcloneexplorer/RcloneRcd.java#L169

add:

environmentValues.add("RCLONE_USE_SERVER_MODTIME=true");

walterwick commented 6 months ago

when is the new version coming out, I couldn't build it

newhinton commented 6 months ago

when is the new version coming out

When i get to it. There will never be timelines.

But there will be other cases where it will be useful to add arbitrary options

Yeah, that is a much requested feature. For now i would guide you to this ticket: #152 However, i still dont have a good idea how to solve this.