Written data not kept in cache

[seth-hunter]

What happened: Data written to JFS is not kept in local cache, despite cache being nowhere near size limit

What you expected to happen: JFS should prefer to keep recently-written data in cache to avoid unnecessary immediate reads from object storage (e.g. by backup software making first pass over data moved to JFS).

How to reproduce it (as minimally and precisely as possible): Fresh JFS volume, copy data to JFS, observe that cache briefly grows and then shrinks as data is uploaded to object storage.

Anything else we need to know?: Compression (zstd) enabled, writeback enabled, delay-upload set to 5 (behavior with default settings is better, but still not as desired -- more written data is kept in cache, but not all)

Environment:

• JuiceFS version: 1.0.0-beta3
• OS: AlmaLinux 8
• Kernel: 4.18
• Object storage (cloud provider and region, or self maintained): Backblaze B2
• Metadata engine info (version, cloud provider managed or self maintained): Sqlite

[T-TRz879]

When reading, it will be written to the cache; If you use S3, it seems that you can cache it locally and submit it to S3 in the background for you. That's just my opinion ：）

[SandyXSD]

In beta3 cached blocks are removed after they are uploaded to the object storage. This PR #2168 makes them stayed in the cache directory.

[seth-hunter]

Initial testing shows this is mostly fixed, thank you for the quick help! I am still seeing some unexpected cache misses n this scenario, but far less than before.