This PR adds a soft-reference based cache for recently fetched fragments from S3 for faster lookup.
This cache would be especially useful for smaller files which have fewer fragments and can fit in the cache.
Documentation:
Removed the config parameter S3.maxFragmentSize since we won't use a buffer pool internally anymore.
Updated defaults for following S3 specific instructions: maxConcurrentRequests, readAheadCount, fragmentSize, maxCacheSize.
This issue was auto-generated
PR: https://github.com/deephaven/deephaven-core/pull/5357 Author: malhotrashivam
Original PR Body
This PR adds a soft-reference based cache for recently fetched fragments from S3 for faster lookup. This cache would be especially useful for smaller files which have fewer fragments and can fit in the cache.
Documentation:
S3.maxFragmentSizesince we won't use a buffer pool internally anymore.maxConcurrentRequests,readAheadCount,fragmentSize,maxCacheSize.