Open yma11 opened 3 months ago
@oerling Do you have any idea about this? I think a file cache entry is easy to be larger than 8M
, why we limit to this size here?
@xiaoxmeng @zacw7 Do you guys happen to know about this? Thanks.
SsdRun only reserves 23 bits (out of 64 bits) for size. Maybe we can expand it to 128 bits.
velox/common/caching/SsdFile.h
@yma11 what's the loadQuantum size used in the query? Thanks!
It's 256MB
. So I need to set it 8M
if want to enable SSD cache?
It's
256MB
. So I need to set it8M
if want to enable SSD cache?
@yma11 that's the current implementation limitation which need to fix @zacw7. We shall also put limitation on the max size of loadQuantum that we support.
Bug description
When use AsyncDataCache together with SSDCache enabled, I got an error about size check failure on file_entry_size vs
8M
. Here is the whole stack:System information
Velox System Info v0.0.2 Commit: 6ea98b611d27c081cf07291c2f9b05fdca332e24 CMake Version: 3.28.3 System: Linux-5.4.0-156-generic Arch: x86_64 C++ Compiler: /usr/bin/c++ C++ Compiler Version: 9.4.0 C Compiler: /usr/bin/cc C Compiler Version: 9.4.0 CMake Prefix Path: /usr/local;/usr;/;/usr/local/lib/python3.8/dist-packages/cmake/data;/usr/local;/usr/X11R6;/usr/pkg;/opt
Relevant logs
No response