search

HDFGroup / vol-cache

HDF5 Cache VOL connector for caching data on fast storage layers and moving data asynchronously to the parallel file system to hide I/O overhead.
https://vol-cache.readthedocs.io
BSD 3-Clause "New" or "Revised" License
16 stars 8 forks source link

Question about HDF5_VOL_CONNECTOR setting #9

Closed gsjaardema closed 3 years ago

gsjaardema commented 3 years ago

In the README file for this VOL, the instructions say to set:

export HDF5_VOL_CONNECTOR="cache_ext config=config1.dat;under_vol=0;under_info={};"

However, in the notes for the VOL talk at the recent ECP Meeting, the instructions say to specify is as:

export HDF5_VOL_CONNECTOR="cache_ext config=config1.dat;under_vol=707;under_info={under_vol=0;under_info={}};"

If I use the first setting, then the output says "Do not have Async VOL underneath it"

If I use the second setting, then I get errors about not being able to create the output file. In my config file, I am specifying MEMORY for the cache storage type.

So --

Thanks.

zhenghh04 commented 3 years ago

Thank you for your comment. We suggest to stack async VOL below the cache VOL to perform the data migration from the node local storage to the global storage in the background. This is the suggested mode. I just updated README.md to include Async VOL in the stacking.