search

HDFGroup / vol-log-based

Log VOL - an HDF5 VOL connector for storing data in a time-log layout in files
Other
3 stars 7 forks source link

Case studies #26

Open khou2020 opened 2 years ago

khou2020 commented 2 years ago

Propose case studies

wkliao commented 2 years ago

@khou2020 @yzanhua Please check h5bench and see if the I/O patterns there can be case studies. https://h5bench.readthedocs.io/en/latest/

khou2020 commented 2 years ago

@khou2020 @yzanhua Please check h5bench and see if the I/O patterns there can be case studies.

They are similar to what we have in terms of I/O pattern: h5bench read/write: a selection of block appending, strided (interleaved) I/O pattern that can be recreated with IOR. Metadata Stress: Subarray pattern but written in many rounds AMReX: Similar to FLASH Exerciser: Subarray pattern OpenPMD: Similar to FLASH E3SM-IO: Borrowed from our benchmark

The differences between them are only in file metadata (attrs, #vars, var dimensions ...) and a slight difference on data assignment (ex. different blocks per process between blocking appending patterns).