ornladios / ADIOS

The old ADIOS 1.x code repository. Look for ADIOS2 for new repo
https://csmd.ornl.gov/adios
Other
54 stars 40 forks source link

data Reading is unusually slow #145

Closed lwh1990 closed 6 years ago

lwh1990 commented 6 years ago

Hi, I recently make some read/write test using ADIOS. The dataset is a 2-dimension float array(1000X1000). Each time I read a 5X5 grid until the whole dataset is all read. It costs about 100 second which I think is too slow. Same test with HDF5 only cost 2 second. So is there anything I did wrong? Or maybe I didn't set buffer for read which lead to each read must accessing disk? BTW, I use adios_selection_boundingbox and adios_schedule_read to perform the reading. Thanks.

pnorbert commented 6 years ago

Hi. ADIOS does not do buffering at read (pre-fetching). Each 5x5 read operation will be access the disk. You need to read in the whole thing if you want good performance.

lwh1990 commented 6 years ago

Thanks for the reply!