Open ax3l opened 1 year ago
For DASK-style parallel reading, we would like to avoid the overhead of opening and parsing file meta data for chunked, parallel reads.
This strategy is implemented already in MPI-parallel backends, where one rank does the open and parsing and broadcasts to all other ranks involved.
Exposing this to non-MPI parallel reads is started by ADIOS2 via https://github.com/ornladios/ADIOS2/pull/3651 and we want to integrate it.
For DASK-style parallel reading, we would like to avoid the overhead of opening and parsing file meta data for chunked, parallel reads.
This strategy is implemented already in MPI-parallel backends, where one rank does the open and parsing and broadcasts to all other ranks involved.
Exposing this to non-MPI parallel reads is started by ADIOS2 via https://github.com/ornladios/ADIOS2/pull/3651 and we want to integrate it.