search

pdidev / pdi

The PDI Data Interface
https://pdi.dev
BSD 3-Clause "New" or "Revised" License
6 stars 0 forks source link

decl'hdf5 feature: posibility to choose at low level of HDF5 (independent or collective) #491

Open jmorice91 opened 6 days ago

jmorice91 commented 6 days ago

For more details see section "Low-level collective vs. independent I/O" in the webpage: https://github.com/HDFGroup/hdf5/blob/develop/doc/parallel-compression.md

After the choice of mpio pointer (collective of independent) with the following code:

H5Pset_dxpl_mpio(dxpl_id, H5FD_MPIO_COLLECTIVE)

line 291 of "file_op.cxx" in main.

We want to add the possibility to use the "low level independent I/O" in adding this line:

H5Pset_dxpl_mpio_collective_opt(dxpl_id, H5FD_MPIO_INDIVIDUAL_IO); /* Try independent I/O */

The second argument can be H5FD_MPIO_INDIVIDUAL_IO or H5FD_MPIO_COLLECTIVE_IO. This parameter is collective by default.

jbigot commented 15 hours ago

Seems to be related to parallel compression. I don't think we expose it in Decl'HDF5 yet.