search

Parallel-NetCDF / PnetCDF

Source code repository of PnetCDF library and utilities
https://parallel-netcdf.github.io
Other
80 stars 22 forks source link

Using GPU-Direct with pnetcdf? #96

Open jedwards4b opened 1 year ago

jedwards4b commented 1 year ago

Has anyone tried this?

wkliao commented 1 year ago

Hi, Jim I think it would be a great feature in PnetCDF. Could you point me to some use cases that can benefit from this feature?

jedwards4b commented 1 year ago

I am working on an asyncronous implementation of the parallelio library in cesm. I would like to prototype parallelio moving data from cpu memory to gpu memory, doing some data compression and/or data postprocessing on the gpu and then writing directly from the gpu to disk.

wkliao commented 1 year ago

Can I assume the parallel write part of your use case is the typical concurrent writes from multiple processes to a single shared file? e.g. using collective I/O mode.

Just a reminder, PnetCDF (and the classic NetCDF file formats) does not support data compression.