Parallel I/O

[GoogleCodeExporter]

Dear all,

I have a script that could write my seismological data (around 1GB which 
consists of 1000 different files) to one netCDF file and read the data back. 
Since my data is very big, I decided to write it in parallel (using pprocess). 
Unfortunately, I could not really write the data in this case (while the 
function works perfectly fine in serial). Do you have any idea how I could run 
my code in parallel (basically read and write the data)?

I have found some documentation for FORTRAN or other programming languages in 
the following address:

http://www.unidata.ucar.edu/software/netcdf/workshops/2010/pnetcdf/index.html

thanks in advance for your help

Kasra

Original issue reported on code.google.com by k.hosseinizad on 13 Aug 2012 at 2:27

[GoogleCodeExporter]

the python interface does not yet support the parallel IO capabilities of 
netCDF-4 and HDF5. Regarding pprocess, I don't know why that didn't work (or 
even whether it should).  Sorry, but I just don't have an experience in this 
area.

Original comment by whitaker.jeffrey@gmail.com on 14 Aug 2012 at 2:00

[GoogleCodeExporter]

Thanks for your comment. I will try to figure it out.

Best,
Kasra

Original comment by k.hosseinizad on 17 Aug 2012 at 11:48

[GoogleCodeExporter]

Hey did you or anyone else manage to get parallel I/O to work... this would be 
a most useful feature.  I see there is the pypnetcdf project that claims to do 
this but it doesn't have the same capabilities as netCDF4.

Original comment by dan.pott...@googlemail.com on 22 Jul 2013 at 10:10

[GoogleCodeExporter]

No, the parallel interface is not supported - and I don't have any plans to 
work on it right now.  

Original comment by whitaker.jeffrey@gmail.com on 22 Jul 2013 at 12:24

[GoogleCodeExporter]

Original comment by whitaker.jeffrey@gmail.com on 26 Feb 2014 at 2:04

• Changed state: Fixed

[GoogleCodeExporter]

Original comment by whitaker.jeffrey@gmail.com on 26 Feb 2014 at 2:04

• Changed state: Fixed