ornladios / ADIOS

The old ADIOS 1.x code repository. Look for ADIOS2 for new repo
https://csmd.ornl.gov/adios
Other
54 stars 41 forks source link

bp2ncd example from the manual doesn't work #49

Open sungeunbae opened 9 years ago

sungeunbae commented 9 years ago

I am practising with the manual (1.8.0) and the example code. Section 14.8.1 of manual and examples/C/global-array,

seb56@c061hk:~/pkg/adios-1.8.0/examples/C/global-array$ mpirun -n 4 ./adios_global seb56@c061hk:~/pkg/adios-1.8.0/examples/C/global-array$ bpls -latd adios_global.bp -n 10 integer NX scalar = 10 integer size scalar = 4 integer rank scalar = 0 double temperature {4, 10} = 0 / 39 / 19.5 / 11.5434 (0,0) 0 1 2 3 4 5 6 7 8 9 (1,0) 10 11 12 13 14 15 16 17 18 19 (2,0) 20 21 22 23 24 25 26 27 28 29 (3,0) 30 31 32 33 34 35 36 37 38 39

string temperature/description attr = "Global array written from 'size' processes"

So far so good. However, converting to netcdf seems to corrupt the data.

seb56@c061hk:~/pkg/adios-1.8.0/examples/C/global-array$ bp2ncd adios_global.bp local[0]: 80 local[0]: 80 local[0]: 80 local[0]: 80 seb56@c061hk:~/pkg/adios-1.8.0/examples/C/global-array$ ncdump adios_global.nc netcdf adios_global { dimensions: NX = 10 ; size = 4 ; temperature_0 = 80 ; rank = 1 ; variables: byte temperature(temperature_0) ; data:

temperature = 0, 0, 0, 0, 0, 0, 62, 64, 0, 0, 0, 0, 0, 0, 63, 64, 0, 0, 0, 0, 0, 0, 64, 64, 0, 0, 0, 0, 0, -128, 64, 64, 0, 0, 0, 0, 0, 0, 65, 64, 0, 0, 0, 0, 0, -128, 65, 64, 0, 0, 0, 0, 0, 0, 66, 64, 0, 0, 0, 0, 0, -128, 66, 64, 0, 0, 0, 0, 0, 0, 67, 64, 0, 0, 0, 0, 0, -128, 67, 64 ; }

From the manual, I was expecting to see temperature ranging 0...39

pnorbert commented 9 years ago

Thank you for the report. I confirmed this bug, it worked fine in 1.6.0 and fails like this in 1.7.0

On Fri, Apr 10, 2015 at 12:19 AM, Sung Bae notifications@github.com wrote:

I am practising with the manual (1.8.0) and the example code. Section 14.8.1 of manual and examples/C/global-array,

seb56@c061hk:~/pkg/adios-1.8.0/examples/C/global-array$ mpirun -n 4 ./adios_global seb56@c061hk:~/pkg/adios-1.8.0/examples/C/global-array$ bpls -latd adios_global.bp -n 10 integer NX scalar = 10 integer size scalar = 4 integer rank scalar = 0 double temperature {4, 10} = 0 / 39 / 19.5 / 11.5434 (0,0) 0 1 2 3 4 5 6 7 8 9 (1,0) 10 11 12 13 14 15 16 17 18 19 (2,0) 20 21 22 23 24 25 26 27 28 29 (3,0) 30 31 32 33 34 35 36 37 38 39 string temperature/description attr = "Global array written from 'size' processes"

So far so good. However, converting to netcdf seems to corrupt the data.

seb56@c061hk:~/pkg/adios-1.8.0/examples/C/global-array$ bp2ncd adios_global.bp local[0]: 80 local[0]: 80 local[0]: 80 local[0]: 80 seb56@c061hk:~/pkg/adios-1.8.0/examples/C/global-array$ ncdump adios_global.nc netcdf adios_global { dimensions: NX = 10 ; size = 4 ; temperature_0 = 80 ; rank = 1 ; variables: byte temperature(temperature_0) ; data:

temperature = 0, 0, 0, 0, 0, 0, 62, 64, 0, 0, 0, 0, 0, 0, 63, 64, 0, 0, 0, 0, 0, 0, 64, 64, 0, 0, 0, 0, 0, -128, 64, 64, 0, 0, 0, 0, 0, 0, 65, 64, 0, 0, 0, 0, 0, -128, 65, 64, 0, 0, 0, 0, 0, 0, 66, 64, 0, 0, 0, 0, 0, -128, 66, 64, 0, 0, 0, 0, 0, 0, 67, 64, 0, 0, 0, 0, 0, -128, 67, 64 ; }

From the manual, I was expecting to see temperature ranging 0...39

— Reply to this email directly or view it on GitHub https://github.com/ornladios/ADIOS/issues/49.

pnorbert commented 9 years ago

The problem is that 1) the example is using a transformation while 2) bp2ncd is an old code using low level adios functions and is not aware of transformations. bp2h5 works fine because it was rewritten using the public read API. The example works fine if you change the transformation in the XML file for variable 'temperature' from 'identity' to 'none'.

The NetCDF conversion needs the connection between the array variable's dimension and the scalars that define the dimensions. This is not provided by the public read API, so we have never rewritten this tool (and extend the API).

Another route to fix this is to detect the transformations and use other low-level functions to de-transform the data.