bp2ncd example from the manual doesn't work

[sungeunbae]

I am practising with the manual (1.8.0) and the example code. Section 14.8.1 of manual and examples/C/global-array,

seb56@c061hk:~/pkg/adios-1.8.0/examples/C/global-array$ mpirun -n 4 ./adios_global seb56@c061hk:~/pkg/adios-1.8.0/examples/C/global-array$ bpls -latd adios_global.bp -n 10 integer NX scalar = 10 integer size scalar = 4 integer rank scalar = 0 double temperature {4, 10} = 0 / 39 / 19.5 / 11.5434 (0,0) 0 1 2 3 4 5 6 7 8 9 (1,0) 10 11 12 13 14 15 16 17 18 19 (2,0) 20 21 22 23 24 25 26 27 28 29 (3,0) 30 31 32 33 34 35 36 37 38 39

string temperature/description attr = "Global array written from 'size' processes"

So far so good. However, converting to netcdf seems to corrupt the data.

seb56@c061hk:~/pkg/adios-1.8.0/examples/C/global-array$ bp2ncd adios_global.bp local[0]: 80 local[0]: 80 local[0]: 80 local[0]: 80 seb56@c061hk:~/pkg/adios-1.8.0/examples/C/global-array$ ncdump adios_global.nc netcdf adios_global { dimensions: NX = 10 ; size = 4 ; temperature_0 = 80 ; rank = 1 ; variables: byte temperature(temperature_0) ; data:

temperature = 0, 0, 0, 0, 0, 0, 62, 64, 0, 0, 0, 0, 0, 0, 63, 64, 0, 0, 0, 0, 0, 0, 64, 64, 0, 0, 0, 0, 0, -128, 64, 64, 0, 0, 0, 0, 0, 0, 65, 64, 0, 0, 0, 0, 0, -128, 65, 64, 0, 0, 0, 0, 0, 0, 66, 64, 0, 0, 0, 0, 0, -128, 66, 64, 0, 0, 0, 0, 0, 0, 67, 64, 0, 0, 0, 0, 0, -128, 67, 64 ; }

From the manual, I was expecting to see temperature ranging 0...39

[pnorbert]

Thank you for the report. I confirmed this bug, it worked fine in 1.6.0 and fails like this in 1.7.0

On Fri, Apr 10, 2015 at 12:19 AM, Sung Bae notifications@github.com wrote:

I am practising with the manual (1.8.0) and the example code. Section 14.8.1 of manual and examples/C/global-array,

seb56@c061hk:~/pkg/adios-1.8.0/examples/C/global-array$ mpirun -n 4 ./adios_global seb56@c061hk:~/pkg/adios-1.8.0/examples/C/global-array$ bpls -latd adios_global.bp -n 10 integer NX scalar = 10 integer size scalar = 4 integer rank scalar = 0 double temperature {4, 10} = 0 / 39 / 19.5 / 11.5434 (0,0) 0 1 2 3 4 5 6 7 8 9 (1,0) 10 11 12 13 14 15 16 17 18 19 (2,0) 20 21 22 23 24 25 26 27 28 29 (3,0) 30 31 32 33 34 35 36 37 38 39 string temperature/description attr = "Global array written from 'size' processes"

So far so good. However, converting to netcdf seems to corrupt the data.

seb56@c061hk:~/pkg/adios-1.8.0/examples/C/global-array$ bp2ncd adios_global.bp local[0]: 80 local[0]: 80 local[0]: 80 local[0]: 80 seb56@c061hk:~/pkg/adios-1.8.0/examples/C/global-array$ ncdump adios_global.nc netcdf adios_global { dimensions: NX = 10 ; size = 4 ; temperature_0 = 80 ; rank = 1 ; variables: byte temperature(temperature_0) ; data:

temperature = 0, 0, 0, 0, 0, 0, 62, 64, 0, 0, 0, 0, 0, 0, 63, 64, 0, 0, 0, 0, 0, 0, 64, 64, 0, 0, 0, 0, 0, -128, 64, 64, 0, 0, 0, 0, 0, 0, 65, 64, 0, 0, 0, 0, 0, -128, 65, 64, 0, 0, 0, 0, 0, 0, 66, 64, 0, 0, 0, 0, 0, -128, 66, 64, 0, 0, 0, 0, 0, 0, 67, 64, 0, 0, 0, 0, 0, -128, 67, 64 ; }

From the manual, I was expecting to see temperature ranging 0...39

— Reply to this email directly or view it on GitHub https://github.com/ornladios/ADIOS/issues/49.

[pnorbert]

The problem is that 1) the example is using a transformation while 2) bp2ncd is an old code using low level adios functions and is not aware of transformations. bp2h5 works fine because it was rewritten using the public read API. The example works fine if you change the transformation in the XML file for variable 'temperature' from 'identity' to 'none'.

The NetCDF conversion needs the connection between the array variable's dimension and the scalars that define the dimensions. This is not provided by the public read API, so we have never rewritten this tool (and extend the API).

Another route to fix this is to detect the transformations and use other low-level functions to de-transform the data.