Open hyoklee opened 8 years ago
I can confirm the problem and I have a unit test that demonstrates it. I think I have a fix too, but I don't have time to finish it just yet.
In TestNc4IospReading.issue273()
I'm comparing the NetcdfFile
produced via JNI calls to the C-library versus the NetcdfFile
produced by H5iosp
, which is pure Java.
Here's the JNI version, which is almost certainly the correct one:
netcdf /Users/cwardgar/Desktop/comp_complex_more\.h5\ (jni\) {
dimensions:
n = 4;
variables:
Structure {
Structure {
Structure {
short i;
int j;
} x;
double y(3);
} yy(3);
} phony_compound_var(n);
data:
phony_compound_var =
{
yy =
{
x = i =1
j =100000
y =
{100000.285657, 100.1, 1.1415926}
} cmp2(0)
{
x = i =2
j =200000
y =
{200000.151617, 200.2, 273.15}
} cmp2(1)
{
x = i =-1
j =-100000
y =
{-100000.285657, -100.1, -1.1415926}
} cmp2(2)
} cmp3(0)
{
yy =
{
x = i =3
j =300000
y =
{300000.285657, 300.3, 3.1415926}
} cmp2(0)
{
x = i =4
j =400000
y =
{400000.151617, 400.4, 473.15}
} cmp2(1)
{
x = i =-2
j =-200000
y =
{-200000.285657, -200.2, -2.1415926}
} cmp2(2)
} cmp3(1)
{
yy =
{
x = i =5
j =500000
y =
{500000.285657, 500.5, 5.1415926}
} cmp2(0)
{
x = i =6
j =600000
y =
{600000.151617, 600.6, 673.15}
} cmp2(1)
{
x = i =-3
j =-300000
y =
{-300000.285657, -300.3, -3.1415926}
} cmp2(2)
} cmp3(2)
{
yy =
{
x = i =7
j =700000
y =
{700000.275657, 700.7, 7.1415926}
} cmp2(0)
{
x = i =8
j =800000
y =
{800000.151617, 800.8, 873.15}
} cmp2(1)
{
x = i =-4
j =-400000
y =
{-400000.285657, -400.4, -4.1415926}
} cmp2(2)
} cmp3(3)
}
Here's the H5iosp
version:
netcdf /Users/cwardgar/Desktop/comp_complex_more\.h5 {
dimensions:
n = 4;
variables:
Structure {
Structure {
Structure {
short i;
int j;
} x;
double y(3);
} yy(3);
} phony_compound_var(n);
data:
phony_compound_var =
{
yy =
{
x = i =1
j =100000
y =
{100000.285657, 100.1, 1.1415926}
} yy(0)
{
x = i =16370
j =222298112
y =
{4.216208244882606E202, 7.897725394396646E-278, 4.95748257787734E-220}
} yy(1)
{
x = i =4710
j =65535
y =
{-1.005413417931744E-221, 1.9035990250237878E185, -1.0209102935083844E-182}
} yy(2)
} phony_compound_var(0)
{
yy =
{
x = i =3
j =300000
y =
{300000.285657, 300.3, 3.1415926}
} yy(0)
{
x = i =16393
j =444596224
y =
{4.067657446704244E202, 7.897725394396667E-278, -4.957482577877352E-220}
} yy(1)
{
x = i =-28058
j =65534
y =
{6.282463199079756E43, 1.90359902534037E185, 1.9380845872298713E63}
} yy(2)
} phony_compound_var(1)
{
yy =
{
x = i =5
j =500000
y =
{500000.285657, 500.5, 5.1415926}
} yy(0)
{
x = i =16404
j =666894336
y =
{9.217702764569826E74, -2.7200830941429086E23, 2.3817986479925644E-264}
} yy(1)
{
x = i =2355
j =65533
y =
{8.48325333365319E-133, -9.255967814244581E61, 1.9380845874631876E63}
} yy(2)
} phony_compound_var(2)
{
yy =
{
x = i =7
j =700000
y =
{700000.275657, 700.7, 7.1415926}
} yy(0)
{
x = i =16412
j =889192448
y =
{3.993382047615067E202, 7.897725394396687E-278, 4.281743078120584E44}
} yy(1)
{
x = i =18739
j =65532
y =
{8.483253333496778E-133, 1.9035990256569527E185, -4.8071669131868766E-123}
} yy(2)
} phony_compound_var(3)
}
The metadata match (after I made a change to H5header.makeVariableMember()
) but the actual data do not. Strangely, the phony_compound_var(i).yy(0)
records match but the others do not. Why? Is it an offset issue? Perhaps endian-ness?
Anyway, I'm shelving this one for now. The HDF5-reading code is extremely complex and I have no idea how long it would take me to figure it out. Would probably need to debug the C and Java libraries side-by-side to find the difference. My work is on the "issue273" branch.
For the following HDF5 file, netCDF-C can handle it correctly but netCDF-Java can't.
netCDF-C CDL output:
netCDF-Java NcML output:
It doesn't show i,j,x,y.