search

lutraconsulting / MDAL

Mesh Data Abstraction Library
http://www.mdal.xyz/
MIT License
160 stars 50 forks source link

Openning ugrid file causes core dump #188

Closed saberraz closed 4 years ago

saberraz commented 4 years ago

If I try to open this file: https://github.com/lutraconsulting/MDAL/blob/master/tests/data/ugrid/D-Flow1.2/bw_11_zonder_riviergrid_met_1dwtg_clm.nc

It causes core dump. To reproduce it, you can try and add it twice (while the first one is open).

PeterPetrik commented 4 years ago

@saberraz any chance to attach core dump report from linux?

saberraz commented 4 years ago 
[Thread 0x7fffc77fe700 (LWP 25613) exited]
HDF5-DIAG: Error detected in HDF5 (1.10.4) thread 140737047364800:
  #000: ../../../src/H5A.c line 425 in H5Aopen(): unable to load attribute info from object header for attribute: 'File Type'
    major: Attribute
    minor: Can't open object
  #001: ../../../src/H5Aint.c line 478 in H5A__open(): unable to load attribute info from object header for attribute: 'File Type'
    major: Attribute
    minor: Can't open object
  #002: ../../../src/H5Oattribute.c line 497 in H5O__attr_open_by_name(): can't open attribute
    major: Attribute
    minor: Can't open object
  #003: ../../../src/H5Adense.c line 398 in H5A__dense_open(): can't locate attribute in name index
    major: Attribute
    minor: Object not found
[New Thread 0x7fffc77fe700 (LWP 25649)]
[Thread 0x7fffc77fe700 (LWP 25649) exited]
[New Thread 0x7fffc77fe700 (LWP 25650)]
[New Thread 0x7fff6bd9b700 (LWP 25651)]
[Thread 0x7fffc77fe700 (LWP 25650) exited]
[Thread 0x7fff6bd9b700 (LWP 25651) exited]
[New Thread 0x7fff6bd9b700 (LWP 25652)]
[New Thread 0x7fffc77fe700 (LWP 25653)]
[New Thread 0x7fff6b163700 (LWP 25654)]
double free or corruption (out)
Thread 1 "qgis" received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffe5b704c0 (LWP 25592)]
PeterPetrik commented 4 years ago

I cannot reproduce it on master (macos) @wonder-sk can you reproduce it on your machine?

wonder-sk commented 4 years ago

I can't reproduce either

saberraz commented 4 years ago

See attached for the backtrace. bt.txt

PeterPetrik commented 4 years ago

@saberraz what is yours 1) netcdf version 2) hdf5 version 3) MDAL version 4) linux dist version

saberraz commented 4 years ago
  1. netcdf: 4.6.2 of Dec 12 2018 10:27:40 $
  2. hdf5: 1.10.4
  3. mdal 0.4.90
  4. Ubuntu 19.10
PeterPetrik commented 4 years ago

mine (not crashing) are

  1. netcdf 4.7.2
  2. hdf5 1.10.5
  3. mdal 0.4.9x (master)
  4. macos catalina 10.15.x
PeterPetrik commented 4 years ago

The array that crashes is only array in our test suite that contains BYTE values (1-128). for MDAL 0.5.0 i changed the code so it reads the result to byte/int instead of double, since we will need it for the next release and reading to double caused some random problems on other platforms.

@vcloarec as for debugging I suggest these tries (in any order)

  1. try to load the file with latest production version of MDAL and see if it crashes (you can just use tool mdalinfo to load the file, you do not need to compile it with QGIS)
  2. use ncdump on the file (even multiple times) and see if the result is consistent (so each run gives same results or the results for this array is diffefrent or even crashes sometimes) ==> this would suggest bug in netcdf
  3. download (from git) and complile latest netcdf and compile MDAL with latest netcdf . See if the mdalinfo still crashes. ==> if not it suggest bug in netcdf that was fixed, maybe you can even look at netfdf issue tracker to see if you can find which one ... if this is the case and 1) is not crushing, we can create C-preprocessing if-define for netcdf version in the code to
  4. if ncdump is not crushing, there is still an option we use the netcdf API wrong (e.g sending there some uninitialized variable or so). I suggest using valgrind - memcheck to find some leaks or some use of uninitialzed stuff.
PeterPetrik commented 4 years ago

with netcdf 4.6.2, ncdump crash with netcdf master, ncdump ok

crash on Ubuntu is here: https://github.com/lutraconsulting/MDAL/blob/3add9dee353ebf96a86d57367221d5abd2ecdcf9/mdal/frmts/mdal_netcdf.cpp#L147 on call to nc_get_vars_uchar

PeterPetrik commented 4 years ago

@wonder-sk what is yours netcdf version?

on win there is netcdf, 4.6.2-2

wonder-sk commented 4 years ago

on my ubuntu 18.04 the full package version (of netcdf-bin) is 1:4.6.0-2build1

vcloarec commented 4 years ago

ncdump fails with seg default with version before 4.7.1. Since 4.7.1, ncdump reads all data.

ldebek commented 4 years ago

It crashes (or making QGIS not responding) for me as well. My compilation info on the screenshot below. image

PeterPetrik commented 4 years ago

windows packages are updated today to netcdf, 4.7.3-1

PeterPetrik commented 4 years ago

@DamnBack can you try with the tomorrow nightly windows build if it goes away?

ldebek commented 4 years ago

@PeterPetrik With current nightly windows build QGIS is not crashing anymore.