metasim
commented
5 years ago NB: Does not happen on Linux.
Closed metasim closed 5 years ago
metasim
commented
5 years ago NB: Does not happen on Linux.
pomadchin
commented
5 years ago I will try to reproduce it on my machine:
metasim
commented
5 years ago I think it's happening in the JPEG2000 test. Actually, from the backtrace it looks like the MRF test.
metasim
commented
5 years ago More version info:
$ brew info osgeo-gdal
osgeo/osgeo4mac/osgeo-gdal: stable 2.4.1 (bottled), HEAD
GDAL: Geospatial Data Abstraction Library
https://www.gdal.org/
/usr/local/Cellar/osgeo-gdal/2.4.1_1 (198 files, 55.9MB) *
Poured from bottle on 2019-03-29 at 13:55:12
From: https://github.com/osgeo/homebrew-osgeo4mac/blob/master/Formula/osgeo-gdal.rb
==> Dependencies
Required: pkg-config ✔, libiconv ✘, expat ✔, zlib ✔, qhull ✔, curl-openssl ✘, libpng ✘, freexl ✔, geos ✘, jpeg-turbo ✘, json-c ✔, giflib ✔, libpq ✘, sqlite ✘, pcre ✔, libtiff ✔, numpy ✘, armadillo ✘, sfcgal ✘, ant ✘, swig ✘, mdbtools ✔, libzip ✔, openssl ✔, cryptopp ✔, osgeo-libgeotiff ✘, osgeo-libspatialite ✘, osgeo-libkml ✔, osgeo-netcdf ✘, osgeo-hdf4 ✔, hdf5 ✘, cfitsio ✔, epsilon ✔, jasper ✔, libdap ✘, libxml2 ✔, openjpeg ✘, zstd ✘, webp ✔, unixodbc ✔, xerces-c ✔, xz ✔, osgeo-proj ✘, osgeo-postgresql ✘
$ brew info openjpeg
openjpeg: stable 2.3.1 (bottled), HEAD
Library for JPEG-2000 image manipulation
jamesmcclain
commented
5 years ago Preliminarily, this looks like it may be an issue related to the MRF driver. The stack trace shows execution passing through libgdalwarp_bindings.dylib and through the core of GDAL, then into the MRF driver where it encounters a problem.
In order to investigate further, it might help us to have a core file if we are not able to reproduce the problem. Or better yet, since I will not be able to match your environment exactly, a walk through the stack in a debugger as described here would be envaluable. Without that, I will not be able to investigate this until after work (because my MacOS installation is on my personal computer).
I am especially interested to see what the parameters were to the first function in the core of GDAL called by libgdalwarp_bindings.dylib: it should be easy to see whether they look "normal", and if so to create a small program to attempt to isolate the issue.
jamesmcclain
commented
5 years ago @metasim I was not able to reproduce (please check the top left of the screenshot to ensure that I typed the correct command to launch the tests [I did so without the single quotes around the last two arguments and got the same result]).
I did noticed that I have a different version of GDAL than you do: mine responds to the command brew info gdal whereas yours responds to brew info osgeo-gdal. If those are in fact different builds, that might account for different behavior.
metasim
commented
5 years ago I didn't realize the non osgeo gdal would work. I'll uninstall the osgeo one and try things out. Thanks for hunting down that difference.
jamesmcclain
commented
5 years ago I didn't realize the non osgeo
gdalwould work. I'll uninstall theosgeoone and try things out. Thanks for hunting down that difference.
No problem, my fingers are crossed!
metasim
commented
5 years ago :( Still getting the crash. I'm inclined to close it because it appears to be unique to me. I can provide a core file if it's helpful. Here's what I came up with:
(lldb) thread backtrace
* thread #25, stop reason = signal SIGSTOP
frame #0: 0x00007fff673ddb66 libsystem_kernel.dylib`__pthread_kill + 10
frame #1: 0x00007fff675a8080 libsystem_pthread.dylib`pthread_kill + 333
frame #2: 0x00007fff673391ae libsystem_c.dylib`abort + 127
frame #3: 0x000000010628cc3f libjvm.dylib`os::abort(bool) + 25
frame #4: 0x00000001063b4e26 libjvm.dylib`VMError::report_and_die() + 2304
frame #5: 0x000000010628e882 libjvm.dylib`JVM_handle_bsd_signal + 1131
frame #6: 0x000000010628aabf libjvm.dylib`signalHandler(int, __siginfo*, void*) + 47
* frame #7: 0x00007fff6759bf5a libsystem_platform.dylib`_sigtramp + 26
frame #8: 0x00000001235c1eee libgdal.20.dylib`GDAL_LercNS::BitMask::CountValidBits() const + 42
frame #9: 0x0000000122e7604a libgdal.20.dylib`GDAL_MRF::LERC_Band::Decompress(GDAL_MRF::buf_mgr&, GDAL_MRF::buf_mgr&) + 922
frame #10: 0x000000012302f914 libgdal.20.dylib`GDAL_MRF::GDALMRFRasterBand::IReadBlock(int, int, void*) + 1148
frame #11: 0x00000001231de8f1 libgdal.20.dylib`GDALRasterBand::GetLockedBlockRef(int, int, int) + 373
frame #12: 0x00000001231f2574 libgdal.20.dylib`GDALRasterBand::IRasterIO(GDALRWFlag, int, int, int, int, void*, int, int, GDALDataType, long long, long long, GDALRasterIOExtraArg*) + 2220
frame #13: 0x00000001231ddc99 libgdal.20.dylib`GDALRasterBand::RasterIO(GDALRWFlag, int, int, int, int, void*, int, int, GDALDataType, long long, long long, GDALRasterIOExtraArg*) + 705
frame #14: 0x0000000122e2a6eb libgdal.20.dylib`GDALWarpOperation::WarpRegionToBuffer(int, int, int, int, void*, GDALDataType, int, int, int, int, double, double, double, double) + 699
frame #15: 0x0000000122e2b3a8 libgdal.20.dylib`GDALWarpOperation::WarpRegionToBuffer(int, int, int, int, void*, GDALDataType, int, int, int, int, double, double) + 58
frame #16: 0x00000001231572c2 libgdal.20.dylib`VRTWarpedDataset::ProcessBlock(int, int) + 226
frame #17: 0x000000012315769b libgdal.20.dylib`VRTWarpedRasterBand::IReadBlock(int, int, void*) + 57
frame #18: 0x00000001231de8f1 libgdal.20.dylib`GDALRasterBand::GetLockedBlockRef(int, int, int) + 373
frame #19: 0x00000001231e3dcb libgdal.20.dylib`GDALRasterBand::ComputeRasterMinMax(int, double*) + 1245
frame #20: 0x00000001201396ea libgdalwarp_bindings.dylib`locked_dataset::get_band_max_min(int, int, int, double*, int*) + 122
frame #21: 0x000000012013949f libgdalwarp_bindings.dylib`get_band_min_max + 1359
frame #22: 0x0000000120131ec8 libgdalwarp_bindings.dylib`Java_com_azavea_gdal_GDALWarp_get_1band_1min_1max + 184
frame #23: 0x000000010e8bd337
...I walked up the stack some running frame variable to see if I could get the function parameters but lldb didn't report anything.
e.g.
(lldb) frame select 20
frame #20: 0x00000001201396ea libgdalwarp_bindings.dylib`locked_dataset::get_band_max_min(int, int, int, double*, int*) + 122
libgdalwarp_bindings.dylib`locked_dataset::get_band_max_min:
0x1201396ea <+122>: movq -0x30(%rbp), %rdx
0x1201396ee <+126>: movl $0x1, (%rdx)
0x1201396f4 <+132>: jmp 0x120139736 ; <+198>
0x1201396f9 <+137>: movq -0x38(%rbp), %rdi
(lldb) frame variable
(lldb)Not sure from this what other kind of core file inspection is helpful.
pomadchin
commented
5 years ago My update: I could not reproduce it on my machine :/
metasim
commented
5 years ago Thanks guys. Sorry to waste your time.
jamesmcclain
commented
5 years ago I walked up the stack some running frame variable to see if I could get the function parameters but lldb didn't report anything.
Ah, okay, that is unfortunate.
I am happy to revisit this issue later, if there is impetus.
jamesmcclain
commented
5 years ago Thanks guys. Sorry to waste your time.
No problem! I appreciate the engagement!
Reproduce
Clone
s22s/rasterframes, branchfeature/gdal-warp-upgrade.Or https://github.com/s22s/rasterframes/tree/feature/gdal-warp-upgrade
Then run:
Result
Stop of stack
Full report: hs_err_pid38339.log.zip
Environment
GDAL
Machine
JVM
Libraries
"com.azavea.gdal" % "gdal-warp-bindings" % "33.58d4965""com.azavea.geotrellis" %% "geotrellis-contrib-vlm" % "2.11.0""com.azavea.geotrellis" %% "geotrellis-contrib-gdal" % "2.11.0"rfSparkVersion := "2.3.2"rfGeoTrellisVersion := "2.2.0"rfGeoMesaVersion := "2.2.1"