laurechr commented 4 years ago

the problem present in previous bug 1829 is explain with a litle prog and with an example

Expected behavior and actual behavior.

For example (please modify !!!): I expected to be able to open this raster file (with a link to the raster file, or it as an attachment) and it returns an error message instead.

Steps to reproduce the problem.

the python program is on server bug this is nsb_test_gdal.py

the raw image is geo_20190827-20190908_sd_2rlks.int is and the associated file is geo_20190827-20190908_sd_2rlks.int.rsc

create a source files directory and put image files create a destination file directory (easy to delete) launch the test with this command nsb_test_gdal.py -sd tgdal -sf geo_20190827-20190908_sd_2rlks.int -dd testima -lo 500

the aim is open a same file and create a "lo" tiff image ( lo is number a file to create) this test show my problem that the desallocation of dataset (the del command) generate a allocation problem!! 500 or 1000 is a number of images we want create for one run on our studies

this is the result on HPC with 8 cores: ('create fileima :', 'fileima_0') ('create fileima :', 'fileima_1') ('create fileima :', 'fileima_2') ('create fileima :', 'fileima_3') ('create fileima :', 'fileima_4') ('create fileima :', 'fileima_5') ('create fileima :', 'fileima_6') ('create fileima :', 'fileima_7') ('create fileima :', 'fileima_8') ('create fileima :', 'fileima_9') Segmentation fault

with a a same program in another context there is a segmentaion fault

Dec 10 09:57:05 luke16 kernel: [240697.584561] python[31495]: segfault at 1000000a9 ip 000014ae0c860408 sp 00007fff4348c470 error 4 in _gdal.so[14ae0c3c6000+1544000] Dec 10 09:57:05 luke16 kernel: [240697.584852] Code: f8 0f 9c c0 84 c0 0f 84 cd 01 00 00 8b 45 f8 8d 50 01 48 8b 45 d8 89 d6 48 89 c7 e8 b2 b7 f6 ff 48 8b 10 48 81 c2 a8 00 00 00 <48> 8b 12 48 89 c7 ff d2 89 45 e8 83 7d e8 01 74 1b 83 7d e8 00 74

Operating system

Linux luke 3.16.0-5-amd64 #1 SMP Debian 3.16.51-3+deb8u1 (2018-01-08) x86_64 python 2.7

GDAL version and provenance

2.2.4 but i have the problem on the oldest version

rouault commented 4 years ago

Works fine for me with GDAL 2.4 or above. I had to replace the "import nsbas.gdal as gdal" by "from osgeo import gdal". I'd presume something is wrong with your GDAL build or you might run into a bug that has been fixed by a later version.

You might also try running your script under Valgrind: valgrind python .... . Possibly with a debug build of GDAL

laurechr commented 4 years ago

thank's for your reply

so i explain the build configuration

in first i test my program with a gdal library compiled by the hpc team and shared for the users.

i can create 500 image !!

i think the configuration of my build is not correct:

with our software, a lot of libraries are embedded like gdal

we divide the file from compilations in function of type file c fortran perl python

so the exe of gdal are put on local bin and the python script are put on local python script we have 2 makefiel can you verify that the compilations rules are good?? these makefile is write by another person an i correct it in function on gdal version thanks in advance for your helps

the configuration file define by include ${NSBAS_SOURCE_DIR} /mk/build2.mk contains

Compiler

CC = gcc CXX = g++ FC = gfortran

Linker

LD = ${CC}

Compilation flags

CFLAGS = -O3 -Wall -Wextra -D_BSD_SOURCE -D_FILE_OFFSET_BITS=64 FFLAGS = -cpp -O3 -fcray-pointer -fbackslash -ffixed-line-length-none -ffree-line-length-none -fno-underscoring -D_FILE_OFFSET_BITS=64

Linkage flags

LDFLAGS = OMPFLAGS = -fopenmp -frecursive -lpthread

Specific libs

M_LIBS = -lm DL_LIBS = -ldl ICONV_LIBS = RESOLV_LIBS = PTHREAD_LIBS = -lpthread CXX_LIBS = -lstdc++ FORTRAN_LIBS = -lgfortran

Timer used in lapack

LAPACK_TIMER = INT_ETIME

Python flags

PYTHON_CFLAGS = -fPIC -I/usr/include/python2.7

SBAS_COMMON_SOURCE_DIR= ${NSBAS_SOURCE_DIR} /src/common GDAL_SOURCE_DIR= ${NSBAS_COMMON_SOURCE_DIR} /gdal/build GDAL_BUILD_DIR= ${NSBAS_COMMON_SOURCE_DIR} /gdal/build

INCLUDE_SEARCH_FLAGS= \ -I ${NSBAS_SOURCE_DIR} /contrib/fftw/api \ -I ${NSBAS_COMMON_SOURCE_DIR} /base -I ${NSBAS_COMMON_SOURCE_DIR} /blas \ -I ${NSBAS_COMMON_SOURCE_DIR} /math \ -I ${NSBAS_SOURCE_DIR} /include -I.

GDAL_CFLAGS = \ -I ${GDAL_BUILD_DIR} /port \ -I ${GDAL_BUILD_DIR} /gcore \ -I ${GDAL_BUILD_DIR} /ogr \ -I ${GDAL_BUILD_DIR} /alg \ -I ${GDAL_BUILD_DIR} /apps

CFLAGS+= ${INCLUDE_SEARCH_FLAGS} FFLAGS+= ${INCLUDE_SEARCH_FLAGS}

NSBAS_LIBS = ${NSBAS_COMMON_SOURCE_DIR} /math/libnsbas-math.a ${NSBAS_COMMON_SOURCE_DIR} /base/libnsbas-base.a BLAS_LIBS= ${NSBAS_COMMON_SOURCE_DIR} /blas/libnsbas-blas.a FFTW_LIBS= ${NSBAS_COMMON_SOURCE_DIR} /fftw/libnsbas-fftw3f.a HDF5_LIBS= \ ${NSBAS_COMMON_SOURCE_DIR} /hdf5/local/lib/libhdf5_hl.a \ ${NSBAS_COMMON_SOURCE_DIR} /hdf5/local/lib/libhdf5.a LAPACK_LIBS= ${NSBAS_COMMON_SOURCE_DIR} /lapack/libnsbas-lapack.a NETCDF_LIBS= ${NSBAS_COMMON_SOURCE_DIR} /netcdf/libnsbas-netcdf.a GDAL_LIBS= ${NSBAS_SOURCE_DIR} /src/common/gdal/build/libgdal.a ${NSBAS_COMMON_SOURCE_DIR} /hdf5/local/lib/libhdf5.a ${ICONV_LIBS} ${PTHREAD_LIBS} ${DL_LIBS} ${CXX_LIBS} ${M_LIBS} ROIPAC_LIBS= ${NSBAS_COMMON_SOURCE_DIR} /roipac/libnsbas-roipac.a LIBS= ${ROIPAC_LIBS} ${FFTW_LIBS} ${NSBAS_LIBS} ${LAPACK_LIBS} ${BLAS_LIBS} ${M_LIBS}

BIN= ${NSBAS_SOURCE_DIR} /bin

in attached files this the makefile for gdal exe and for gdal for python binding

thanks in advance for your helps

christophe laurent

De: "Even Rouault" notifications@github.com À: "OSGeo/gdal" gdal@noreply.github.com Cc: "CHRISTOPHE LAURENT" christophe.laurent@univ-grenoble-alpes.fr, "Author" author@noreply.github.com Envoyé: Mercredi 11 Décembre 2019 12:40:34 Objet: Re: [OSGeo/gdal] desalloc problem with GDAL (#2094)