GDAL version must have libtiff version that supports BigTiff - Githubissues

malonzo47 / pynita_GUI

pynita_GUI is python GUI based implementation of Noise Insensitive Trajectory Algorithm (NITA)

4 stars 3 forks source link

GDAL version must have libtiff version that supports BigTiff #85

Closed malonzo47 closed 4 years ago

malonzo47 commented 4 years ago

If this is not the case (i.e., with newest version of gdal), then the program will crash when trying to load a large image stack. What's odd though, is that I think this image stack can cause issues even when it's less than 2 GB or 4 GB or whatever the BigTiff threshold is.

Question: Is the relevant gdal version the one that's within python (e.g., Anaconda) or the standalone gdal installed along with OGR?

FreelanceDev217 commented 4 years ago

After reinstallation of gdal following this tutorial it still fails to allocate memory. Also I tried to check BIGTIFF support but the installation I have does not. C:\Users\Piao David>gdalinfo --format BIGTIFF ERROR 1: --format option given with format 'BIGTIFF', but that format not recognised. Use the --formats option to get a list of available formats, and use the short code (i.e. GTiff or HFA) as the format identifier. I am exploring the solution to support BIGTIFF.

FreelanceDev217 commented 4 years ago

Well, the python WHL package file I use for Windows from here include complete GDAL. screenshot_57 And checking BigTIFF support from Python using the code below from Stackoverflow, it seems to be fine. screenshot_60 Do you know how to check if my installation is fine with BIGTIFF format?

(About the memory allocation issue, I still have it and exploring)

FreelanceDev217 commented 4 years ago

The memory allocation fails at X block offset 1, Y block offset 2. Do you have any idea from the image below? screenshot_61

FreelanceDev217 commented 4 years ago

Please let me know your python version. I am using x32 and some articles say that x64 version would not have this memory allocation problem.

malonzo47 commented 4 years ago

I have 64 bit python on 64 bit windows.

FreelanceDev217 commented 4 years ago

OK, so I believe the memory issue is from that. With 32bit python, it's limited to allocate 2GB memory at a time and it causes this error.

malonzo47 commented 4 years ago

I've never seen the memory allocation issue. I can't even tell if this is related to BigTiff or not. What about allocating more RAM like here:

export GDAL_CACHEMAX=4000

no idea if that's useful or not... (source: https://github.com/tum-gis/cesium-terrain-builder-docker/issues/3)

FreelanceDev217 commented 4 years ago

I believe this is related to the 32bit/64bit. I tried GDAL_CACHEMAX option and it does not help.

So I want to proceed this way.

I want you to try the windows installation steps on a fresh new Windows environment (64bit).
We will resolve the installation steps together.
I want you to prepare a good example that can test BIGTIFF support status.
We will resolve BIGTIFF support issue if it fails.

malonzo47 commented 4 years ago

Since we never actually encountered the BigTiff issue, I will close. The issue was indeed related to 32 bit python and limited memory allocation for large image stacks.