JP2 not recognized as a supported file format

[SFrav]

Using SIAC-V2.3.5 and GDAL 2.3.3 on a recent Sentinel 2 observation returns the following error: ... .jp2' not recognized as a supported file format.

This is returned after the following output: Doing per pixel angle resampling.

Do you have any suggestions on how to address this?

P.S. Thank you for contributing to open source EO tools. This has the potential to be quite useful for batch processing.

[MarcYin]

Hi,

This error generally comes from either gdal not compiled with jpg2000 driver or the jp2 file is corrupted. So, firstly please check the file is OK. If the file is OK, you will probably need to uninstall gdal first and reinstall gdal with conda install -c conda-forge gdal>2.1.

Thanks a lot for your testing of SIAC and it has been used to do large processing (thousands of files) on the UK Jasmin platform. If you have any issue of using it to do large scale processing, we are happy to help you to resolve it.

Marc.

[SFrav]

Hi Marc, Thanks for the speedy response.

I see the issue - I have a few versions of GDAL on my system and was looking at the formats in the OSGEO4W install, not the Anaconda virtual environment install. I've reinstalled gdal as you suggested above, but still no jpg2000 drivers. I'll try and start again with a new environment and see if that helps.

It should be an easy fix, so will mark as closed.

By the way, there was an extra dependency that I needed to install: gp_emulator

[MarcYin]

Hi, thanks a lot for the reminds and I will add it to the dependency list in the next version.

[SFrav]

All working well now. Just waiting on MCD43 downloads to test the correction. Thanks again

[MarcYin]

Hi, the downloading of MCD43 is bit slow, so for batch processing, you may consider to get hold of all the required MCD43 in the first place as earthdata.nasa.gov would only allow you to download 5 file at the same time.

[SFrav]

Maybe an issue to a gdal versioning, I have the following error: ERROR 1: TIFFRewriteDirectory:Error fetching directory count. This occurs towards the end of the function, after the output "Composing RGB".

I'm using gdal 2.2.2 and libgdal 2.2.2, in Python 3.6.

Two posts suggest that this is related to gdal

• https://trac.osgeo.org/gdal/ticket/4029
• http://maptools-org.996276.n3.nabble.com/Error-in-TIFFWriteDirectory-when-writting-multi-page-TIFF-td876.html

I thought it might be to do with gdal.Translate (in _compose_rgb, line 573 in the_correction.py), but gdal.Translate works well when testing in Python eg, this works: gdal.Translate('output.tif', input, format = 'PNG', widthPct=10, heightPct=10, resampleAlg=gdal.GRA_Bilinear).

Any suggestions would be appreciated. No rush - enjoy your weekend!

[MarcYin]

Hi, I saw this error appeared sometime (rarely) before but cannot find out the reason behind it. When I checked the composed RGB, they are all OK so it's bizarre. I have looked through the issues related to this, it seems that this error is likely to related to libtiff library used in gdal. So, again this is likely to be a gdal issue so maybe a fresh installation of gdal in a new environment is required to make sure that the libraries are compatible and up to date. Suggested gdal installation is with: conda update --all -c conda-forge conda install -c conda-forge gdal>2.1

[SFrav]

I've been trying a few different configurations this one progressed the furthest: conda env create --file=saic.yml #Using the environment yml on github conda activate siac conda install -c f0xy -c conda-forge siac conda update --all -c conda-forge conda install -c conda-forge "gdal>2.1" #Not needed as update does the job

It gave an error when doing pixel angle resampling File "C:\Users\Simon\Anaconda3\envs\siac\lib\site-packages\SIAC\s2_angle.py", line 316, in resample_s2_angles example_file = toa_refs[1] IndexError: list index out of range

Also, I noticed that the BOA_RGB.tif file was created from previous runs.

I'll keep at it...

[MarcYin]

Just realised that you are using Windows system, but the whole code is developed under linux platform and I have not tested it with a windows machine...It's good to know that it runs till composing RGB in your first run, so my suggestion of the installation would be:

conda env create --file=saic.yml #Using the environment yml on github
conda update --all -c conda-forge
conda activate siac
pip install gp_emulator # since it's missing
pip install https://github.com/MarcYin/SIAC/archive/master.zip

[SFrav]

Yes, I should have mentioned that I've been troubleshooting on a Windows machine. I tried on Arch Linux with pip, but didn't get that far.

After Composing RGB, I get a message saying that it's done.

This time I used pip and c++ build tools from visual studio. Using Python 3.6 in a virtual environment:

pip install -U pylibjpeg-openjpeg
pip install numpy==1.19.3 #error with 1.19.4
#Download gdal whl https://www.lfd.uci.edu/~gohlke/pythonlibs/#gdal
pip install GDAL-3.1.4-cp36-cp36m-win_amd64.whl
pip install gp_emulator
pip install https://github.com/MarcYin/SIAC/archive/master.zip