Problem importing EnMAP data into QGIS 3.36

[grantboxer]

I get an error trying to import an EnMAP L2A file into 3.36, see message below.

QGIS version: 3.36.1-Maidenhead QGIS code revision: 3e589453 Qt version: 5.15.3 Python version: 3.9.18 GDAL version: 3.8.4 GEOS version: 3.12.1-CAPI-1.18.1 PROJ version: Rel. 9.3.1, December 1st, 2023 PDAL version: 2.6.0 (git-version: 3fced5) Algorithm started at: 2024-04-09T09:54:54 Algorithm 'Import EnMAP L2A product' starting… Input parameters: { 'detectorOverlap' : 1, 'excludeBadBands' : True, 'file' : 'J:\Dropbox\EnMAP\Temp\Scene1\ENMAP01-____L2A-DT0000051064_20231121T023506Z_013_V010402_20240408T193411Z-METADATA.XML', 'outputEnmapL2ARaster' : 'TEMPORARY_OUTPUT', 'setBadBands' : True }

Traceback (most recent call last): File "C:\Users\Grant\AppData\Roaming\Python\Python39\site-packages\typeguard__init__.py", line 1033, in wrapper retval = func(*args, **kwargs) File "C:\Users/Grant/AppData/Roaming/QGIS/QGIS3\profiles\Grant/python/plugins\enmapboxplugin\enmapboxprocessing\algorithm\importenmapl2aalgorithm.py", line 249, in processAlgorithm redMin, redMax = reader.provider.cumulativeCut(redBandNo, 0.02, 0.98) TypeError: QgsRasterInterface.cumulativeCut(): argument 1 has unexpected type 'NoneType'

Execution failed after 14.63 seconds

Loading resulting layers Algorithm 'Import EnMAP L2A product' finished

[janzandr]

Is this problem reproducable with different L2A products?

[janzandr]

Can't reproduce this with my test product. Please share your product (e.g. send download link to andreas.janz@geo.hu-berlin.de).

[grantboxer]

Hi Andreas,

See this link “https://www.dropbox.com/scl/fo/fdp2zvf7rqhx7g6gm35lt/AI1bMrbDd2woi0jJY-NuUHA?rlkey=vstot99fm2c01nj7t5uzp3aqw&dl=0”.

I have the same problem with QGIS v 3.34 and I have re-run the python dependencies to make sure they are up to date. Let me know if you need any other information.

I could not reply to your email. Kept getting refused by server.

Cheers Grant

Grant Boxer Consultant Geologist (FAIG R.P. Geo) Perth, Western Australia Email: boxerg@iinet.net.au Mobile: 0418 954 059 Skype: grantboxer Website: https://grantboxer.github.io/

[jakimowb]

I can reproduce the error

[grantboxer]

I guess that's good!

[janzandr]

I can reproduce the error

@jakimowb can you reproduce the problem with the provided product from the dropbox, or can you also reproduce it with any other EnMAP L2A product?

[jakimowb]

@janzandr I tested this product only. The *.tifs are ok and can be visualized without any problem. The scene has many no-data areas which might affect the bandStats routines to setup the renderer:

 # setup default renderer
            layer = QgsRasterLayer(filename)
            reader = RasterReader(layer)
            redBandNo = reader.findWavelength(CreateSpectralIndicesAlgorithm.WavebandMapping['R'][0])
            greenBandNo = reader.findWavelength(CreateSpectralIndicesAlgorithm.WavebandMapping['G'][0])
            blueBandNo = reader.findWavelength(CreateSpectralIndicesAlgorithm.WavebandMapping['B'][0])
            redMin, redMax = reader.provider.cumulativeCut(redBandNo, 0.02, 0.98)
            greenMin, greenMax = reader.provider.cumulativeCut(greenBandNo, 0.02, 0.98)
            blueMin, blueMax = reader.provider.cumulativeCut(blueBandNo, 0.02, 0.98)
            renderer = Utils().multiBandColorRenderer(
                reader.provider, [redBandNo, greenBandNo, blueBandNo], [redMin, greenMin, blueMin],
                [redMax, greenMax, blueMax]
            )

relates to #842 (VRTs use absolute paths and therefore become useless if moved)

[janzandr]

Alright, with the provided product, I can also reproduce it. I guess it has to do with the specific product, which is mainly set to no data, which probably messes up the statistics measurement. Will fix this.

[grantboxer]

OK many thanks for your efforts!!

[janzandr]

@janzandr I tested this product only. The *.tifs are ok and can be visualized without any problem. The scene has many no-data areas which might affect the bandStats routines to setup the renderer:

Yupp.

[janzandr]

relates to #842 (VRTs use absolute paths and therefore become useless if moved)

This has nothing to do with this bug.

[grantboxer]

It might be that the salt lakes mess with the statistics as they are very bright.

[janzandr]

The stats are only estimated on a subset of the data. Together with the large amouint of no data pixels, the stats are most likely invalid, leading to the problem. I will fix this.

[grantboxer]

Any reason why there would be a lot of no data pixels? Looked like a good image on the preview.

[janzandr]

That would be a question for ENMAP_APPLICATION_SP@dlr.de.

[janzandr]

Alright, it was a bit more complicated, than I thought. The problem was with the automatic detection of bad bands. For bad band detection, we currently test only the data of the middle image line. e.g.:

Usually, this should always work, but given that strange product, all the bands where assigned as bad bands, which resulted in the bug down stream :-(

[jakimowb]

@janzandr how is the bad-band testing internally done?

[janzandr]

Now I check if the whole band consists of NoData. (Before the bug fix, I only tested the image line in the center of the image)

[grantboxer]

Does this now require a plug-in update?

[janzandr]

Yes, will be available with v3.14.1 this week.

[grantboxer]

Again, many thanks. The EnMap-Box plug-in is great so much appreciated getting the bugs out!

[jakimowb]

@janzandr you could use the stats for pixels whose quality class image shows a pixel value of 2

[janzandr]

yes, sounds good