search

actinia-org / actinia-core

Actinia Core is an open source REST API for scalable, distributed, high performance processing of geographical data that uses mainly GRASS GIS for computational tasks (DOI: https://doi.org/10.5281/zenodo.5879231) | Tutorial: https://actinia-org.github.io/actinia-core/ | Docker: https://hub.docker.com/r/mundialis/actinia-core
https://actinia.mundialis.de/
GNU General Public License v3.0
78 stars 23 forks source link

Evaluate Sentinel-1 processing tools using SNAP GPT #467

Open mmacata opened 12 months ago

mmacata commented 12 months ago

... use it for Geocoding of Sentinel-1 images, you may have a look here: https://github.com/NVE/actinia_modules_nve/blob/main/src/imagery/i.sentinel1.pyrosargeocode/ for a solution with pyrosar and without snappy. I hope I will be able to merge the addons that can be of more general interest into the official addons repo at some point in time...

Originally posted by @ninsbl in https://github.com/actinia-org/actinia-core/discussions/395#discussioncomment-6748254

ninsbl commented 12 months ago

Now, I added compiled / rendered manual pages for the modules: https://nve.github.io/actinia_modules_nve/

The module repository structure is copied from the official repo, so you should be able to install them with e.g.:

grass --tmp-location XY --exec g.extension -o extension="i.sentinel1.pyrosargeocode" \
url="https://github.com/NVE/actinia_modules_nve"

In the long run, my aim would be as mentioned to move the modules that are useful beyond our specific setup to the official GRASS GIS addon repository (e.g. i.asf.downlaod, i.sentinel1.pyrosargeocode, t.rast.stats, t.rast.reclass).

So, I am keen on getting your feedback and looking forward to both tickets and PRs in case, and I am totally open to make adjustments in the code...

griembauer commented 7 months ago

Hi @ninsbl, we finally found time to look into your great addons! I haven't managed to run i.sentinel1.pyrosargeocode yet, maybe you could advise:

I am trying i.sentinel1.pyrosargeocode -fndm input=/data/s1_input/S1A_IW_GRDH_1SDV_20230807T181458_20230807T181523_049773_05FC30_73BF.SAFE.zip output_directory=/data/s1_output aoi=/data/aoi_small/aoi_small.geojson elevation=ireland_dem_native_external polarizations=VV,VH where ireland_dem_native_external is an external DEM.tif linked via r.external, that can be also inspected using gdalinfo. Previously I installed the GDAL-GRASS driver (https://github.com/OSGeo/gdal-grass), I understand this is necessary for the addon and it would be great to have this in the requirements section of the addon manual.

I get:

WARNING: Cells NS- and EW-resolution in raster map
         <ireland_dem_native_external> differ. Using NS-resolution.
Start geocoding scene
S1A_IW_GRDH_1SDV_20230807T181458_20230807T181523_049773_05FC30_73BF.SAFE.zip
ERROR: Geocoding failed with the following error: raster input must be of
       type str, list or gdal.Dataset; is: PosixPath
       Please check the log files

Looking at the SNAP log file:

[...]
SEVERE [org.openide.util.RequestProcessor]: Error in RequestProcessor org.openide.loaders.FolderInstance$1R
java.lang.SecurityException: sealing violation
    at org.netbeans.JarClassLoader.doLoadClass(JarClassLoader.java:265)
    at org.netbeans.ProxyClassLoader.selfLoadClass(ProxyClassLoader.java:259)
    at org.netbeans.ProxyClassLoader.loadClass(ProxyClassLoader.java:196)
    at org.netbeans.ModuleManager$SystemClassLoader.loadClass(ModuleManager.java:779)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:370)
    at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
    at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
    at javax.imageio.spi.IIORegistry.registerApplicationClasspathSpis(IIORegistry.java:210)
    at javax.imageio.spi.IIORegistry.<init>(IIORegistry.java:138)
    at javax.imageio.spi.IIORegistry.getDefaultInstance(IIORegistry.java:159)
    at javax.imageio.ImageIO.<clinit>(ImageIO.java:66)
Caused: java.lang.ExceptionInInitializerError
    at org.openide.util.ImageUtilities.<clinit>(ImageUtilities.java:126)
    at org.openide.awt.ToolbarWithOverflow.setupOverflowButton(ToolbarWithOverflow.java:305)
    at org.openide.awt.ToolbarWithOverflow.<init>(ToolbarWithOverflow.java:137)
    at org.openide.awt.ToolbarWithOverflow.<init>(ToolbarWithOverflow.java:107)
    at org.openide.awt.ToolbarWithOverflow.<init>(ToolbarWithOverflow.java:94)
    at org.openide.awt.Toolbar.<init>(Toolbar.java:143)
    at org.openide.awt.ToolbarPool$Folder.acceptFolder(ToolbarPool.java:482)
    at org.openide.loaders.FolderInstance.acceptDataObject(FolderInstance.java:442)
    at org.openide.loaders.FolderInstance.defaultProcessObjects(FolderInstance.java:798)
    at org.openide.loaders.FolderInstance.access$000(FolderInstance.java:103)
    at org.openide.loaders.FolderInstance$1R.init(FolderInstance.java:693)
    at org.openide.loaders.FolderInstance$1R.run(FolderInstance.java:720)
    at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1443)
    at org.netbeans.modules.openide.util.GlobalLookup.execute(GlobalLookup.java:68)
    at org.openide.util.lookup.Lookups.executeWith(Lookups.java:303)
[catch] at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:2058)
[...]

Do you have any hints?

griembauer commented 7 months ago

Update: It seems that this specific error is caused by the value of externalDEMFile in the kwargs dict:

(Pdb) geocode_kwargs["externalDEMFile"]
PosixPath('/data/dem/ireland_dem_native_int_utm29n.tif')

which comes from the Path(... calls in def get_raster_gdalpath(map_name, check_linked=True):

Manually changing this to 

geocode_kwargs["externalDEMFile"]='/data/dem/ireland_dem_native_int_utm29n.tif'

solved the issue and processing was succesful!

griembauer commented 7 months ago

Update: Moving the discussion to https://github.com/NVE/actinia_modules_nve/issues/34

ninsbl commented 7 months ago

Update: It seems that this specific error is caused by the value of externalDEMFile in the kwargs dict:

(Pdb) geocode_kwargs["externalDEMFile"]
PosixPath('/data/dem/ireland_dem_native_int_utm29n.tif')

which comes from the Path(... calls in def get_raster_gdalpath(map_name, check_linked=True):

Manually changing this to

geocode_kwargs["externalDEMFile"]='/data/dem/ireland_dem_native_int_utm29n.tif'

solved the issue and processing was succesful!

I think the module should probably support a path to a GeoTiff as elevation input - in addition to a map name of a linked GeoTiff. GDAL-GRASS driver is no good solution as I think a DEM in GRASS format would not work anyway. SNAP has a bundled GDAL version without. So I think the SNAP run would fail with a DEM in GRASS format...

griembauer commented 7 months ago

After some tests I would recommend to go ahead with @ninsbl really great addon i.sentinel1.pyrosargeocode as the new standard for S1-Preprocessing in the actinia image. Snappy does not work with Python 3.11, and even if it did, calling SNAP via pyroSAR and gpt directly, as implemented in i.sentinel1.pyrosargeocode is reported to be the preferred option performance-wise. Currently there are some ongoing WIP PRs (will update here once ready):