Closed PedroVenancio closed 3 years ago
saga_cmd dev_tools is now included and the descriptions et. al. are replaced on install (upload ongoing) It still warns about saga 7 support being unofficial.
saga_cmd dev_tools is now included and the descriptions et. al. are replaced on install (upload ongoing) It still warns about saga 7 support being unofficial.
@jef-n what versions of SAGA will be shipped with new osgeo4w installer? Only 7.* or also the very old one that we support on the current LTR?
saga 2 doesn't support current versions of proj.
saga_cmd dev_tools is now included and the descriptions et. al. are replaced on install (upload ongoing) It still warns about saga 7 support being unofficial.
Great @jef-n !
It's also needed to replace the SagaNameDecorator.py file, with the one given by SAGA team:
On other hand, I'm testing the new version and I'm getting an io_gdal
error:
io_gdal 0 -TRANSFORM 1 -RESAMPLING 3 -GRIDS "D:/Temp/processing_LqImgX/12045f9c2ae6477b98d7d621150df270/Rasterized.sgrd" -FILES "D:/Temp/processing_LqImgX/861dd24ba97f4c6984d21cad5aa91e9b/OUTPUT.tif"
imagery_segmentation "Grid Skeletonization" -INPUT "D:/Temp/processing_LqImgX/12045f9c2ae6477b98d7d621150df270/Rasterized.sgrd" -METHOD 0 -INIT_METHOD 1 -INIT_THRESHOLD 0.0 -CONVERGENCE 3 -RESULT "D:/Temp/processing_LqImgX/7641a17c0035420cad14c26c30b32ddb/RESULT.sdat" -VECTOR "D:/Temp/processing_LqImgX/e123b2cb66664c78963e32353b101575/VECTOR.shp"
C:\OSGeo4W64\bin>set SAGA=C:/OSGEO4~1/apps\saga
C:\OSGeo4W64\bin>set SAGA_MLB=C:/OSGEO4~1/apps\saga\modules
C:\OSGeo4W64\bin>PATH=C:\OSGEO4~1\apps\Python39\lib\site-packages\shapely\DLLs;C:\OSGEO4~1\apps\Python39\DLLs;C:\OSGEO4~1\apps\Python39\lib\site-packages\numpy\DLLs;C:\OSGEO4~1\apps\qgis-dev\bin;C:\OSGEO4~1\apps\grass\grass78\lib;C:\OSGEO4~1\apps\grass\grass78\bin;C:\OSGEO4~1\apps\qt5\bin;C:\OSGEO4~1\apps\Python39\Scripts;C:\OSGEO4~1\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\system32\WBem;;C:/OSGEO4~1/apps\saga;C:/OSGEO4~1/apps\saga\modules
C:\OSGeo4W64\bin>call saga_cmd io_gdal 0 -TRANSFORM 1 -RESAMPLING 3 -GRIDS "D:/Temp/processing_LqImgX/12045f9c2ae6477b98d7d621150df270/Rasterized.sgrd" -FILES "D:/Temp/processing_LqImgX/861dd24ba97f4c6984d21cad5aa91e9b/OUTPUT.tif"
____________________________
##### ## ##### ##
### ### ## ###
### # ## ## #### # ##
### ##### ## # #####
##### # ## ##### # ##
____________________________
SAGA Version: 7.8.2 (64 bit)
88 loaded tool libraries (775 tools):
- tin_viewer
- tin_tools
- table_tools
- table_calculus
- ta_slope_stability
- ta_profiles
- ta_preprocessor
- ta_morphometry
- ta_lighting
- ta_hydrology
- ta_compound
- ta_cliffmetrics
- ta_channels
- statistics_regression
- statistics_points
- statistics_kriging
- statistics_grid
- sim_rivflow
- sim_qm_of_esp
- sim_landscape_evolution
- sim_ihacres
- sim_hydrology
- sim_geomorphology
- sim_fire_spreading
- sim_erosion
- sim_ecosystems_hugget
- sim_cellular_automata
- sim_air_flow
- shapes_transect
- shapes_tools
- shapes_polygons
- shapes_points
- shapes_lines
- shapes_grid
- pointcloud_viewer
- pointcloud_tools
- pj_proj4
- pj_geotrans
- pj_georeference
- io_webservices
- io_virtual
- io_table
- io_shapes_dxf
- io_shapes
- io_pdal
- io_grid_image
- io_grid
- io_gps
- io_esri_e00
- imagery_tools
- imagery_svm
- imagery_segmentation
- imagery_photogrammetry
- imagery_maxent
- imagery_isocluster
- imagery_classification
- grids_tools
- grid_visualisation
- grid_tools
- grid_spline
- grid_gridding
- grid_filter
- grid_calculus_bsl
- grid_calculus
- grid_analysis
- garden_webservices
- garden_learn_to_program
- garden_games
- garden_fractals
- garden_3d_viewer
- docs_pdf
- docs_html
- dev_tools
- db_pgsql
- db_odbc
- contrib_perego
- climate_tools
- tta_tools *
- toolchains *
- terrain_analysis *
- imagery *
- group_files *
- grid_filter *
- gc_tools *
- climate_tools *
- Shapes Tools *
- Polygon Tools *
- Grid Tools *
*) tool chain libraries
Error: select a library
type -h or --help for further information
Looking at SAGA folder, io_gdal is not available:
It is on SAGA default windows package:
Checking saga_gui, here is the OSGeo4W version:
and SAGA default windows version:
So it seems that io_gdal is missing and that's the tool librarie used to import raster data to SAGA, which I suspect that will make all raster algorithms to fail inside QGIS.
saga 2 doesn't support current versions of proj.
@jef-n so I guess is ok make PR with the description files for SAGA 7.*, correct?
@jef-n so I guess is ok make PR with the description files for SAGA 7.*, correct?
@jef-n sorry it seems you already did it(?).
@jef-n sorry it seems you already did it(?).
No, the saga team implemented a tool that produces the files - and the packaging now calls it on install to produce the files.
@jef-n
I've tested several SAGA raster algorithms and all fail, pointing to the missing io_gdal.
I've tested several SAGA raster algorithms and all fail, pointing to the missing io_gdal.
@PedroVenancio @jef-n this issue kind of remembers me of https://github.com/qgis/QGIS/issues/32953#issuecomment-555649562 but not sure if it is related at all.
@jef-n
Testing the last version available (QGIS code revision | b25fe874a), I get several issues:
2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_FILE registered for alg createrandomraster
2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_RASTER registered for alg preplandscape
2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_RASTER registered for alg landintersect
2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_FILE registered for alg landscapestat
2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_FILE registered for alg patchstat
2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_FILE registered for alg countrastercell
2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_FILE registered for alg zonalstat
2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_RASTER registered for alg zonalstat
2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_FILE registered for alg poloverlayraster
2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_FILE registered for alg queryraster
2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_RASTER registered for alg labellandscape
2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_FILE registered for alg nanalysis
2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_RASTER registered for alg incdecpatch
2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_RASTER registered for alg patchedges
2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_RASTER registered for alg minmaxpatch
2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_RASTER registered for alg closeholes
2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_RASTER registered for alg cleanlandscape
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: qgis:distancematrix
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: qgis:listuniquevalues
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: qgis:basicstatisticsforfields
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: qgis:randomselection
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: qgis:randomselectionwithinsubsets
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: qgis:randompointsinlayerbounds
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: qgis:randompointsinsidepolygons
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: qgis:regularpoints
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: qgis:eliminateselectedpolygons
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: qgis:checkvalidity
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: qgis:exportaddgeometrycolumns
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: qgis:delaunaytriangulation
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: qgis:voronoipolygons
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: qgis:linestopolygons
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:warpreproject
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:extractprojection
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:assignprojection
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:rasterize
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:polygonize
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:translate
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:rgbtopct
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:pcttorgb
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:contour
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:cliprasterbyextent
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:cliprasterbymasklayer
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:sieve
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:nearblack
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:fillnodata
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:proximity
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:griddatametrics
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:gridaverage
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:gridinversedistance
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:gridnearestneighbor
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:aspect
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:hillshade
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:roughness
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:slope
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:tpitopographicpositionindex
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:triterrainruggednessindex
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:buildvirtualraster
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:merge
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:gdalinfo
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:overviews
2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:tileindex
No models, scripts, GDAL, GRASS, SAGA providers ...
Fixed in jef-n/OSGeo4W@647edd972477f2
No, the saga team implemented a tool that produces the files - and the packaging now calls it on install to produce the files.
@jef-n I knew about those description files, but didn't know that are now pushed during packaging. So what is supposed to do if we find something wrong in one of these description files (which I already did)? We must fix them upstream? If yes, what about if the SAGA project do not what to merge/accept the fix?
The descriptions are not pushed to git - they are created on install with saga_cmd dev_tools
. I didn't follow closely how SAGA support in processing is maintained, I thought there was an (external) plugin for newer saga versions and in turn our internal saga support would eventually go away. QGIS support on the SAGA side was news to me. So this is just a temporary kludge for OSGeo4W. Not sure if we want to update the descriptions in QGIS (and remove SAGA 2 support?) or replace internal saga support with an external plugin. If SAGA wouldn't accept fixes, we could still patch the OSGeo4W package - but why wouldn't they. Anyway, either way we'd need to update a OSGeo4W package anyway - be it qgis or saga.
@jef-n
I thought there was an (external) plugin for newer saga versions
There is, but honestly there's proven to be very little community interest in these.
and in turn our internal saga support would eventually go away
That's certainly what @alexbruy and I want to see. Have you still got work to do regarding SAGA? If you can say that getting this in shape will take up hours of your time, then I think that's the final catalyst for us to hit the kill switch...
That's certainly what @alexbruy and I want to see. Have you still got work to do regarding SAGA? If you can say that getting this in shape will take up hours of your time, then I think that's the final catalyst for us to hit the kill switch...
Well, I gave that little thought and originally wanted to just rebuild SAGA 2. That didn't work well as it doesn't support current PROJ. Then I built SAGA 7 - because I thought "BETA_SUPPORT_VERSION" could mean that we have some support for SAGA 7. But apparently that needs new descriptions files, so I digged up saga_cmd dev_tools
and a way to update the description files on install. Not sure if we're done now. The warning about SAGA 7 could trigger community interest…
That's certainly what @alexbruy and I want to see. Have you still got work to do regarding SAGA? If you can say that getting this in shape will take up hours of your time, then I think that's the final catalyst for us to hit the kill switch...
Well, I gave that little thought and originally wanted to just rebuild SAGA 2. That didn't work well as it doesn't support current PROJ. Then I built SAGA 7 - because I thought "BETA_SUPPORT_VERSION" could mean that we have some support for SAGA 7. But apparently that needs new descriptions files, so I digged up
saga_cmd dev_tools
and a way to update the description files on install. Not sure if we're done now. The warning about SAGA 7 could trigger for community interest…
I also think this is the best solution at this point, more or less like Orfeo Toolbox. In some way, this put some responsibility in SAGA team, that show some desire in give some support to QGIS integration, making the description files available. One problem I see is in keeping the compatibility between OSGeo4W and Linux, where the versions supported are too variable. Putting SAGA only as a plugin would solve that.
Hi @PedroVenancio, @jef-n, @nyalldawson , I don't know if this helps but recently when I download the last SAGA GIS for Windows I discovered that the SAGA Team have a zip folder inside of SAGA binaries called SAGA4qgis.zip with instructions to deactivate SAGA GIS default version from QGIS and activate this version changing the qgis python configuration.
This attach have the instruction they use to activate the new versions of SAGA in QGIS. saga4qgis.zip
They have a readme saying:
Adjust the interface in:
Replace the SAGA binaries in:
C:\Program Files\QGIS X.Y\apps\saga(-ltr)
C:\Program Files\QGIS X.Y\apps\qgis(-ltr)\python\plugins\processing\algs\saga(-ltr)
Adjust version in the file:
SagaAlgorithmProvider.py
REQUIRED_VERSION = '7.7.'_
Then you need to copy the description files and the SagaNameDecorator.py from the saga4gis.zip to the right installation folder
This is for Windows example I don't know I this can be replicated for Linux. I tested and this hack works, QGIS use the new descriptions files and recognize most of the algorithms (don't test all) this hack but isn´t very friendly for a QGIS regular user.
Cheers
Hi @PedroVenancio, @jef-n, @nyalldawson , I don't know if this helps but recently when I download the last SAGA GIS for Windows I discovered that the SAGA Team have a zip folder inside of SAGA binaries called SAGA4qgis.zip with instructions to deactivate SAGA GIS default version from QGIS and activate this version changing the qgis python configuration.
um, see @PedroVenancio's initial comment.
Hi @jef-n , I spoke with Pedro and realized that he already mention this. Sorry for the noise.
@jef-n
Testing the last version available (QGIS code revision | b25fe87), I get several issues:
2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_FILE registered for alg createrandomraster 2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_RASTER registered for alg preplandscape 2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_RASTER registered for alg landintersect 2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_FILE registered for alg landscapestat 2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_FILE registered for alg patchstat 2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_FILE registered for alg countrastercell 2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_FILE registered for alg zonalstat 2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_RASTER registered for alg zonalstat 2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_FILE registered for alg poloverlayraster 2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_FILE registered for alg queryraster 2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_RASTER registered for alg labellandscape 2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_FILE registered for alg nanalysis 2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_RASTER registered for alg incdecpatch 2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_RASTER registered for alg patchedges 2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_RASTER registered for alg minmaxpatch 2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_RASTER registered for alg closeholes 2021-02-19T18:54:30 WARNING Duplicate output OUTPUT_RASTER registered for alg cleanlandscape 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: qgis:distancematrix 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: qgis:listuniquevalues 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: qgis:basicstatisticsforfields 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: qgis:randomselection 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: qgis:randomselectionwithinsubsets 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: qgis:randompointsinlayerbounds 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: qgis:randompointsinsidepolygons 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: qgis:regularpoints 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: qgis:eliminateselectedpolygons 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: qgis:checkvalidity 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: qgis:exportaddgeometrycolumns 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: qgis:delaunaytriangulation 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: qgis:voronoipolygons 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: qgis:linestopolygons 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:warpreproject 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:extractprojection 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:assignprojection 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:rasterize 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:polygonize 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:translate 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:rgbtopct 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:pcttorgb 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:contour 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:cliprasterbyextent 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:cliprasterbymasklayer 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:sieve 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:nearblack 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:fillnodata 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:proximity 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:griddatametrics 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:gridaverage 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:gridinversedistance 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:gridnearestneighbor 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:aspect 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:hillshade 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:roughness 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:slope 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:tpitopographicpositionindex 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:triterrainruggednessindex 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:buildvirtualraster 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:merge 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:gdalinfo 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:overviews 2021-02-19T18:54:47 WARNING Invalid algorithm ID for menu: gdal:tileindex
@jef-n I'm still seeing this error in qgis-rel-dev 3.18.0-6-2d3150d0-1 (code revision | 2d3150d0). Seems fixed in all other versions, including 3.18.0-5 (code revision | bdef9fb3).
Perfect @jef-n , fixed on QGIS code revision | 981c306f
@jef-n
I was testing SAGA 7.8.1-1 now available from new OSGeo4W installer but it seems incomplete and unusable even with saga_cmd.exe or saga_gui.exe
Besides that, I wonder why not use SAGA 7.8.2 as it is a bug fix release: https://sourceforge.net/p/saga-gis/wiki/Changelog%207.8.2/
Finally, the description files available at
..\OSGeo4W64\apps\qgis-dev\python\plugins\processing\algs\saga
are still the older ones, for SAGA 2.3.2. Now SAGA team is shipping the description files inside the binaries package for Windows https://sourceforge.net/projects/saga-gis/files/SAGA%20-%207/SAGA%20-%207.8.2/saga-7.8.2_x64.zip/download (saga4qgis.zip). Could this also be updated in parallel with SAGA version on OSGeo4W?