qgis / QGIS

QGIS is a free, open source, cross platform (lin/win/mac) geographical information system (GIS)
https://qgis.org
GNU General Public License v2.0
10.29k stars 2.96k forks source link

qgis_process: also show progress bar when using JSON for input or output #55692

Open florisvdh opened 8 months ago

florisvdh commented 8 months ago

Feature description

Simple use of qgis_process invokes a progress bar being shown like:

0...10...20...30...40...50...60...70...80...90...

This serves users especially for processes that take a while.

However the progress bar is not show when either using the --json flag (for output) or passing a JSON string through standard input.

It would be great if the feature could also be present in the JSON input/output cases.

Comparison of current behaviours ```bash $ qgis_process run native:filedownloader --URL=https://zenodo.org/records/10418745/files/r-spatial/qgisprocess-v0.2.0.zip --OUTPUT=temp.zip ---------------- Inputs ---------------- OUTPUT: temp.zip URL: https://zenodo.org/records/10418745/files/r-spatial/qgisprocess-v0.2.0.zip 0...10...20...30...40...50...60...1.22 MB of 1.77 MB downloaded 70...80...90...100 - done. Successfully downloaded https://zenodo.org/records/10418745/files/r-spatial/qgisprocess-v0.2.0.zip ---------------- Results ---------------- OUTPUT: temp.zip $ $ qgis_process --json run native:filedownloader --URL=https://zenodo.org/records/10418745/files/r-spatial/qgisprocess-v0.2.0.zip --OUTPUT=temp.zip { "algorithm_details": { "can_cancel": true, "deprecated": false, "group": "File tools", "has_known_issues": false, "help_url": null, "id": "native:filedownloader", "name": "Download file", "requires_matching_crs": false, "short_description": null, "tags": [ "file", "downloader", "internet", "url", "fetch", "get", "https" ] }, "gdal_version": "3.6.4", "geos_version": "3.11.1-CAPI-1.17.1", "inputs": { "OUTPUT": "temp.zip", "URL": "https://zenodo.org/records/10418745/files/r-spatial/qgisprocess-v0.2.0.zip" }, "log": { "info": [ "Successfully downloaded https://zenodo.org/records/10418745/files/r-spatial/qgisprocess-v0.2.0.zip" ] }, "proj_version": "Rel. 9.1.1, December 1st, 2022", "provider_details": { "can_be_activated": true, "default_raster_file_extension": "nc", "default_vector_file_extension": "xtf", "is_active": true, "long_name": "QGIS (native c++)", "name": "QGIS (native c++)", "supported_output_raster_extensions": [ "tif", "gen", "bag", "bmp", "bt", "byn", "bil", "ers", "fits", "gpkg", "grd", "grd", "gtx", "img", "mpr", "lbl", "kro", "ter", "mbtiles", "hdr", "mrf", "ntf", "gsb", "grd", "pix", "map", "pdf", "xml", "pgm", "rsw", "grd", "rst", "sdat", "rgb", "ter", "vrt", "nc" ], "supported_output_table_extensions": [ "gpkg", "shp", "000", "csv", "dgn", "dxf", "fgb", "gdb", "geojson", "geojsonl", "geojsons", "gml", "gpx", "gxt", "ili", "itf", "json", "kml", "ods", "sql", "sqlite", "tab", "txt", "xlsx", "xml", "xtf" ], "supported_output_vector_extensions": [ "gpkg", "shp", "000", "csv", "dgn", "dxf", "fgb", "gdb", "geojson", "geojsonl", "geojsons", "gml", "gpx", "gxt", "ili", "itf", "json", "kml", "ods", "sql", "sqlite", "tab", "txt", "xlsx", "xml", "xtf" ], "supports_non_file_based_output": true, "version": null }, "python_version": "3.10.12", "qgis_code_revision": "133927424d9", "qgis_version": "3.34.1-Prizren", "qt_version": "5.15.3", "results": { "OUTPUT": "temp.zip" } }$ $ $ myJSON=$(

Additional context

This is a feature request forwarded from R package {qgisprocess}: https://github.com/r-spatial/qgisprocess/issues/185

florisvdh commented 8 months ago

Just encountered that more is missing in the JSON case, e.g. the warning emitted when running native:zonalstatistics.

With warning:

$ wget -q https://github.com/inbo/coding-club/raw/main/data/20231214/20231214_land_use_nara_2016_100m.tif
$ wget -q https://github.com/inbo/coding-club/raw/main/data/20231214/20231214_natura2000_protected_areas.gpkg
$ 
$ qgis_process run native:zonalstatistics --INPUT_RASTER=20231214_land_use_nara_2016_100m.tif --INPUT_VECTOR=20231214_natura2000_protected_areas.gpkg --COLUMN_PREFIX=landuse_ --STATISTICS="8,9"

****************
Warning: this algorithm is deprecated and may be removed in a future QGIS version!
****************

----------------
Inputs
----------------

COLUMN_PREFIX:  landuse_
INPUT_RASTER:   20231214_land_use_nara_2016_100m.tif
INPUT_VECTOR:   20231214_natura2000_protected_areas.gpkg
STATISTICS: 8,9

0...10...20...30...40...50...60...70...80...90...100 - done.

----------------
Results
----------------

INPUT_VECTOR:   20231214_natura2000_protected_areas_cd17e388_b092_49ce_98d5_5e33dfca54dd

No warning:

qgis_process --json returns a JSON string (click to expand) ```bash $ wget -q https://github.com/inbo/coding-club/raw/main/data/20231214/20231214_natura2000_protected_areas.gpkg -O 20231214_natura2000_protected_areas.gpkg $ $ qgis_process --json run native:zonalstatistics --INPUT_RASTER=20231214_land_use_nara_2016_100m.tif --INPUT_VECTOR=20231214_natura2000_protected_areas.gpkg --COLUMN_PREFIX=landuse_ --STATISTICS="8,9" { "algorithm_details": { "can_cancel": true, "deprecated": true, "group": "Raster analysis", "has_known_issues": false, "help_url": null, "id": "native:zonalstatistics", "name": "Zonal statistics (in place)", "requires_matching_crs": false, "short_description": "Calculates statistics for a raster layer's values for each feature of an overlapping polygon vector layer.", "tags": [ "stats", "statistics", "zones", "layer", "sum", "maximum", "minimum", "mean", "count", "standard", "deviation", "median", "range", "majority", "minority", "variety", "variance", "summary", "raster" ] }, "gdal_version": "3.6.4", "geos_version": "3.11.1-CAPI-1.17.1", "inputs": { "COLUMN_PREFIX": "landuse_", "INPUT_RASTER": "20231214_land_use_nara_2016_100m.tif", "INPUT_VECTOR": "20231214_natura2000_protected_areas.gpkg", "STATISTICS": "8,9" }, "log": {}, "proj_version": "Rel. 9.1.1, December 1st, 2022", "provider_details": { "can_be_activated": true, "default_raster_file_extension": "nc", "default_vector_file_extension": "xtf", "is_active": true, "long_name": "QGIS (native c++)", "name": "QGIS (native c++)", "supported_output_raster_extensions": [ "tif", "gen", "bag", "bmp", "bt", "byn", "bil", "ers", "fits", "gpkg", "grd", "grd", "gtx", "img", "mpr", "lbl", "kro", "ter", "mbtiles", "hdr", "mrf", "ntf", "gsb", "grd", "pix", "map", "pdf", "xml", "pgm", "rsw", "grd", "rst", "sdat", "rgb", "ter", "vrt", "nc" ], "supported_output_table_extensions": [ "gpkg", "shp", "000", "csv", "dgn", "dxf", "fgb", "gdb", "geojson", "geojsonl", "geojsons", "gml", "gpx", "gxt", "ili", "itf", "json", "kml", "ods", "sql", "sqlite", "tab", "txt", "xlsx", "xml", "xtf" ], "supported_output_vector_extensions": [ "gpkg", "shp", "000", "csv", "dgn", "dxf", "fgb", "gdb", "geojson", "geojsonl", "geojsons", "gml", "gpx", "gxt", "ili", "itf", "json", "kml", "ods", "sql", "sqlite", "tab", "txt", "xlsx", "xml", "xtf" ], "supports_non_file_based_output": true, "version": null }, "python_version": "3.10.12", "qgis_code_revision": "133927424d9", "qgis_version": "3.34.1-Prizren", "qt_version": "5.15.3", "results": { "INPUT_VECTOR": "20231214_natura2000_protected_areas_7c74b47d_0622_4969_a74d_f48b74d81812" } } ```
florisvdh commented 8 months ago

I see caveats when adding these requested additions. Currently --json just provides a clean JSON string in stdout. If other strings are going to be added before that, it's getting harder to process stdout, although it can still be parsed...

Would there be other alternatives? Feel free to close if you feel there's no better implementation than the one in place.