search

r-spatial / qgisprocess

R package to use QGIS processing algorithms
https://r-spatial.github.io/qgisprocess/
GNU General Public License v3.0
196 stars 20 forks source link

Algorithm 'native:printlayouttopdf' segfaults on MacOS #68

Open paleolimbot opened 2 years ago

paleolimbot commented 2 years ago

Thanks to the r-spatial panel question at FOSS4G!

library(qgisprocess)
#> Using 'qgis_process' at '/Applications/QGIS-LTR.app/Contents/MacOS/bin/qgis_process'.
#> QGIS version: 3.16.10-Hannover
#> Metadata of 971 algorithms successfully cached.
#> Run `qgis_configure()` for details.
curl::curl_download(
  "https://gist.githubusercontent.com/paleolimbot/f2fa6f409c48c3d18fe5462fb29f8996/raw/3c1e14b7834c2175acbb6cb1b6f646906f9f89ed/test.qgs",
  "test.qgs"
)

qgis_run_algorithm(
  "native:printlayouttopdf",
  LAYOUT = "Layout 1",
  OUTPUT = "output.pdf",
  PROJECT_PATH = "test.qgs"
)
#> Argument `LAYERS` is unspecified (using QGIS default value).
#> Argument `DPI` is unspecified (using QGIS default value).
#> Argument `FORCE_VECTOR` is unspecified (using QGIS default value).
#> Argument `GEOREFERENCE` is unspecified (using QGIS default value).
#> Argument `INCLUDE_METADATA` is unspecified (using QGIS default value).
#> Argument `DISABLE_TILED` is unspecified (using QGIS default value).
#> Argument `SIMPLIFY` is unspecified (using QGIS default value).
#> Using `TEXT_FORMAT = "Always Export Text as Paths (Recommended)"`
#> Argument `SEPARATE_LAYERS` is unspecified (using QGIS default value).
#> Running /Applications/QGIS-LTR.app/Contents/MacOS/bin/qgis_process run \
#>   'native:printlayouttopdf' '--LAYOUT=Layout 1' '--TEXT_FORMAT=0' \
#>   '--OUTPUT=output.pdf' '--PROJECT_PATH=test.qgs'
#> /Applications/QGIS-LTR.app/Contents/MacOS/bin/qgis_process: line 6: 40584 Segmentation fault: 11  $THISDIR/../qgis_process.app/Contents/MacOS/qgis_process "$@"
#> Error in processx::run(path, args, ...): System command 'qgis_process' failed, exit status: 139, stderr:
#> E> /Applications/QGIS-LTR.app/Contents/MacOS/bin/qgis_process: line 6: 40584 Segmentation fault: 11  $THISDIR/../qgis_process.app/Contents/MacOS/qgis_process "$@"

Created on 2021-09-30 by the reprex package (v2.0.1)

gavg712 commented 2 years ago

Any progress on this issue? I've been making some corrections that might be useful, do you want to test them before doing the PR? Please check my gavg712/qgisprocess/tree/project_path branch

florisvdh commented 1 year ago

Since a solution was applied to #75, this also seems to have solved this issue but only when no JSON input is used, as above. But for QGIS >= 3.23 JSON input is used as the default, and that gives an error (maybe related cause, maybe another problem).

oldopt <- options(qgisprocess.use_json_input = FALSE)

library(qgisprocess)
#> Using 'qgis_process' in the system PATH.
#> QGIS version: 3.26.1-Buenos Aires
#> Configuration loaded from '~/.cache/R-qgisprocess/cache-0.0.0.9000.rds'
#> Run `qgis_configure(use_cached_data = TRUE)` to reload cache and get more details.
#> >>> If you need another installed QGIS version, run `qgis_configure()`;
#>     see its documentation if you need to preset the path of qgis_process.
#> - Using JSON for output serialization.
curl::curl_download(
  "https://gist.githubusercontent.com/paleolimbot/f2fa6f409c48c3d18fe5462fb29f8996/raw/3c1e14b7834c2175acbb6cb1b6f646906f9f89ed/test.qgs",
  "test.qgs"
)

qgis_run_algorithm(
  "native:printlayouttopdf",
  LAYOUT = "Layout 1",
  OUTPUT = "output.pdf",
  PROJECT_PATH = "test.qgs"
)
#> Argument `LAYERS` is unspecified (using QGIS default value).
#> Argument `DPI` is unspecified (using QGIS default value).
#> Argument `FORCE_VECTOR` is unspecified (using QGIS default value).
#> Argument `GEOREFERENCE` is unspecified (using QGIS default value).
#> Argument `INCLUDE_METADATA` is unspecified (using QGIS default value).
#> Argument `DISABLE_TILED` is unspecified (using QGIS default value).
#> Argument `SIMPLIFY` is unspecified (using QGIS default value).
#> Using `TEXT_FORMAT = "Always Export Text as Paths (Recommended)"`
#> Argument `SEPARATE_LAYERS` is unspecified (using QGIS default value).
#> Running qgis_process --json run 'native:printlayouttopdf' '--LAYOUT=Layout 1' \
#>   '--TEXT_FORMAT=0' '--OUTPUT=output.pdf' '--PROJECT_PATH=test.qgs'
#> qt5ct: using qt5ct plugin
#> Problem with OTB installation: OTB folder is not set.
#> <Result of `qgis_run_algorithm("native:printlayouttopdf", ...)`>
#> List of 1
#>  $ OUTPUT: 'qgis_outputFile' chr "output.pdf"

file.exists("output.pdf")
#> [1] TRUE

options(oldopt)

qgis_run_algorithm(
  "native:printlayouttopdf",
  LAYOUT = "Layout 1",
  OUTPUT = "output.pdf",
  PROJECT_PATH = "test.qgs"
)
#> Argument `LAYERS` is unspecified (using QGIS default value).
#> Argument `DPI` is unspecified (using QGIS default value).
#> Argument `FORCE_VECTOR` is unspecified (using QGIS default value).
#> Argument `GEOREFERENCE` is unspecified (using QGIS default value).
#> Argument `INCLUDE_METADATA` is unspecified (using QGIS default value).
#> Argument `DISABLE_TILED` is unspecified (using QGIS default value).
#> Argument `SIMPLIFY` is unspecified (using QGIS default value).
#> Using `TEXT_FORMAT = "Always Export Text as Paths (Recommended)"`
#> Argument `SEPARATE_LAYERS` is unspecified (using QGIS default value).
#> JSON input ----
#> {
#>   "inputs": {
#>     "LAYOUT": "Layout 1",
#>     "TEXT_FORMAT": 0,
#>     "OUTPUT": "output.pdf",
#>     "PROJECT_PATH": "test.qgs"
#>   }
#> }
#> 
#> Running qgis_process --json run 'native:printlayouttopdf' -
#> qt5ct: using qt5ct plugin
#> Problem with OTB installation: OTB folder is not set.
#> Cannot find layout with name "Layout 1"
#> Cannot find layout with name "Layout 1"
#> Error in "processx::run(path, args, ...)": ! System command 'qgis_process' failed

Created on 2022-08-24 by the reprex package (v2.0.1)

Session info ``` r sessioninfo::session_info() #> ─ Session info ─────────────────────────────────────────────────────────────── #> setting value #> version R version 4.2.1 (2022-06-23) #> os Linux Mint 20 #> system x86_64, linux-gnu #> ui X11 #> language nl_BE:nl #> collate nl_BE.UTF-8 #> ctype nl_BE.UTF-8 #> tz Europe/Brussels #> date 2022-08-24 #> pandoc 2.18 @ /usr/lib/rstudio/bin/quarto/bin/tools/ (via rmarkdown) #> #> ─ Packages ─────────────────────────────────────────────────────────────────── #> package * version date (UTC) lib source #> cli 3.3.0 2022-04-25 [1] CRAN (R 4.2.0) #> curl 4.3.2 2021-06-23 [1] CRAN (R 4.2.0) #> digest 0.6.29 2021-12-01 [1] CRAN (R 4.2.0) #> evaluate 0.16 2022-08-09 [1] CRAN (R 4.2.1) #> fansi 1.0.3 2022-03-24 [1] CRAN (R 4.2.0) #> fastmap 1.1.0 2021-01-25 [1] CRAN (R 4.2.0) #> fs 1.5.2 2021-12-08 [1] CRAN (R 4.2.0) #> glue 1.6.2 2022-02-24 [1] CRAN (R 4.2.0) #> highr 0.9 2021-04-16 [1] CRAN (R 4.2.0) #> htmltools 0.5.3 2022-07-18 [1] RSPM (R 4.2.1) #> jsonlite 1.8.0 2022-02-22 [1] CRAN (R 4.2.0) #> knitr 1.39 2022-04-26 [1] CRAN (R 4.2.0) #> lifecycle 1.0.1 2021-09-24 [1] CRAN (R 4.2.0) #> magrittr 2.0.3 2022-03-30 [1] CRAN (R 4.2.0) #> pillar 1.8.0 2022-07-18 [1] RSPM (R 4.2.1) #> pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.2.0) #> processx 3.7.0 2022-07-07 [1] RSPM (R 4.2.1) #> ps 1.7.1 2022-06-18 [1] RSPM (R 4.2.1) #> qgisprocess * 0.0.0.9000 2022-08-19 [1] local #> R6 2.5.1 2021-08-19 [1] CRAN (R 4.2.0) #> rappdirs 0.3.3 2021-01-31 [1] CRAN (R 4.2.0) #> reprex 2.0.1 2021-08-05 [1] CRAN (R 4.2.0) #> rlang 1.0.4 2022-07-12 [1] RSPM (R 4.2.1) #> rmarkdown 2.14 2022-04-25 [1] CRAN (R 4.2.0) #> rstudioapi 0.13 2020-11-12 [1] CRAN (R 4.2.0) #> sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.2.0) #> stringi 1.7.8 2022-07-11 [1] RSPM (R 4.2.1) #> stringr 1.4.0 2019-02-10 [1] CRAN (R 4.2.0) #> tibble 3.1.8 2022-07-22 [1] RSPM (R 4.2.1) #> utf8 1.2.2 2021-07-24 [1] CRAN (R 4.2.0) #> vctrs 0.4.1 2022-04-13 [1] CRAN (R 4.2.0) #> withr 2.5.0 2022-03-03 [1] CRAN (R 4.2.0) #> xfun 0.32 2022-08-10 [1] CRAN (R 4.2.1) #> yaml 2.3.5 2022-02-21 [1] CRAN (R 4.2.0) #> #> [1] /home/floris/lib/R/library #> [2] /usr/local/lib/R/site-library #> [3] /usr/lib/R/site-library #> [4] /usr/lib/R/library #> #> ────────────────────────────────────────────────────────────────────────────── ```

To be continued!

florisvdh commented 1 year ago

In the JSON string, the project_path element has to be passed differently (outside inputs and lowercase). That's the reason of the latest error. This also needs a fix at QGIS, since this element is not exported either from the processing dialog (GUI) when copying the JSON string, while it is needed for this type of algorithms. Will be reported.

library(qgisprocess)
#> Using 'qgis_process' in the system PATH.
#> QGIS version: 3.26.1-Buenos Aires
#> Configuration loaded from '~/.cache/R-qgisprocess/cache-0.0.0.9000.rds'
#> Run `qgis_configure(use_cached_data = TRUE)` to reload cache and get more details.
#> >>> If you need another installed QGIS version, run `qgis_configure()`;
#>     see its documentation if you need to preset the path of qgis_process.
#> - Using JSON for input serialization.
#> - Using JSON for output serialization.

curl::curl_download(
  "https://gist.githubusercontent.com/paleolimbot/f2fa6f409c48c3d18fe5462fb29f8996/raw/3c1e14b7834c2175acbb6cb1b6f646906f9f89ed/test.qgs",
  "test.qgs"
)

json_input <- '{
  "inputs": {
    "LAYOUT": "Layout 1",
    "TEXT_FORMAT": 0,
    "OUTPUT": "output.pdf"
  },
  "project_path": "test.qgs"
}'
qgis_run_algorithm("native:printlayouttopdf", .raw_json_input = json_input)
#> JSON input ----
#> {
#>   "inputs": {
#>     "LAYOUT": "Layout 1",
#>     "TEXT_FORMAT": 0,
#>     "OUTPUT": "output.pdf"
#>   },
#>   "project_path": "test.qgs"
#> }
#> 
#> Running qgis_process --json run 'native:printlayouttopdf' -
#> qt5ct: using qt5ct plugin
#> Problem with OTB installation: OTB folder is not set.
#> <Result of `qgis_run_algorithm("native:printlayouttopdf", ...)`>
#> List of 1
#>  $ OUTPUT: 'qgis_outputFile' chr "output.pdf"

file.exists("output.pdf")
#> [1] TRUE

Created on 2022-08-24 by the reprex package (v2.0.1)

Session info ``` r sessioninfo::session_info() #> ─ Session info ─────────────────────────────────────────────────────────────── #> setting value #> version R version 4.2.1 (2022-06-23) #> os Linux Mint 20 #> system x86_64, linux-gnu #> ui X11 #> language nl_BE:nl #> collate nl_BE.UTF-8 #> ctype nl_BE.UTF-8 #> tz Europe/Brussels #> date 2022-08-24 #> pandoc 2.18 @ /usr/lib/rstudio/bin/quarto/bin/tools/ (via rmarkdown) #> #> ─ Packages ─────────────────────────────────────────────────────────────────── #> package * version date (UTC) lib source #> cli 3.3.0 2022-04-25 [1] CRAN (R 4.2.0) #> curl 4.3.2 2021-06-23 [1] CRAN (R 4.2.0) #> digest 0.6.29 2021-12-01 [1] CRAN (R 4.2.0) #> evaluate 0.16 2022-08-09 [1] CRAN (R 4.2.1) #> fansi 1.0.3 2022-03-24 [1] CRAN (R 4.2.0) #> fastmap 1.1.0 2021-01-25 [1] CRAN (R 4.2.0) #> fs 1.5.2 2021-12-08 [1] CRAN (R 4.2.0) #> glue 1.6.2 2022-02-24 [1] CRAN (R 4.2.0) #> highr 0.9 2021-04-16 [1] CRAN (R 4.2.0) #> htmltools 0.5.3 2022-07-18 [1] RSPM (R 4.2.1) #> jsonlite 1.8.0 2022-02-22 [1] CRAN (R 4.2.0) #> knitr 1.39 2022-04-26 [1] CRAN (R 4.2.0) #> lifecycle 1.0.1 2021-09-24 [1] CRAN (R 4.2.0) #> magrittr 2.0.3 2022-03-30 [1] CRAN (R 4.2.0) #> pillar 1.8.0 2022-07-18 [1] RSPM (R 4.2.1) #> pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.2.0) #> processx 3.7.0 2022-07-07 [1] RSPM (R 4.2.1) #> ps 1.7.1 2022-06-18 [1] RSPM (R 4.2.1) #> qgisprocess * 0.0.0.9000 2022-08-24 [1] local #> R6 2.5.1 2021-08-19 [1] CRAN (R 4.2.0) #> rappdirs 0.3.3 2021-01-31 [1] CRAN (R 4.2.0) #> reprex 2.0.1 2021-08-05 [1] CRAN (R 4.2.0) #> rlang 1.0.4 2022-07-12 [1] RSPM (R 4.2.1) #> rmarkdown 2.14 2022-04-25 [1] CRAN (R 4.2.0) #> rstudioapi 0.13 2020-11-12 [1] CRAN (R 4.2.0) #> sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.2.0) #> stringi 1.7.8 2022-07-11 [1] RSPM (R 4.2.1) #> stringr 1.4.0 2019-02-10 [1] CRAN (R 4.2.0) #> tibble 3.1.8 2022-07-22 [1] RSPM (R 4.2.1) #> utf8 1.2.2 2021-07-24 [1] CRAN (R 4.2.0) #> vctrs 0.4.1 2022-04-13 [1] CRAN (R 4.2.0) #> withr 2.5.0 2022-03-03 [1] CRAN (R 4.2.0) #> xfun 0.32 2022-08-10 [1] CRAN (R 4.2.1) #> yaml 2.3.5 2022-02-21 [1] CRAN (R 4.2.0) #> #> [1] /home/floris/lib/R/library #> [2] /usr/local/lib/R/site-library #> [3] /usr/lib/R/site-library #> [4] /usr/lib/R/library #> #> ────────────────────────────────────────────────────────────────────────────── ```
florisvdh commented 1 year ago

The above problem has been fixed for non-MacOS systems by #111.

However, the problem on MacOS is still present as shown in https://github.com/paleolimbot/qgisprocess/pull/111#issuecomment-1234278953.

Awaiting further localisation of the problem in MacOS (QGIS or R?); meanwhile refining this issue's title.

florisvdh commented 1 year ago

My colleague @peterdesmet uses macOS and found that the qgis_process command does this job fine, both with and without JSON input!

qgis_process using native:printlayouttopdf (no JSON input) (click to expand) ```bash $ wget -O test.qgs https://gist.githubusercontent.com/paleolimbot/f2fa6f409c48c3d18fe5462fb29f8996/raw/3c1e14b7834c2175acbb6cb1b6f646906f9f89ed/test.qgs --2022-09-05 13:23:50-- https://gist.githubusercontent.com/paleolimbot/f2fa6f409c48c3d18fe5462fb29f8996/raw/3c1e14b7834c2175acbb6cb1b6f646906f9f89ed/test.qgs Resolving [gist.githubusercontent.com](http://gist.githubusercontent.com/) ([gist.githubusercontent.com](http://gist.githubusercontent.com/))... 185.199.108.133, 185.199.111.133, 185.199.109.133, ... Connecting to [gist.githubusercontent.com](http://gist.githubusercontent.com/) ([gist.githubusercontent.com](http://gist.githubusercontent.com/))|185.199.108.133|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 27069 (26K) [text/plain] Saving to: ‘test.qgs’ test.qgs 100%[========================>] 26.43K --.-KB/s in 0.004s 2022-09-05 13:23:50 (6.67 MB/s) - ‘test.qgs’ saved [27069/27069] $ /Applications/QGIS-LTR.app/Contents/MacOS/bin/qgis_process run native:printlayouttopdf --LAYOUT="Layout 1" --OUTPUT=output.pdf --PROJECT_PATH=test.qgs ---------------- Inputs ---------------- LAYOUT: Layout 1 OUTPUT: output.pdf Successfully exported layout to output.pdf 0...10...20...30...40...50...60...70...80...90...100 - done. ---------------- Results ---------------- OUTPUT: output.pdf ```
qgis_process using native:printlayouttopdf (with JSON input) (click to expand) ```bash cat my_json.json { "inputs": { "LAYOUT": "Layout 1", "TEXT_FORMAT": 0, "OUTPUT": "output.pdf" }, "project_path": "test.qgs" } cat my_json.json | /Applications/QGIS.app/Contents/MacOS/bin/qgis_process run native:printlayouttopdf - { "algorithm_details": { "can_cancel": true, "deprecated": false, "group": "Cartography", "has_known_issues": false, "help_url": null, "id": "native:printlayouttopdf", "name": "Export print layout as PDF", "requires_matching_crs": false, "short_description": "Exports a print layout as a PDF.", "tags": [ "layout", "composer", "composition", "save" ] }, "gdal_version": "3.3.2", "geos_version": "3.9.1-CAPI-1.14.2", "inputs": { "LAYOUT": "Layout 1", "OUTPUT": "output.pdf", "TEXT_FORMAT": 0 }, "log": { "info": [ "Successfully exported layout to output.pdf" ] }, "proj_version": "Rel. 8.1.1, September 1st, 2021", "project_path": "test.qgs", "provider_details": { "can_be_activated": true, "default_raster_file_extension": "tif", "default_vector_file_extension": "gpkg", "is_active": true, "long_name": "QGIS (native c++)", "name": "QGIS (native c++)", "supported_output_raster_extensions": [ "tif", "gen", "bag", "bmp", "bt", "byn", "bil", "ers", "gpkg", "grd", "grd", "gtx", "img", "mpr", "lbl", "kro", "ter", "mbtiles", "hdr", "mrf", "ntf", "gsb", "grd", "pix", "map", "pdf", "xml", "pgm", "rsw", "grd", "rst", "sdat", "rgb", "ter", "vrt", "nc" ], "supported_output_table_extensions": [ "gpkg", "shp", "000", "csv", "dgn", "dxf", "fgb", "geojson", "geojsonl", "geojsons", "gml", "gpx", "gxt", "ili", "itf", "json", "kml", "ods", "sql", "sqlite", "tab", "txt", "xlsx", "xml", "xtf" ], "supported_output_vector_extensions": [ "gpkg", "shp", "000", "csv", "dgn", "dxf", "fgb", "geojson", "geojsonl", "geojsons", "gml", "gpx", "gxt", "ili", "itf", "json", "kml", "ods", "sql", "sqlite", "tab", "txt", "xlsx", "xml", "xtf" ], "supports_non_file_based_output": true, "version": null }, "python_version": "3.9.5", "qgis_code_revision": "65e4edfdad", "qgis_version": "3.26.3-Buenos Aires", "qt_version": "5.15.2", "results": { "OUTPUT": "output.pdf" } } ```

The output file is created in both cases.

This means that the problem is not on the QGIS-on-macOS side, but on the R-on-macOS side.

If someone wants to get this algorithm running with R on macOS, then debugging is needed in R on macOS in order to find the cause. For example with following R code derived from the corresponding test:

library(qgisprocess)

tmp_pdf <- qgis_tmp_file(".pdf")

result <- qgis_run_algorithm(
  "native:printlayouttopdf",
  LAYOUT = "Layout 1",
  OUTPUT = tmp_pdf,
  PROJECT_PATH = system.file("extdata/longlake.qgs", package = "qgisprocess")
)

file.exists(tmp_pdf)
identical(tmp_pdf, as.character(result$OUTPUT))
unlink(tmp_pdf)