Use newest QGIS version by default

wlangera commented 11 months ago

Description of the requested feature

Currently, the R package defaults to using the first QGIS version it finds, which may not necessarily be the most recent one available on the device. I would propose to utilize the most recent QGIS version available on the system by default.

Additional context

Steps to Reproduce:

Install the R package on a system with multiple QGIS versions.
Observe which QGIS version the package selects by default.

florisvdh commented 11 months ago

Thanks for reporting this!

qgis_process is not necessarily installed in a file path that reflects its version, e.g. in case of OSGeo4W. When qgis_path(query = TRUE) is called (as done by qgis_configure()), and provided that the qgisprocess.path option is not set, qgis_process installations are searched for in typical locations and internally listed. Then, the first one on the list that actually works is taken and the rest of the procedure is aborted, favouring speed over a more complete comparison, e.g. by actually querying the version of each.

AFAIK the QGIS standalone on Win & macOS is by default installed in a path that contains a version number. So what we can do at least to improve the default sorting, is to put such installations at the top of the list in descending order; it doesn't matter in current philosophy and the delay to do that based on file path characteristics would be negligible. Considering that other installations may be present as well, not necessarily with version number in the file path, such behaviour may or may not lead to the latest installed version being at the top; however it would be convenient in several situations at least. Other setups (e.g. OSGeo4W, Linux, maybe some macOS approaches) by default upgrade QGIS in a single place.

The general idea is that you should use the qgisprocess.path option, followed by qgis_configure() to permanently set a preferred qgis_process file path in the package cache if you want to take control. You can also set the environment variable R_QGISPROCESS_PATH. Once the path is stored in the package cache, the package will continue using that specific path as long as it works, no matter which QGIS installations are going to be installed in other places. Manual control also enables support for QGIS installations that are in non-standard paths, e.g. local builds. This approach favours path stability and speed over searching and switching each time.

florisvdh commented 11 months ago

@wlangera can you have a try with remotes::install_github("r-spatial/qgisprocess@sort_paths").

Run following in a new R session; first removing current settings:

pkg_cache_path <- file.path(
  rappdirs::user_cache_dir("R-qgisprocess"),
  glue::glue("cache-{packageVersion('qgisprocess')}.rds")
)
if (file.exists(pkg_cache_path)) file.remove(pkg_cache_path)
library(qgisprocess)
qgis_path()
qgis_detect_windows_paths()

wlangera commented 11 months ago

Thanks for the explanation and the quick reply! It seems to work as expected :)

Before (install qgisprocess from CRAN):

library(qgisprocess)
#> Warning: package 'qgisprocess' was built under R version 4.3.2
#> Attempting to load the package cache ... No cache found.
#> Will try to reconfigure qgisprocess and build new cache ...
#> QGIS version: 3.26.3-Buenos
#> Having access to 1164 algorithms from 6 QGIS processing providers.
#> Run `qgis_configure(use_cached_data = TRUE)` to reload cache and get more details.
#> >>> Run `qgis_enable_plugins()` to enable 1 disabled plugin and access
#>     its algorithms: otbprovider
qgis_path()
#> [1] "C:/Program Files/QGIS 3.26.3/bin/qgis_process-qgis.bat"
qgis_detect_windows_paths()
#> [1] "C:/Program Files/QGIS 3.26.3/bin/qgis_process-qgis.bat"    
#> [2] "C:/Program Files/QGIS 3.28.6/bin/qgis_process-qgis-ltr.bat"

^{Created on 2023-12-19 with reprex v2.0.2}

After (install qgisprocess from branch sort_paths):

library(qgisprocess)
#> Attempting to load the package cache ... No cache found.
#> Will try to reconfigure qgisprocess and build new cache ...
#> 
#> Standard error message from 'qgis_process':
#> Problem with GRASS installation: GRASS was not found or is not correctly installed
#> 
#> 
#> 
#> Standard error message from 'qgis_process':
#> Problem with GRASS installation: GRASS was not found or is not correctly installed
#> 
#> 
#> QGIS version: 3.28.6-Firenze
#> Having access to 860 algorithms from 5 QGIS processing providers.
#> Run `qgis_configure(use_cached_data = TRUE)` to reload cache and get more details.
#> >>> Run `qgis_enable_plugins()` to enable 1 disabled plugin and access
#>     its algorithms: otbprovider
qgis_path()
#> [1] "C:/Program Files/QGIS 3.28.6/bin/qgis_process-qgis-ltr.bat"
qgis_detect_windows_paths()
#> [1] "C:/Program Files/QGIS 3.28.6/bin/qgis_process-qgis-ltr.bat"
#> [2] "C:/Program Files/QGIS 3.26.3/bin/qgis_process-qgis.bat"

^{Created on 2023-12-19 with reprex v2.0.2}

florisvdh commented 11 months ago

Thanks for testing!

florisvdh commented 11 months ago

@wlangera I'm trying a further addition to optionally drop the need for a proactive cache rebuilding by the user to get at the newer version, i.e. qgis_configure() still needed to be invoked by the user (also triggered in your test by removing the cache file before loading the package). Further explained in PR #192.

Could you install {qgisprocess} from the paths branch, and then:

first tell it to use your old QGIS again (3.26.3), by calling options(qgisprocess.path = "<path to 3.26.3 qgis_process-qgis-ltr.bat>") before running library(qgisprocess).
then restart R, call options(qgisprocess.detect_newer_qgis = TRUE), then run library(qgisprocess) again and please report what you see happen on Windows.
- A question should be asked relating to the available newer QGIS.
- Best first test the outcome of answering 'no' – it shouldn't rebuild the cache – then start over, answer 'yes' and this should trigger the uptake of the new version.
- Once 3.28.6 is in use, the question won't be asked again.

Note that in practice this option can be made default through e.g. Rprofile.site (potentially organisation-wide, as in https://github.com/inbo/tutorials/pull/331).

wlangera commented 11 months ago

Hi @florisvdh Here you can see what the code does.

# install qgisprocess from the paths branch
remotes::install_github("r-spatial/qgisprocess@paths")

# tell it to use old QGIS
options(qgisprocess.path = "C:/Program Files/QGIS 3.26.3/bin/qgis_process-qgis.bat")
library(qgisprocess)

# restart R (working in RStudio)
rstudioapi::executeCommand("restartR")

# execute 
options(qgisprocess.detect_newer_qgis = TRUE)
library(qgisprocess)

It asks the following question:

#> Attempting to load the package cache ... 
#> A newer QGIS installation seems to be available: 3.28.6.
#> Do you want to try it and rebuild the cache? (y/n)

I answer: n Response:

#> Success!
#> QGIS version: 3.26.3-Buenos
#> Having access to 1164 algorithms from 6 QGIS processing providers.
#> Run `qgis_configure(use_cached_data = TRUE)` to reload cache and get more details.
#> >>> Run `qgis_enable_plugins()` to enable 1 disabled plugin and access its algorithms: otbprovider

I run the code again.

# restart R (working in RStudio)
rstudioapi::executeCommand("restartR")

# execute 
options(qgisprocess.detect_newer_qgis = TRUE)
library(qgisprocess)

It asks the same question. This time I answer y Response:

#> Will try to reconfigure qgisprocess and build new cache ...
#> 
#> Standard error message from 'qgis_process':
#> Problem with GRASS installation: GRASS was not found or is not correctly installed
#> 
#> 
#> 
#> Standard error message from 'qgis_process':
#> Problem with GRASS installation: GRASS was not found or is not correctly installed
#> 
#> 
#> QGIS version: 3.28.6-Firenze
#> Having access to 860 algorithms from 5 QGIS processing providers.
#> Run `qgis_configure(use_cached_data = TRUE)` to reload cache and get more details.
#> >>> Run `qgis_enable_plugins()` to enable 1 disabled plugin and access its algorithms: otbprovider

The most recent version of QGIS is in use now. What happens if we rerun the code?

# restart R (working in RStudio)
rstudioapi::executeCommand("restartR")

# execute 
options(qgisprocess.detect_newer_qgis = TRUE)
library(qgisprocess)

This time it does not ask the question again:

#> Attempting to load the package cache ... 
#> Standard error message from 'qgis_process':
#> Problem with GRASS installation: GRASS was not found or is not correctly installed
#> 
#> 
#> Success!
#> QGIS version: 3.28.6-Firenze
#> Having access to 860 algorithms from 5 QGIS processing providers.
#> Run `qgis_configure(use_cached_data = TRUE)` to reload cache and get more details.
#> >>> Run `qgis_enable_plugins()` to enable 1 disabled plugin and access its algorithms: otbprovider

So this seems to work as expected. Going back to the old version works also as expected:

# restart R (working in RStudio)
rstudioapi::executeCommand("restartR")

# can we go back to old version if we want?
options(qgisprocess.path = "C:/Program Files/QGIS 3.26.3/bin/qgis_process-qgis.bat")
library(qgisprocess)

Response:

#> Attempting to load the package cache ... 
#> The user's qgisprocess.path option or the R_QGISPROCESS_PATH environment variable specify a different qgis_process path (C:/Program Files/QGIS 3.26.3/bin/qgis_process-qgis.bat) than the cache did (C:/Program Files/QGIS 3.28.6/bin/qgis_process-qgis-ltr.bat).
#> Hence rebuilding cache to reflect this change ...
#> QGIS version: 3.26.3-Buenos
#> Having access to 1164 algorithms from 6 QGIS processing providers.
#> Run `qgis_configure(use_cached_data = TRUE)` to reload cache and get more details.
#> >>> Run `qgis_enable_plugins()` to enable 1 disabled plugin and access its algorithms: otbprovider

florisvdh commented 11 months ago

Great, thanks a lot for testing and reporting this in a clear way @wlangera! :rocket:

r-spatial / qgisprocess

Use newest QGIS version by default #187

Description of the requested feature

Additional context