package dependency installation for subdirectories in GIT

[siri16100]

I have some R packages in my companies repo. But they are not in the root path but in a subdirectory. I can easly install this packages using the subdir argument in remotes::install_git. However I do not know how to correctly specify the sub-directory in the Remotes section of the DESCRIPTION file.

All works find if each package is in one corresponding git repos using :

Remotes: 
  git::https://<company>@dev.azure.com/<company>/<department>/_git/<package>

but if I use something like

 Remotes: 
  git::https://<company>@dev.azure.com/<company>/<department>/_git/<reponame>/packages/<package>

to address that the package lives in a subdir of a repo I get an error:

Error: Failed to install 'unknown package' from Git:
  can't convert package <package> with RemoteType 'git' to remote

I did consult this link however it is not clear to me how to specify the git subdirectory .

[gaborcsardi]

I am afraid that there is no way to do that currently.

[kevinushey]

For what it's worth, in renv, we've adopted the convention of using : to delimit sub-directories. I added my rationale here:

https://github.com/rstudio/renv/issues/497#issuecomment-664771124

Is this something that remotes could also accept?

[gaborcsardi]

It is unlikely that we would implement this in remotes, but we could implement it in pak.

[siri16100]

Hi @gaborcsardi thanks for sharing that sub-directories are currently not supported by remotes. As I'm not very familiar with pak, would a implementation of : as delimiter for subdirectories indeed fix the problem of specifying package dependencies in the DESCRIPTION file of a package?

[gaborcsardi]

I am sorry, I am not sure what you mean.

[siri16100]

Regarding my upper question you stated, that it is currently not possible to add a sub-directory GIT sources to the Remotes in a DESCRIPTION file of a package. Is there any chance this is possible in future releases? Further I don't quite get (as I'm not really familiar with pak ) how this:

It is unlikely that we would implement this in remotes, but we could implement it in pak.

would help solving the issue?

[gaborcsardi]

Is there any chance this is possible in future releases?

As we are focusing on pak, we are not adding new features to remotes, so it is unlikely.

how this would help solving the issue?

You could use pak instead of remotes as it is mostly superior to remotes.

[siri16100]

Thanks @gaborcsardi. As I used used renv:install before I just played around a little with pak:pgk_install. I was able to install a package without any dependencies from azure. However if I try a package with dependencies I run into following error:

Error: 
! error in pak subprocess
Caused by error: 
! Could not solve package dependencies:
* git::https://dev.azure.com/<company>/<department>/_git/<package>:
  * Can't install dependency <package dep 1> (>= 0.1.3) (>= 0.0.3) (>= 0.0.2)
  * Can't install dependency <package dep 2> (>= 0.0.3) (>= 0.0.2)
  * Can't install dependency <package dep 3> (>= 0.0.2)
* <package dep 1>: Can't find package called <package dep 1>, <package dep 2>, <package dep 3>, <package dep 4>.
* <package dep 2>: Can't find package called <package dep 1>, <package dep 2>, <package dep 3>, <package dep 4>.
* <package dep 3>: Can't find package called <package dep 1>, <package dep 2>, <package dep 3>, <package dep 4>.
---
Backtrace:
1. pak::pkg_install("git::https://dev.azure.com/<company>/<department>/_git/<package>?reinstall")
2. pak:::remote(function(...) get("pkg_install_make_plan", asNamespace("pak"))(...), …
3. err$throw(res$error)
---
Subprocess backtrace:
1. base::withCallingHandlers(cli_message = function(msg) { …
2. get("pkg_install_make_plan", asNamespace("pak"))(...)
3. prop$stop_for_solution_error()
4. private$plan$stop_for_solve_error()
5. pkgdepends:::pkgplan_stop_for_solve_error(self, private)
6. base::throw(new_error("Could not solve package dependencies:\n", msg, …
7. | base::signalCondition(cond)
8. global (function (e) …

what I still don't get is the link between the pak package and this vignette. Is pak:pgk_install able to make use of Remotes in a packages DESCRIPTION file? If yes I wonder why renv:install does work but pak:pgk_install not....

> sessionInfo()
R version 4.3.2 (2023-10-31 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19045)

Matrix products: default

locale:
[1] LC_COLLATE=German_Germany.utf8  LC_CTYPE=German_Germany.utf8    LC_MONETARY=German_Germany.utf8 LC_NUMERIC=C                   
[5] LC_TIME=German_Germany.utf8    

time zone: UTC
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] devtools_2.4.5 usethis_2.2.2 

loaded via a namespace (and not attached):
 [1] miniUI_0.1.1.1    renv_1.0.3        compiler_4.3.2    crayon_1.5.2      promises_1.2.1    Rcpp_1.0.11       stringr_1.5.0    
 [8] callr_3.7.3       later_1.3.1       fastmap_1.1.1     mime_0.12         R6_2.5.1          pak_0.6.0.9001    htmlwidgets_1.6.2
[15] profvis_0.3.8     shiny_1.7.5.1     rlang_1.1.2       cachem_1.0.8      stringi_1.7.12    httpuv_1.6.12     fs_1.6.3         
[22] pkgload_1.3.3     memoise_2.0.1     cli_3.6.1         magrittr_2.0.3    ps_1.7.5          digest_0.6.33     processx_3.8.2   
[29] rstudioapi_0.15.0 xtable_1.8-4      remotes_2.4.2.1   lifecycle_1.0.4   prettyunits_1.1.1 vctrs_0.6.4       glue_1.6.2       
[36] urlchecker_1.0.1  sessioninfo_1.2.2 pkgbuild_1.4.2    purrr_1.0.1       tools_4.3.2       ellipsis_0.3.2    htmltools_0.5.5 

[gaborcsardi]

what I still don't get is the link between the pak package and this vignette. Is pak:pgk_install able to make use of Remotes in a packages DESCRIPTION file?

Yes: https://pak.r-lib.org/reference/pak_package_sources.html#the-remotes-field

If yes I wonder why renv:install does work but pak:pgk_install not....

Hard to say without seeing your DESCRIPTION file and knowing the layout of your git repos, unfortunately.

[siri16100]

Thanks for sharing! I have currently 2 repos each containing exactly one package. Let's say: https://dev.azure.com/<company>/<department>/_git/pkgA and https://dev.azure.com/<company>/<department>/_git/pkgB

pkgB has a dependency of pkgA which is specified as following in package pkgB DESCRIPTION file:

Package: pkgB
Title: foo 
Version: 0.0.6
Authors@R: 

Description: foo.
License: file LICENSE
Encoding: UTF-8
Roxygen: list(markdown = TRUE)
RoxygenNote: 7.2.3
Imports: 
    pkgA (>= 0.1.3),
    dplyr (>= 1.1.2),
    glue (>= 1.6.2),
    lubridate (>= 1.9.2),
    purrr (>= 1.0.1),
    RPostgres (>= 1.4.5),
    stringr (>= 1.5.0),
    tidyr (>= 1.3.0)
Remotes: 
  git::https://dev.azure.com/<company>/<department>/_git/pkgA

[gaborcsardi]

Yes, AFAICT that should work, provided that you have the credentials to access these repositories in the git credential store, and you are not running into this bug: https://github.com/r-lib/pkgdepends/issues/319

[siri16100]

Ok thanks. As I have successfully installed pkgA using pak::pkg_install, there shouldn't be any problems with the authentication I guess. But anyway, thanks for confirming that I used the right specification in DESCRIPTION...