search

rstudio / renv

renv: Project environments for R.
https://rstudio.github.io/renv/
MIT License
1.01k stars 153 forks source link

renv::install() of github packages causes issues with deploying plumber APIs #185

Closed s-fleck closed 4 years ago

s-fleck commented 5 years ago

I am using renv to lock the dependencies for a plumber API that I am deploying via rsconnect. If I install a package from github via remotes::install_github() I can deploy the api without issue. If I instead install the package with renv::install() it installs fine, but I cannot deploy the api as rsconnect::deployAPI() tries to download the package from CRAN and cannot find it.

I am using the latest github version of renv

kevinushey commented 5 years ago

I wasn't able to reproduce this. Can you be more explicit on how you're using renv::install() to install the package? What is the output of:

readLines(system.file("DESCRIPTION", package = <package>))

(replacing <package> with your package name)?

s-fleck commented 5 years ago

I have to check on my collegues computer, I'll get back to this next week. Maybe this helps:

I have the following in the DESCRIPTION of my cached httpuv of the library that I fixed with remotes::install_github().

[...]
RemoteType: github
RemoteHost: api.github.com
RemoteRepo: httpuv
RemoteUsername: rstudio
RemoteRef: master
RemoteSha: a178e4a3e73605f4494b23b8b3e6b11c0e9f7310
GithubRepo: httpuv
GithubUsername: rstudio
GithubRef: master
GithubSHA1: a178e4a3e73605f4494b23b8b3e6b11c0e9f7310
NeedsCompilation: yes

Is seems renv::install() only adds the Remote* entries, while remotes::install_github() adds the Remote* and Github* entries. Maybe rsconnect has problems with the one but not with the other?

kevinushey commented 5 years ago

rsconnect will use Packrat behind the scenes, and I think things should work as long as you have the development version of Packrat installed. Does installing that make a difference? Try e.g.

renv::install("rstudio/packrat")

to install it.

s-fleck commented 5 years ago

hmm nope that does not change anything. The packrat on the remote rsconnect server just goes into

2019/09/20 08:42:40.357335107 Installing httpuv (1.5.2.9000) ... 
2019/09/20 08:42:40.431634382 curl: (22) The requested URL returned error: 404 Not Found
...
2019/09/20 08:42:52.099048710 FAILED
2019/09/20 08:42:52.099545244 Error in getSourceForPkgRecord(pkgRecord, srcDir(project), availablePackagesSource(repos = repos), : Failed to retrieve package sources for httpuv 1.5.2.9000 from CRAN (internet connectivity issue?) [1.5.2 is current]

DESCRIPTION after renv::install() (deploying on rsconnect does not work because of the problem described above)

Package: httpuv
Type: Package
Encoding: UTF-8
Title: HTTP and WebSocket Server Library
Version: 1.5.2.9000
Author: Joe Cheng, Hector Corrada Bravo [ctb], Jeroen Ooms [ctb],
 Winston Chang [ctb]
Copyright: RStudio, Inc.; Joyent, Inc.; Nginx Inc.; Igor Sysoev; Niels Provos;
    Internet Systems Consortium, Inc.; Alexander Chemeris; Berkeley Software
    Design; Google Inc.; Sony Mobile Communications AB; Alexander Peslyak; Free
    Software Foundation, Inc.; X Consortium; Ben Noordhuis; StrongLoop, Inc.;
    Saúl Ibarra Corretgé; Bert Belder; Fedor Indutny; libuv project; Refael
    Ackermann; Kenneth MacKay; Emergya; Diego Pettenò; xine project, The
    Regents of the University of California, Dariusz Dwornikowski
Maintainer: Joe Cheng <joe@rstudio.com>
Description: Provides low-level socket and protocol support for handling
 HTTP and WebSocket requests directly from within R. It is primarily
 intended as a building block for other packages, rather than making it
 particularly easy to create complete web applications using httpuv alone.
 httpuv is built on top of the libuv and http-parser C libraries, both of
 which were developed by Joyent, Inc. (See LICENSE file for libuv and
 http-parser license information.)
License: GPL (>= 2) | file LICENSE
Depends: R (>= 2.15.1)
Imports: Rcpp (>= 0.11.0), utils, R6, promises, later (>= 0.8.0)
LinkingTo: Rcpp, BH, later
URL: https://github.com/rstudio/httpuv
SystemRequirements: GNU make
RoxygenNote: 6.1.1
Suggests: testthat, callr, curl, websocket
Collate: 'RcppExports.R' 'httpuv.R' 'random_port.R' 'server.R' 'static_paths.R'
    'utils.R'
Built: R 3.6.0; x86_64-pc-linux-gnu; 2019-09-20 06:39:46 UTC; unix
RemoteType: github
RemoteHost: api.github.com
RemoteUsername: rstudio
RemoteRepo: httpuv
RemoteRef: master
RemoteSha: a086ee2ab018ec35788600c3315da0c94527eb7c

DESCRIPTION after remotes::install_github() (deploying on rsconenct now works after restarting the R session)

Package: httpuv
Type: Package
Encoding: UTF-8
Title: HTTP and WebSocket Server Library
Version: 1.5.2.9000
Author: Joe Cheng, Hector Corrada Bravo [ctb], Jeroen Ooms [ctb],
    Winston Chang [ctb]
Copyright: RStudio, Inc.; Joyent, Inc.; Nginx Inc.; Igor Sysoev; Niels
        Provos; Internet Systems Consortium, Inc.; Alexander Chemeris;
        Berkeley Software Design; Google Inc.; Sony Mobile
        Communications AB; Alexander Peslyak; Free Software Foundation,
        Inc.; X Consortium; Ben Noordhuis; StrongLoop, Inc.; Saúl
        Ibarra Corretgé; Bert Belder; Fedor Indutny; libuv project;
        Refael Ackermann; Kenneth MacKay; Emergya; Diego Pettenò; xine
        project, The Regents of the University of California, Dariusz
        Dwornikowski
Maintainer: Joe Cheng <joe@rstudio.com>
Description: Provides low-level socket and protocol support for handling
    HTTP and WebSocket requests directly from within R. It is primarily
    intended as a building block for other packages, rather than making it
    particularly easy to create complete web applications using httpuv alone.
    httpuv is built on top of the libuv and http-parser C libraries, both of
    which were developed by Joyent, Inc. (See LICENSE file for libuv and
    http-parser license information.)
License: GPL (>= 2) | file LICENSE
Depends: R (>= 2.15.1)
Imports: Rcpp (>= 0.11.0), utils, R6, promises, later (>= 0.8.0)
LinkingTo: Rcpp, BH, later
URL: https://github.com/rstudio/httpuv
SystemRequirements: GNU make
RoxygenNote: 6.1.1
Suggests: testthat, callr, curl, websocket
Collate: 'RcppExports.R' 'httpuv.R' 'random_port.R' 'server.R'
        'static_paths.R' 'utils.R'
RemoteType: github
RemoteHost: api.github.com
RemoteRepo: httpuv
RemoteUsername: rstudio
RemoteRef: master
RemoteSha: a086ee2ab018ec35788600c3315da0c94527eb7c
GithubRepo: httpuv
GithubUsername: rstudio
GithubRef: master
GithubSHA1: a086ee2ab018ec35788600c3315da0c94527eb7c
NeedsCompilation: yes
Packaged: 2019-09-20 06:44:42 UTC; fleck
Built: R 3.6.0; x86_64-pc-linux-gnu; 2019-09-20 06:44:44 UTC; unix
kevinushey commented 4 years ago

Sorry for taking so long to come back to this issue.

There have been a variety of changes in Packrat that I believe should resolve this issue; e.g.

https://github.com/rstudio/packrat/commit/29a4fdcb5d34113222b73e4eb2affd0e3fc3f1fd

Unfortunately, this update is not yet on CRAN, but I believe should be vendored by newer versions of RStudio Connect.

kevinushey commented 4 years ago

I'm going to close this issue, but please feel free to re-open if you are still having trouble.

oude-gao commented 3 years ago

Hi I am experiencing the same issue, it's an internal package on Github, the DESCRIPTION file looks like this:

Package: utlbia ... License: GPL-3 Encoding: UTF-8 LazyData: true Roxygen: list(markdown = TRUE) RoxygenNote: 7.1.1 Depends: R (>= 2.10) Imports: odbc, shiny, dplyr RemoteType: github RemoteHost: api.github.com RemoteRepo: utlbia RemoteUsername: vnsny-bia RemoteRef: dev_datasci RemoteSha: 0a4eaf5764b0028e8775565886e9f3b0af7c8858 GithubRepo: utlbia GithubUsername: vnsny-bia GithubRef: dev_datasci GithubSHA1: 0a4eaf5764b0028e8775565886e9f3b0af7c8858 NeedsCompilation: no Packaged: 2021-02-11 20:34:00 UTC; root Built: R 4.0.3; ; 2021-02-11 20:34:01 UTC; unix

And when I am deploying to rsconnect, it's still trying to install the package from CRAN:

[Connect] 2021/02/11 21:13:41.029774085 Installing utlbia (0.0.1.0031) ... [Connect] 2021/02/11 21:13:41.136226299 curl: HTTP 200 https://cran.rstudio.com/src/contrib/PACKAGES.rds [Connect] 2021/02/11 21:13:41.894970372 curl: HTTP 404 https://cran.rstudio.com/src/contrib/Archive/utlbia/utlbia_0.0.1.0031.tar.gz

The most recent packrat is installed: [Connect] 2021/02/11 21:13:40.091572456 Managed packrat version: 0.5.0.26

I would really appreciate it if some solution could be provided. Thanks!

kevinushey commented 3 years ago

This appears to be an RStudio Connect + Packrat issue rather than an renv issue; I'd recommend following up with RStudio Connect's support team specifically. (Most likely, the manifest generated during deployment is capturing a different version of the utlbia package, or something else is going on at that stage)

oude-gao commented 3 years ago

Thanks! Apologies for mixing it up, i was search for the same error and came across this.