rstudio / rsconnect

Publish Shiny Applications, RMarkdown Documents, Jupyter Notebooks, Plumber APIs, and more
http://rstudio.github.io/rsconnect/
133 stars 81 forks source link

default image from RSCONNECT_IMAGE #1065

Closed aronatkins closed 5 months ago

aronatkins commented 5 months ago

fixes #1063

@hadley - preference towards R option versus environment variable? The original request was for an environment variable, which feels more "global", especially if we have non-R tooling use the same variable.

dethmasque commented 4 months ago

EDIT: RESTART YOUR R SESSION IF YOU HIT THIS:

I thought I was seeing issues due to me having built and tested out rsconnect deploy logs to a file locally, but after many forced reinstalls via GH I am starting to think this corrupted RDB file isn't from that. Maybe it's being generated off of some lingering bad bits?

Error in fetch(key) : 
  lazy-load database '/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/library/rsconnect/help/rsconnect.rdb' is corrupt
Error in fetch(key) : 
  lazy-load database '/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/library/rsconnect/help/rsconnect.rdb' is corrupt
> devtools::install_github("https://github.com/rstudio/rsconnect", force = TRUE)
Downloading GitHub repo rstudio/rsconnect@HEAD
Enter one or more numbers, or an empty line to skip updates: 
* installing *source* package ‘rsconnect’ ...
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
*** copying figures
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (rsconnect)
Error in fetch(key) : 
  lazy-load database '/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/library/rsconnect/help/rsconnect.rdb' is corrupt
total 304
-rw-rw-r--@  1 dethmasque  admin     680 May  9 09:22 paths.rds
-rw-rw-r--@  1 dethmasque  admin  135771 May  9 09:22 rsconnect.rdb
-rw-rw-r--@  1 dethmasque  admin    1347 May  9 09:22 rsconnect.rdx
-rw-rw-r--@  1 dethmasque  admin    1661 May  9 09:22 AnIndex
-rw-rw-r--@  1 dethmasque  admin     600 May  9 09:22 aliases.rds
drwxrwxr-x@  8 dethmasque  admin     256 May  9 09:22 .
drwxrwxr-x@ 13 dethmasque  admin     416 May  9 09:22 figures
drwxrwxr-x@ 13 dethmasque  admin     416 May  9 09:22 ..
dethmasque commented 4 months ago

Is there a way to not truncate the argument list?

image

I was hoping to see that we had a default value in there as part of testing.

dethmasque commented 4 months ago

@aronatkins Do we expect writeManifest() to fail in the same way deployApp() does when a bogus image is passed? I am seeing it succeed:

> rsconnect::writeManifest(image = "bogus")
ℹ Capturing R dependencies with renv
✔ Found 31 dependencies

Where deployApp() fails:

> rsconnect::deployApp(image = "bogus")
── Preparing for deployment ────────────────────────────────────────────────────────────────
✔ Re-deploying "newlog" using "server: localhost / username: toni"
ℹ Looking up application with id "78"...
✔ Found application <http://localhost:3939/content/1a63d3a7-c900-4a64-81dd-c1ab8d171d15/>
ℹ Bundling 2 files: app.R and testing.txt
ℹ Capturing R dependencies with renv
✔ Found 31 dependencies
✔ Created 21,144b bundle
ℹ Uploading bundle...
✔ Uploaded bundle with id 129
── Deploying to server ─────────────────────────────────────────────────────────────────────
Building Shiny application...
The manifest.json requests image bogus.
A target image was selected: bogus, checking if it is compatible...
Cannot find compatible environment: no compatible Kubernetes environment with image named bogus and R version 4.2.2
── Deployment complete ─────────────────────────────────────────────────────────────────────
✖ Deployment failed with error: Cannot find compatible environment: no compatible Kubernetes environment with image named bogus and R version 4.2.2

I confirmed that the environment variable is used by default as well:

> Sys.getenv("RSCONNECT_IMAGE")
[1] "bogus"
> rsconnect::deployApp()
── Preparing for deployment ────────────────────────────────────────────────────────────────
✔ Re-deploying "newlog" using "server: localhost / username: toni"
ℹ Looking up application with id "78"...
✔ Found application <http://localhost:3939/content/1a63d3a7-c900-4a64-81dd-c1ab8d171d15/>
ℹ Bundling 2 files: app.R and testing.txt
ℹ Capturing R dependencies with renv
✔ Found 31 dependencies
✔ Created 21,145b bundle
ℹ Uploading bundle...
✔ Uploaded bundle with id 131
── Deploying to server ─────────────────────────────────────────────────────────────────────
Building Shiny application...
The manifest.json requests image bogus.
A target image was selected: bogus, checking if it is compatible...
Cannot find compatible environment: no compatible Kubernetes environment with image named bogus and R version 4.2.2
── Deployment complete ─────────────────────────────────────────────────────────────────────
✖ Deployment failed with error: Cannot find compatible environment: no compatible Kubernetes environment with image named bogus and R version 4.2.2

Associated Connect logs:

time="2024-05-09T18:53:29.727Z" level=info msg="Writing bundle to: /connect/data/bundles/app-78-132.tar.gz" bundle_id=132 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 correlation_id=9d45d829-f2d4-4195-9acc-7ff0b9ae4ba2
time="2024-05-09T18:53:29.728Z" level=info msg="Bundle /connect/data/bundles/app-78-132.tar.gz uploaded with 21145 bytes" bundle_id=132 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 correlation_id=9d45d829-f2d4-4195-9acc-7ff0b9ae4ba2
time="2024-05-09T18:53:29.728Z" level=info msg="Extracting bundle /connect/data/bundles/app-78-132.tar.gz to: /connect/data/apps/78/132" bundle_id=132 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 correlation_id=9d45d829-f2d4-4195-9acc-7ff0b9ae4ba2
time="2024-05-09T18:53:29.773Z" level=info msg="Building Shiny application..." bundle_id=132 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 correlation_id=527d75a7-d65e-48eb-9bac-586fe49e0bd1
time="2024-05-09T18:53:29.775Z" level=info msg="The manifest.json requests image bogus." bundle_id=132 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 correlation_id=527d75a7-d65e-48eb-9bac-586fe49e0bd1
time="2024-05-09T18:53:29.775Z" level=info msg="A target image was selected: bogus, checking if it is compatible..." bundle_id=132 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 correlation_id=527d75a7-d65e-48eb-9bac-586fe49e0bd1
time="2024-05-09T18:53:29.775Z" level=error msg="Cannot find compatible environment: no compatible Kubernetes environment with image named bogus and R version 4.2.2" bundle_id=132 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 correlation_id=527d75a7-d65e-48eb-9bac-586fe49e0bd1

With a manifest.json that was created while RSCONNECT_IMAGE="bogus", deployApp() succeeded:

Deployment failed with error: Cannot find compatible environment: no compatible Kubernetes environment with image named bogus and R version 4.2.2
> rsconnect::writeManifest()
ℹ Capturing R dependencies with renv
✔ Found 31 dependencies
> Sys.unsetenv("RSCONNECT_IMAGE")
> rsconnect::deployApp()
── Preparing for deployment ────────────────────────────────────────────────────────────────
✔ Re-deploying "newlog" using "server: localhost / username: toni"
ℹ Looking up application with id "78"...
✔ Found application <http://localhost:3939/content/1a63d3a7-c900-4a64-81dd-c1ab8d171d15/>
ℹ Bundling 2 files: app.R and testing.txt
ℹ Capturing R dependencies with renv
✔ Found 31 dependencies
✔ Created 21,128b bundle
ℹ Uploading bundle...
✔ Uploaded bundle with id 133
── Deploying to server ─────────────────────────────────────────────────────────────────────
Building Shiny application...
Bundle created with R version 4.2.2 is compatible with environment Kubernetes::rstudio/connect:ubuntu22 with R version 4.3.0 from /opt/R/4.3.0/bin/R 
Bundle requested R version 4.2.2; using /opt/R/4.3.0/bin/R from Kubernetes::rstudio/connect:ubuntu22 which has version 4.3.0
Performing manifest.json to packrat transformation.
Determining session server location ...
[rsc-session] Content GUID: 1a63d3a7-c900-4a64-81dd-c1ab8d171d15
[rsc-session] Content ID: 78
[rsc-session] Bundle ID: 133
[rsc-session] Job Key: RU38RrMQhSvoc4ch
Connecting to session server http://192.168.65.3:32537 ...
Connected to session server http://192.168.65.3:32537
Running on host: packrat-restore-np242-jgbns
Linux distribution: Ubuntu 22.04.2 LTS (jammy)
Running as user: uid=999(jenkins) gid=999(jenkins) groups=999(jenkins)
Connect version: 2024.05.0-dev+64-ge3bb4ab213
LANG: C.utf8
Working directory: /connect/mnt/app
Using R 4.3.0
R.home(): /opt/R/4.3.0/lib/R
Using user agent string: 'RStudio R (4.3.0 x86_64-pc-linux-gnu x86_64 linux-gnu)' 
Configuring packrat to use available credentials for private repository access.
# Validating R library read / write permissions --------------------------------
Using R library for packrat bootstrap: /connect/mnt/R/rstudio_connect__ubuntu22/4.3.0
# Validating managed packrat installation --------------------------------------
Vendored packrat archive: /connect/ext/R/packrat_0.9.2.9000_70625806c44bda42a7f3aeaa92ee65542cc590be.tar.gz
Vendored packrat SHA: 70625806c44bda42a7f3aeaa92ee65542cc590be
Managed packrat SHA:  70625806c44bda42a7f3aeaa92ee65542cc590be
Managed packrat version: 0.9.2.9000
Managed packrat is up-to-date.
# Validating packrat cache read / write permissions ----------------------------
Using packrat cache directory: /connect/mnt/packrat/rstudio_connect__ubuntu22/4.3.0
# Setting packrat options and preparing lockfile -------------------------------
Audited package hashes with local packrat installation.
# Resolving R package repositories ---------------------------------------------
Received repositories from published content:
- CRAN = "https://cran.rstudio.com"
# Installing required R packages with `packrat::restore()` ---------------------
Warning in packrat::restore(overwrite.dirty = TRUE, prompt = FALSE, restart = FALSE) :
  The most recent snapshot was generated using R version 4.2.2
Installing R6 (2.5.1) ... 
    OK (symlinked cache)
Installing Rcpp (1.0.9) ... 
    OK (symlinked cache)
Installing base64enc (0.1-3) ... 
    OK (symlinked cache)
Installing cli (3.6.1) ... 
    OK (symlinked cache)
Installing commonmark (1.8.1) ... 
    OK (symlinked cache)
Installing crayon (1.5.2) ... 
    OK (symlinked cache)
Installing digest (0.6.33) ... 
    OK (symlinked cache)
Installing fastmap (1.1.1) ... 
    OK (symlinked cache)
Installing fs (1.5.2) ... 
    OK (symlinked cache)
Installing glue (1.6.2) ... 
    OK (symlinked cache)
Installing jsonlite (1.8.7) ... 
    OK (symlinked cache)
Installing magrittr (2.0.3) ... 
    OK (symlinked cache)
Installing mime (0.12) ... 
    OK (symlinked cache)
Installing rappdirs (0.3.3) ... 
    OK (symlinked cache)
Installing rlang (1.1.1) ... 
    OK (symlinked cache)
Installing sourcetools (0.1.7-1) ... 
    OK (symlinked cache)
Installing withr (2.5.0) ... 
    OK (symlinked cache)
Installing xtable (1.8-4) ... 
    OK (symlinked cache)
Installing cachem (1.0.6) ... 
    OK (symlinked cache)
Installing ellipsis (0.3.2) ... 
    OK (symlinked cache)
Installing htmltools (0.5.8.1) ... 
    OK (symlinked cache)
Installing later (1.3.0) ... 
    OK (symlinked cache)
Installing lifecycle (1.0.3) ... 
    OK (symlinked cache)
Installing memoise (2.0.1) ... 
    OK (symlinked cache)
Installing fontawesome (0.5.0) ... 
    OK (symlinked cache)
Installing jquerylib (0.1.4) ... 
    OK (symlinked cache)
Installing sass (0.4.9) ... 
    OK (symlinked cache)
Installing promises (1.2.0.1) ... 
    OK (symlinked cache)
Installing bslib (0.7.0) ... 
    OK (symlinked cache)
Installing httpuv (1.6.9) ... 
    OK (symlinked cache)
Installing shiny (1.7.4) ... 
    OK (symlinked cache)
Stopped session pings to http://192.168.65.3:32537
Launching Shiny application...
── Deployment complete ─────────────────────────────────────────────────────────────────────
✔ Successfully deployed to <http://localhost:3939/content/1a63d3a7-c900-4a64-81dd-c1ab8d171d15/>
> 

Associated Connect logs:

time="2024-05-09T18:54:06.210Z" level=info msg="Writing bundle to: /connect/data/bundles/app-78-133.tar.gz" bundle_id=133 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 correlation_id=bbb0b9c6-27ba-4f8b-8a04-df8bfd4e206b
time="2024-05-09T18:54:06.220Z" level=info msg="Bundle /connect/data/bundles/app-78-133.tar.gz uploaded with 21128 bytes" bundle_id=133 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 correlation_id=bbb0b9c6-27ba-4f8b-8a04-df8bfd4e206b
time="2024-05-09T18:54:06.221Z" level=info msg="Extracting bundle /connect/data/bundles/app-78-133.tar.gz to: /connect/data/apps/78/133" bundle_id=133 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 correlation_id=bbb0b9c6-27ba-4f8b-8a04-df8bfd4e206b
time="2024-05-09T18:54:06.269Z" level=info msg="Building Shiny application..." bundle_id=133 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 correlation_id=38288f4a-f066-4929-ac71-90ad83694233
time="2024-05-09T18:54:06.271Z" level=info msg="Bundle created with R version 4.2.2 is compatible with environment Kubernetes::rstudio/connect:ubuntu22 with R version 4.3.0 from /opt/R/4.3.0/bin/R " bundle_id=133 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 correlation_id=38288f4a-f066-4929-ac71-90ad83694233
time="2024-05-09T18:54:06.271Z" level=info msg="Bundle requested R version 4.2.2; using /opt/R/4.3.0/bin/R from Kubernetes::rstudio/connect:ubuntu22 which has version 4.3.0" bundle_id=133 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 correlation_id=38288f4a-f066-4929-ac71-90ad83694233
time="2024-05-09T18:54:06.272Z" level=warning msg="A plain text value is potentially being used for the sensitive setting 'GitCredential\"github\".Password'. The Configuration Appendix of the Posit Connect Admin Guide explains encrypted strings and how to use them to protect this value."
time="2024-05-09T18:54:06.272Z" level=warning msg="A plain text value is potentially being used for the sensitive setting 'GitCredential\"gitlab\".Password'. The Configuration Appendix of the Posit Connect Admin Guide explains encrypted strings and how to use them to protect this value."
time="2024-05-09T18:54:06.272Z" level=warning msg="A plain text value is potentially being used for the sensitive setting 'GitCredential\"bitbucket\".Password'. The Configuration Appendix of the Posit Connect Admin Guide explains encrypted strings and how to use them to protect this value."
time="2024-05-09T18:54:06.272Z" level=info msg="Performing manifest.json to packrat transformation." bundle_id=133 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 correlation_id=38288f4a-f066-4929-ac71-90ad83694233
time="2024-05-09T18:54:06.272Z" level=debug msg="Creating working directory in /connect/data/apps/78/133" region=rproc
time="2024-05-09T18:54:06.273Z" level=debug msg="Using working directory: /connect/data/apps/78/133/packrat.working.3057790753" region=rproc
time="2024-05-09T18:54:06.273Z" level=debug msg="Creating directory: /connect/data/apps/78/133/packrat.working.3057790753/desc" region=rproc
time="2024-05-09T18:54:06.274Z" level=debug msg="Writing lock file: /connect/data/apps/78/133/packrat.working.3057790753/packrat.lock" region=rproc
time="2024-05-09T18:54:06.287Z" level=debug msg="Writing package descriptions: /connect/data/apps/78/133/packrat.working.3057790753/desc" region=rproc
time="2024-05-09T18:54:06.379Z" level=debug msg="Renaming working directory: /connect/data/apps/78/133/packrat.working.3057790753 => /connect/data/apps/78/133/packrat" region=rproc
time="2024-05-09T18:54:06.405Z" level=info msg="Determining session server location ..." bundle_id=133 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 correlation_id=38288f4a-f066-4929-ac71-90ad83694233
time="2024-05-09T18:54:07.430Z" level=info msg="Connecting to session server http://192.168.65.3:32537 ..." bundle_id=133 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 correlation_id=38288f4a-f066-4929-ac71-90ad83694233
time="2024-05-09T18:54:07.436Z" level=info msg="Connected to session server http://192.168.65.3:32537" bundle_id=133 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 correlation_id=38288f4a-f066-4929-ac71-90ad83694233
time="2024-05-09T18:54:10.292Z" level=info msg="Stopped session pings to http://192.168.65.3:32537" bundle_id=133 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 correlation_id=38288f4a-f066-4929-ac71-90ad83694233
time="2024-05-09T18:54:10.294Z" level=info msg="Launching Shiny application..." bundle_id=133 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 correlation_id=38288f4a-f066-4929-ac71-90ad83694233
time="2024-05-09T18:54:11.481Z" level=info msg="Starting an initial worker; 0 workers are servicing 1 connections" bundle_id=133 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 mode=shiny
time="2024-05-09T18:54:11.483Z" level=info msg="Using environment Kubernetes::rstudio/connect:ubuntu22" bundle_id=133 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 mode=shiny worker=d4efe777
time="2024-05-09T18:54:11.484Z" level=info msg="Using /opt/R/4.3.0/bin/R with version 4.3.0 from Kubernetes::rstudio/connect:ubuntu22" bundle_id=133 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 mode=shiny worker=d4efe777
time="2024-05-09T18:54:11.528Z" level=info msg="WorkerStart {workerId:d4efe777}" bundle_id=133 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 mode=shiny
time="2024-05-09T18:54:11.528Z" level=info msg="Determining worker location ..." bundle_id=133 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 mode=shiny pid=0 worker=d4efe777
time="2024-05-09T18:54:11.528Z" level=info msg="Determining session server location ..." bundle_id=133 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 mode=shiny pid=0 worker=d4efe777
time="2024-05-09T18:54:12.583Z" level=info msg="Connecting to session server http://192.168.65.3:32102 ..." bundle_id=133 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 mode=shiny pid=0 worker=d4efe777
time="2024-05-09T18:54:12.611Z" level=info msg="Connected to session server http://192.168.65.3:32102" bundle_id=133 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 mode=shiny pid=0 worker=d4efe777
time="2024-05-09T18:54:12.623Z" level=info msg="Connecting to worker at 192.168.65.3:30378 ..." bundle_id=133 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 mode=shiny pid=0 worker=d4efe777
time="2024-05-09T18:54:13.914Z" level=info msg="Connected to worker at 192.168.65.3:30378" bundle_id=133 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 mode=shiny pid=0 worker=d4efe777
time="2024-05-09T18:54:56.493Z" level=info msg="Worker d4efe777 is not needed; 1 workers are servicing 0 connections" bundle_id=133 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 mode=shiny
time="2024-05-09T18:54:56.493Z" level=info msg="Stopping process for worker d4efe777" bundle_id=133 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 mode=shiny
time="2024-05-09T18:54:56.493Z" level=info msg="WorkerRetire {workerId:d4efe777}" bundle_id=133 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 mode=shiny
time="2024-05-09T18:54:56.510Z" level=debug msg="failed to cancel launcher job: cannot load job control response from https://connect:5559/v1/jobs/S3ViZXJuZXRlczpydW4tc2hpbnktYXBwbGljYXRpb24tdjZzNno=/control; response code: 409"
time="2024-05-09T18:54:58.026Z" level=info msg="Stopped session pings to http://192.168.65.3:32102" bundle_id=133 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 mode=shiny pid=0 worker=d4efe777
time="2024-05-09T18:54:58.027Z" level=info msg="Worker complete: exited normally with code 0" bundle_id=133 content_guid=1a63d3a7-c900-4a64-81dd-c1ab8d171d15 content_id=78 mode=shiny pid=0 worker=d4efe777
aronatkins commented 4 months ago

Do we expect writeManifest() to fail in the same way deployApp() does when a bogus image is passed? I am seeing it succeed:

rsconnect::writeManifest() does not perform a deployment nor interact with Connect; it does not know if the named image is correct.

aronatkins commented 4 months ago

With a manifest.json that was created while RSCONNECT_IMAGE="bogus", deployApp() succeeded:

rsconnect::deployApp() does not use a previously created manifest.

dethmasque commented 4 months ago

@aronatkins How can I validate that the manifest is aware of the supplied image name, or am I misunderstanding your comment and writeManifest() doesn't know about the image?

aronatkins commented 4 months ago

@dethmasque - writeManifest() follows your instructions and creates a manifest with whatever data you have provided. The target image is not used until the manifest is deployed. That can happen in a variety of ways, including a Git-backed deployment, the rsconnect deploy manifest CLI, or an API-driven deployment.

You can look at the manifest to confirm that the image data is present. That's probably sufficient, since you have already checked direct deployment of the manifest using deployApp().

dethmasque commented 4 months ago

Ahh I see what I missed in the manifest last week:

image

The lack of scroll position jumping (because bogus was in view) and minimal contrast between highlight and background made me think there were no results when searching for bogus. This work is validated.