during app deployment, force re-installation of specific package(s) ?

[tyner]

Is there an option users can set to disregard the cached version of an installed package and instead re-install it? The use case is for situations where a package may have been originally built in an incompatible environment. For example, below it is attempting to load a cached version of fastmap during the installation of htmltools, however I would like it to first re-install fastmap in order to iron out the GLIBCXX configuration.

* installing *source* package ‘htmltools’ ...
** package ‘htmltools’ successfully unpacked and MD5 sums checked
** using staged installation
** libs
gcc -std=gnu99 -I"/opt/R/3.6.3/lib/R/include" -DNDEBUG   -I/usr/local/include  -fpic  -g -O2  -c init.c -o init.o
gcc -std=gnu99 -I"/opt/R/3.6.3/lib/R/include" -DNDEBUG   -I/usr/local/include  -fpic  -g -O2  -c template.c -o template.o
gcc -std=gnu99 -shared -L/opt/R/3.6.3/lib/R/lib -L/usr/local/lib -o htmltools.so init.o template.o -L/opt/R/3.6.3/lib/R/lib -lR
installing to /opt/rstudio-connect/mnt/app/packrat/lib/x86_64-pc-linux-gnu/3.6.3/00LOCK-htmltools/00new/htmltools/libs
** R
** inst
** tests
** byte-compile and prepare package for lazy loading
Error in dyn.load(file, DLLpath = DLLpath, ...) : 
  unable to load shared object '/opt/rstudio-connect/mnt/packrat/3.6.3/v2/library/fastmap/32d3071f5657e52a92008e77660b45e4/fastmap/libs/fastmap.so':
  /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /opt/rstudio-connect/mnt/packrat/3.6.3/v2/library/fastmap/32d3071f5657e52a92008e77660b45e4/fastmap/libs/fastmap.so)
Calls: <Anonymous> ... asNamespace -> loadNamespace -> library.dynam -> dyn.load
Execution halted
ERROR: lazy loading failed for package ‘htmltools’
* removing ‘/opt/rstudio-connect/mnt/app/packrat/lib/x86_64-pc-linux-gnu/3.6.3/htmltools’

[aronatkins]

@tyner - This might be best addressed in a customer support ticket, as you may want to share details about your internal installation.

My guess is that at some point in the past, a binary version of the fast map package was installed. Unfortunately, that binary package was compiled to work with a different libc version (likely a different version of your Linux distribution). There are a number of ways this happens, mostly around mismatched repository URLs either on the server or defined in deployed content bundles. It can also happen if the compiler or other system libraries have been upgraded.

Unfortunately, we don't have a great way of telling what other packages are in a similar state.

Newer versions of Posit Connect do a better job of avoiding this problem by:

1. Automatically adjusting Posit Package Manager URLs to request binaries for the appropriate distribution.
2. Attempting to load packages before installing them, to avoid installing packages which are incompatible.

Most of these improvements came in Connect 2023.09.0 and 2023.10.0.

At this point, I think I would recommend upgrading Connect (to avoid getting into this situation again) and then clearing the runtime caches, at least for R-3.6.3.

Connect has APIs and tools to help your administrator selectively clear the runtime caches, and their use is described in the Connect Admin Guide: https://docs.posit.co/connect/admin/server-management/runtime-caches/

These tools are available starting with Connect 2023.05.0.

We can help you work through both the upgrade and clearing your runtime cache. If you file a support ticket and reference this GitHub issue, our support folks will loop me in to that conversation.

[tyner]

Thanks Aron. We are currently running version 2022.02.3 of Connect, but are planning to upgrade to 2023.09.0 soon. Sadly, Package Manager has not been enabled. In any case, clearing the runtime caches appears to require an API key for an administrator account. I was hoping for a solution for non-admins. After all, one does not need to be an admin to trigger package installation upon app deployment. I guess package re-installation is a different story since it could interfere with other users' apps.

[aronatkins]

@tyner - You could avoid this problem by installing a different version of fastmap into your R library, then redeploying. Unfortunately, there aren't a lot of fastmap versions to choose from! Additionally, htmltools has a version constraint, which limits your choices even further.

• https://github.com/r-lib/fastmap/blob/main/NEWS.md
• https://github.com/rstudio/htmltools/blob/main/DESCRIPTION

You are absolutely right: Clearing a runtime cache means that all content using that version of R or Python needs to be rebuilt. That's why Connect wants admin permissions.

Unless specifically configured, Connect tries to use the same package repository URLs that you are using in your development environment. If folks in your organization happened to use the Package Manager from a Linux development environment, there's a chance that those URLs reached Connect. Connect and tools like renv have evolved their support for Package Manger, but some of the URL rewriting that happens now hasn't always been available.

On a completely different note: Your 2022.02.3 Connect installation is beyond our 18 month support window. Definitely encourage folks to upgrade and reach out if anyone needs assistance with that process. https://posit.co/about/support-agreement/

[aronatkins]

I'm going to close this for now, as there's nothing immediate to change with rsconnect.

Your original suggestion about Connect having a way to bypass an unusable cached package is valid, and something we're constantly evaluating and tracking internally. We have primarily been focused on preventing that situation from happening, but that's not always enough. The tools we have built are fairly coarse, right now. That's sufficient in situations where lots of packages are invalid, but is too heavy-handed when the problem is narrow.

Please let us know if you hit this type of problem after upgrading and resetting the package cache, as we would like to hear of examples where that continues to happen.