Arrow R package missing system dependencies (libarrow)

[JimMadge]

:white_check_mark: Checklist

• [x] I have searched open and closed issues for duplicates.
• [x] This is a problem observed when using a Data Safe Haven.
• [x] I can reproduce this with the latest version.
• [x] I have read through the documentation.
• [x] This isn't an open-ended question (open a discussion if it is).

:computer: System information

• Data Safe Haven version: 4.0.3
• Browser details: n/a
• Operating system details: n/a

:no_entry_sign: Describe the problem

Discussion in: #1388

The Arrow R package, in the extras allowlist fails to install as building it depends on the libarrow (and other?) libraries. (CRAN packages for Linux are provided as source, not binary).

If we want to support the Arrow R package, we will need to add those system libraries to the SRD image so that it can be built. (I'm uncertain whether libarrow is also a runtime dependency, but it seems it isn't as RStudio provides a Linux binary statically linked to libarrow)

:steam_locomotive: Workarounds or solutions

We implemented a workaround that can be found here: https://github.com/alan-turing-institute/data-safe-haven/issues/1388#issuecomment-1514387230

[jemrobinson]

Which of the libraries listed on #1388 are actually needed to build arrow locally? See list below.

- libarrow-dev # For C++
- libarrow-glib-dev # For GLib (C)
- libarrow-dataset-dev # For Apache Arrow Dataset C++
- libarrow-dataset-glib-dev # For Apache Arrow Dataset GLib (C)
- libarrow-flight-dev # For Apache Arrow Flight C++
- libarrow-flight-glib-dev # For Apache Arrow Flight GLib (C)

If we can determine which are relevant, can we add them to our SRD build image?

[craddm]

A thought - when installing arrow normally in R, on a machine with internet access, it automatically downloads the required system libraries.

If arrow is added to the CRAN core list (i.e. installed in the image by default), would that take care of this for us by downloading all these when the image is built? If so, is there a reason not to add it to the core list and to add the libraries separately instead?

A problem that might come up is that arrow needs to be linked against the same version of the C++ libraries. So if the arrow package gets updated (e.g. from its current 12.0.1 to 13.0.1) it may not install correctly for the user until the system libraries also get updated.

[jemrobinson]

I think it's fine to add it to the core list and also to add the libraries to the core image.

[craddm]

Resolved in #1586