Streamline package dependencies to reduce install time on CI

[annakrystalli]

Comment on draft validation CI demo:

small comment/question: might it be possible to have the hubValidations install require fewer dependencies? I note that the action runs fast, but the install of hubValidation takes a while (over 2.5 minutes) due to I think dependency installation. For a quick, frequently-run validation action, this feels maybe a bit longer than desired.

[gioco]

We are experiencing a similar issue with the time required to install the hubValidations package. It takes up to over 5 minutes in some cases to install, usually for the validation of a single file. I was wondering if it might be possible to pre-compile some objects, and as above if all unnecessary dependencies have been filtered out.

[LucieContamin]

A possible solution might be to cache the installation:

• https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows
• https://github.com/actions/cache

[annakrystalli]

Most relevenant r package caching GA action https://github.com/r-lib/actions/blob/v2/setup-r-dependencies/action.yaml

[annakrystalli]

The only complication, because it is designed for packages, it expects a DESCRIPTION file listing dependencies (hence it wasn't included in the first pass of the validation GA). Will need tinkering but should be a good source of ideas.

Using renv might also be an appropriate option for our use case https://rstudio.github.io/renv/articles/ci.html

[annakrystalli]

YES! Managed to implement caching by including the setup-r-dependecies workflow!

Been testing out in my fork of FluSight, using this workflow script: https://github.com/annakrystalli/FluSight-forecast-hub/actions/runs/7021327711/workflow?pr=4

The first time it runs, it's already a bit faster, under 4 mins: https://github.com/annakrystalli/FluSight-forecast-hub/actions/runs/7021198168/attempts/1

The first time the action succeeds, packages are cached.

The next time it ran, it completed in under 57s! https://github.com/annakrystalli/FluSight-forecast-hub/actions/runs/7021327711/job/19103122808?pr=4

[nickreich]

This is so great!

One question I have about it all is how one would add/update an existing validations workflow to incorporate this update. I think in the current framework the idea would be to just copy the updated action and include it in a workflow file? E.g. this file at the FluSight repo?

Related to a question @annakrystalli asked on Slack, Is there a way to create versions of the actions that can be referenced directly from within the workflows file?