search

CodeForPittsburgh / food-access-map-data

Data for the food access map
MIT License
8 stars 18 forks source link

Speed up Github Actions? #74

Closed maxachis closed 3 years ago

maxachis commented 3 years ago

Our Github actions currently install all dependencies every time they run. This slows down the workflow, and currently occupies the majority of the time within the workflow. Can we speed up the process, such as by caching dependencies, so that this takes less time?

This has been done for some workflows, like with npm. Can we do the same for R and Python?

@actions/cache

Github Docs on Caching Dependencies to Speed Up Workflows

maxachis commented 3 years ago

I modified, via #75, the rtest.yaml package so that it uses both cores of the Github Action Runner (a two-core virtual machine) when installing, cutting installation time in half, so that it now takes around three minutes.

Additional work that we could do includes installing pre-built Linux binaries.

https://www.jumpingrivers.com/blog/faster-r-package-installation-rstudio/

https://community.rstudio.com/t/faster-package-installs-on-linux-with-package-manager-beta-release/39607

Main challenge is that commands to install binaries may be a bit more brittle -- I'm not sure if the same binaries will work if the runners update to a different Linux version, for example.

maxachis commented 3 years ago

Closing this with #76, which changed the installation process to install package binaries instead of ASCII-format files. Now the whole process takes ~2 minutes! I am a proud parent today.