Notes from Emeli (Windows)

[ejanderson1]

Notes on using R Studio's terminal

To login to Emory HPC in terminal before typing in the ssh command you must type this code: set DISPLAY=. Then log in as you normally would. Note that this command must be typed every time you start a new R Studio session before you can log in to the HPC.

• See reference: https://stackoverflow.com/questions/59381260/how-can-i-ssh-login-from-rstudio-terminal

[ejanderson1]

Notes on step 2: Setting up your HPC environment

• Note: this code should be run outside of the HPC
• First generate the key using the following code: ssh-keygen -t rsa -b 4098
• Skip the passphrase as you would in the wiki instructions by pressing enter
• You will be prompted to enter a location for the key to be saved. Direct this to your .ssh folder on your computer. For example, my .ssh folder is here C:\Users\emeli\.ssh\
• Once your key is saved you need to copy the public version to the HPC using the following code: scp <path to ida_rsa.pub on your local computer> NETID@clogin01.sph.emory.edu:. (this will require you to enter your HPC password)
• Now log in to the HPC as you normally would. You need to type 2 more commands to finish the process of saving your public key to the HPC. First: cat id_rsa.pub >> .ssh/authorized_keys Second: chmod 0600 .ssh/authorized_keys
• Finally, exit the HPC and try logging in again to make sure it worked

[ejanderson1]

Using git pull

Git pull does not work in an interactive R session (i.e. when you are on a node to run code) unless you load git first.

• First run: spack load git@2.35.1
• Then: spack load r@4.1.2 (or whatever version you are using)
• Now you can start your interactive R session and pull projects as necessary from GitHub

To create an alias that does all 3 of the above steps use: alias lR='lspack; spack load git@2.35.1; spack load r@4.1.2; R

[ejanderson1]

Creating and using a shell script

• In R studio project: file --> new file --> shell script
• Save as .sh
• To run a batch job with files as described in example 1 from step 7 on the wiki type bash main.sh into the terminal

[ejanderson1]

Using renv with your project on the HPC

• To use renv the first thing you need to do is run export RENV_PATHS_ROOT="/projects/epimodel/renv/" in your HPC session
• Next, if you do not have your .Renviron file setup with your GitHub Private Access Token, you'll need to run export GITHUB_PAT="\<your github private access token\>" so that you can use code from private repos (like ArtNet)
• When you start your interactive session from your GitHub project directory on the HPC for the first time you will need to install renv using the usual code: install.packages("renv"). You will see the following prompt:

Warning in install.packages("renv") : 'lib = "/projects/epimodel/spack/opt/spack/linux-centos8-cascadelake/gcc-11.2.0/r-4.1.3-gpyu76r3akwrtruy2syewy526o6d7ghy/rlib/R/library"' is not writable Would you like to use a personal library instead? (yes/No/cancel)

Select yes. This will install renv in your personal library on the HPC. All the other packages you use will be in your renv library.

• Next run renv::activate to activate your project.
• Restart your interactive R session.
• Finally, run renv::restore to sync your library

REMINDER: you must run renv::restore() on HPC any time you make changes to your library locally and push those to GitHub

REMINDER 2: if you update your EpiModelHIV branch you have to renv::update() the package on the HPC as well

https://github.com/EpiModel/EmoryHPC/wiki/Using-Reproducible-R-Environments-with-the-%60renv%60-Package

[ejanderson1]

Steps to start an interactive R session

• Open R Studio and navigate to the terminal tab
• Sign in to the HPC (step 1 on wiki)
• Navigate to your project directory: cd /projects/epimodel/NETID/ProjectName
• Start an interactive session on the HPC with srun using: srun --cpus-per-task=32 -p epimodel --time=24:00:00 --mem=0 --pty /bin/bash
• Load spack and R
• Start R and copy/paste code that you want to run into the terminal

Note: any output you generate will default save to your project directory on the HPC

[ejanderson1]

slrum workflow

Link to vignette: https://github.com/EpiModel/EpiModelHPC/blob/main/vignettes/epimodelhiv-slurmworkflow.Rmd

[ynchen08]

Side note on the use of renv: I previously ran into an issue in my local machine where renv::snapshot() was not updating the list of packages in my lockfile (i.e., newly installed packages were not found in my lockfile). I found running renv::settings$snapshot.type("all") helpful in my case. However, please note that running "all" option could include potentially undesired development dependencies in the lockfile.

[AdrienLeGuillou]

@ynchen08 this is by design, renv::snapshot() does not save all the installed packaged by default but only the ones that are used by the project.

For instance, most of my projects have languageserver and lintr installed but this is for my text editor and not the project itself. What renv does is looking for library, require or package::function calls on the scripts to find out what packages are actually necessary for the project. renv::settings$snapshot.type("all") should be used parsimoniously as it can lead to weird behavior or at least an imperinflated list of packages to be installed

[ejanderson1]

Using EMHIV after making changes to your branch

Method 1:

1. Make your changes to EMHIV
2. Commit changes to GitHub
3. Open your project in R studio (if already open, restart your session)
4. Use renv::update() to pull the changes you made to EMHIV from GitHub
5. Restart your session
6. You can run your code now using the updated version of EMHIV
7. NOTE: you will need to run renv::snpshot() to record the updated EMHIV to your renv.lock file

Method 2 (for quick development & testing locally):

1. Make your changes to EMHIV
2. Open your project in R studio (or start a new session if already open)
3. Load EMHIV locally (i.e., from the location of you EMHIV folder on your laptop) by running the following code: pkgload::load_all(path/to/EpiModelHIV)
4. Run your code to test the EMHIV changes

Note from Sam For quick development and testing locally, I always use the latter approach. For longer-term and bigger changes, including those that have to be tested against local versus HPC setups, I use the first approach. Here is an example:

# pkgload::load_all("~/git/EpiModelCOVID")
control <- [control.net](http://control.net/)(nsteps = 100,
                       nsims = 1,
                       ncores = 1,
                       [initialize.FUN](http://initialize.fun/) = init_covid_corporate,
                       [aging.FUN](http://aging.fun/) = aging_covid,
                       [departures.FUN](http://departures.fun/) = deaths_covid_corporate,
                       [arrivals.FUN](http://arrivals.fun/) = arrival_covid_corporate,
                       [resim_nets.FUN](http://resim_nets.fun/) = resim_nets_covid_corporate,
                       [infection.FUN](http://infection.fun/) = infect_covid_corporate,
                       [recovery.FUN](http://recovery.fun/) = progress_covid,
                       [dx.FUN](http://dx.fun/) = dx_covid,
                       [vax.FUN](http://vax.fun/) = vax_covid,
                       [prevalence.FUN](http://prevalence.fun/) = prevalence_covid_corporate,
                       module.order = c("[aging.FUN](http://aging.fun/)",
                                        "[departures.FUN](http://departures.fun/)",
                                        "[arrivals.FUN](http://arrivals.fun/)",
                                        "[resim_nets.FUN](http://resim_nets.fun/)",
                                        "[infection.FUN](http://infection.fun/)",
                                        "[recovery.FUN](http://recovery.fun/)",
                                        "[dx.FUN](http://dx.fun/)",
                                        "[vax.FUN](http://vax.fun/)",
                                        "[prevalence.FUN](http://prevalence.fun/)"),
                       resimulate.network = TRUE,
                       tergmLite = TRUE)

sim <- netsim(est, param, init, control)

[smjenness]

Notes on the issue directly above added to a new wiki page here: https://github.com/EpiModel/EpiModeling/wiki/Writing-and-Debugging-EpiModel-Code

[smjenness]

All of the windows-specific comments have been incorporated into the Wiki. Thanks @ejanderson1!