Please add the following R packages to the runtime

[eli-asarian]

Per Jay Qi's instructions (https://community.drivendata.org/t/provisions-for-r-users/9476/6), to enable me to use the R packages that want for this contest, I am requesting that the following packages be added to the runtime from conda-forge: r-plyr, r-tidyverse, r-terra, r-sf, r-zoo, r-doparallel, r-foreach, r-iterators, r-nlme, r-mgcv, r-qgam, r-visreg, r-modelr, r-yardstick, r-missmda, r-mvtnorm, r-mice, r-factominer, r-lattice, r-matrix, r-shiny, r-rcpp, r-dataretrieval-feedstock

[jayqi]

Hi @eli-asarian,

I've added the following requested packages in this commit.

• r-plyr v1.8.9
• r-terra v1.7
• r-sf v1.0
• r-zoo v1.8
• r-doparallel v1.0.17
• r-foreach v1.5.2
• r-iterators v1.0.14
• r-nlme v3.1
• r-mgcv v1.9
• r-qgam v1.3.4
• r-modelr v0.1.11
• r-missmda v1.19
• r-mvtnorm v1.2
• r-mice v3.16.0
• r-factominer v2.9
• r-matrix v1.6
• r-rcpp v1.0.11
• r-dataretrieval v2.7.14

Regarding the remaining requested packages:

• r-tidyverse: Tidyverse is a metapackage containing a lot of subpackages. Can you please specify which subpackages you plan to use? There are many included in tidyverse (such as accessing Google Drive) which I don't believe are relevant to the parameters of the challenge.
• r-visreg, r-yardstick, r-latice, r-shiny: These appear to me to be packages related to evaluation and visualization. Can you please explain whether you need these to perform inference on the test set? The code execution image should be for generating predictions on the test set, and we ask you to do your training, development, and evaluation locally separately from the code execution runtime.

[eli-asarian]

Thanks @jayqi! Perhaps I was overly inclusive in my list of packages. On the other hand, I am concerned that many packages have dependencies on other packages and it is difficult for me to ascertain which of these dependencies are actually required to do what I want to do and which are not necessary (i.e., may be used for parts of the package that I will not be using in my solution). Responses to your specific questions:

• I had suggested tidyverse because I thought it would be easier to install than installing all its subpackages, and less likely to result in a missing dependency. The subpackages from tidyverse that I need are: r-dplyr, r-tidyr, r-lubridate, r-readr, r-stringr. However, those subpackages have a lot of dependencies and I'm not sure which of those dependencies are also needed (possible 10 to 20). That will take me some time to figure out. I will try to install the runntime, test it, and get back to you later today. I'm not used to trying to set up a development environment with the minimum tools necessary, prior to starting my analysis.
• r-yardstick and r-visreg are packages I would use for model diagnostics during model development, so I suppose do not have to be included in the runtime, I can just install them in my version during model development.
• shiny is listed as a dependency of qgam (https://cran.r-project.org/web/packages/qgam/). I'm unclear on if qgam will work without shiny.
• latice is a dependency of nlme (https://cran.r-project.org/web/packages/nlme/). nlme is a dependency of mgcv. I'm unclear on if mgcv will work without latice.

[jayqi]

Hi @eli-asarian,

You should only specify direct dependencies of your solution. You do not need to specify dependencies of those dependencies. As with any functional package manager (like R's install.packages), conda will by default figure out the necessary recursive dependencies and install them.

So for example, if you are not directly using shiny, then you should not list it. The package manager will figure out if shiny is necessary and install a proper version. (You can see that it is in fact included already in the environment because it is a subdependency.) Overspecifying unnecessary things makes the requirements more unwieldly and harder to correctly solve from possible overconstraints.

[eli-asarian]

Thanks for clarifying @jayqi. So then only packages that still need to be added are those listed in my second comment in this thread: r-dplyr, r-tidyr, r-lubridate, r-readr, r-stringr.

[jayqi]

@eli-asarian You should be all set as of this commit.

I am closing this issue to reflect that everything has been addressed. If you run into any issues, feel free to comment here and we can reopen the issue.

[iamo-lsg]

It turned that 'caret' in the code execution environment does not have some dependencies we rely on. Could you please add 'pls' and 'kernlab' packages.

[jayqi]

@iamo-lsg You should be all set: https://github.com/drivendataorg/water-supply-forecast-rodeo-runtime/commit/6ffa5db7af9fa6192e65ae119ebe4ed6f9453fc8

[iamo-lsg]

Hi @jayqi ,

Could you please add 'snotelr' package.

[jayqi]

Hi @iamo-lsg,

Can you please clarify why you need snotelr? Are you planning to use SNOTEL stations that we aren't pre-downloading?

[iamo-lsg]

Hi @iamo-lsg,

Can you please clarify why you need snotelr? Are you planning to use SNOTEL stations that we aren't pre-downloading?

Please disregard our request for adding the snotelr. We use inputs from snotelr, which turns to provide the data in a different format compared to the predownloaded data. We wanted to avoid additional data preprocessing, but now realize that using the updates from the competition folders might be a better solution, since there is always a possibility that APIs fail at some issue dates.