Vignettes

[nevrome]

Hey @bodkan - I started to look at slendr and had an issue when installing with the vignettes via remotes::install_github("bodkan/slendr", build_vignettes = T, force = T)

E  creating vignettes (10m 41.2s)
   --- re-building ‘vignette-01-tutorial.Rmd’ using rmarkdown
   ℹ Loading slendr
   Quitting from lines 103-109 (vignette-01-tutorial.Rmd) 
   Error: processing vignette 'vignette-01-tutorial.Rmd' failed with diagnostics:
   the file ne_110m_land seems not to exist in your local folder ~/Documents/postdoc/data/ne_data/
   Did you download it using ne_download()?
   --- failed re-building ‘vignette-01-tutorial.Rmd’

   --- re-building ‘vignette-02-grid-model.Rmd’ using rmarkdown
   ℹ Loading slendr
   Quitting from lines 172-184 (vignette-02-grid-model.Rmd) 
   Error: processing vignette 'vignette-02-grid-model.Rmd' failed with diagnostics:
   the file ne_110m_land seems not to exist in your local folder ~/Documents/postdoc/data/ne_data/
   Did you download it using ne_download()?
   --- failed re-building ‘vignette-02-grid-model.Rmd’

This should be an easy fix. But a vignette that runs for 10+ mins might generally be problematic. Especially when the links to the pre-rendered ones on the website are broken :sweat_smile: (e.g. https://bodkan.net/slendr/articles/articles/vignette-01-tutorial.html)

[bodkan]

Hello, thanks for taking a look at this silly little package of mine! 👋

Sorry about the vignette building problem. To be completely honest (and risking revealing how little I know about the build process here), I have only been building vignettes for the pkgdown website. I have never used explicit build_vignettes = TRUE in install_github() in practice (in fact, I don't think I have ever read a locally generated vignette for an R package). The Natural Earth project servers where I'm fetching cartographic data from are unresponsive occasionally and I'm now often in a place with dysfunctional internet connection, which is why I've been building the pkgdown website using N.E. data downloaded locally. What I currently have is clearly a bad solution though, so I will look for a better way to do this. Right now I'm fetching the data from my local cache using code in a hidden chunk, but presenting a normal code chunk to the reader on the website.

I will fix the broken links ASAP. I added a couple of new vignettes this week and rearranged the files and I clearly didn't update the links.

Not much I can do about the run times, I'm afraid, because some of those vignettes are running a full SLiM simulation to generate the animated data. Not looking forward to the utter nightmare that is CRAN submission, that's for sure...

[bodkan]

A couple of notes as I’m reading about the topic to finally learn how this works and how to do this properly.

A hack on “How to precompute package vignettes or pkgdown articles”.

From R packages book:

Note that since you build vignettes locally, CRAN only receives the html/pdf and the source code. However, CRAN does not re-build the vignette. It only checks that the code is runnable (by running it).

Precomputing vignettes seems like cheating, but given that it appears to be blessed by rOpenSci, I will go with this as this would be necessary to do for CRAN submission anyway.

[bodkan]

UPDATE: I'm currently in the middle of implementing a rather big feature (scheduling of tree-sequence recording events and some related functionality) which involves adding new vignettes which are even more computationally intensive. I will tackle this issue in one go once I push those changes to the repo.

[nevrome]

This is a good solution to the vignette rendering time issue. I solved this in the past with a script that runs the code independently and generates .RData files and figures in inst/, to be then simply embedded into the vignette. Rendering to .Rmd is more clever.

[bodkan]

So, after starting, stopping, and re-starting the work on pre-rendered vignettes again and again over the last couple of months, I decided to drop the idea of pre-rendering altogether. I might revisit this later once the package is on CRAN.

My reasoning, plus couple of related points for future reference:

• slendr has hundreds of comprehensive unit tests but having the vignette rendering out-of-sync with compiling code in them could do more harm than good. In a sense, I consider the vignette code even better "unit tests" than many of the tests themselves, because they are demonstrating real analyses on real use-cases. Making sure they really work on each CRAN check (all of them done locally, those that can do without SLiM & Python done in CRAN + Win-builder + R-hub) is really valuable. Adding an additional pre-rendering step seems like introducing a potential point of failure getting the codebase and all vignettes out-of-sync which is not worth it.

The above will be even more true now that I'm finally setting up GitHub Actions CI which will include full SLiM and msprime runs, and will also test reticulate-d Python tree sequence analyses.

• In context of this GitHub issue, the hard-coded paths from the first development version that triggered this issue should be removed (a long time ago actually).

CRAN submission is around the corner, I just went through check submissions for Windows/Linux/macOS on Win-builder and R-hub and managed to fix all ERRORs/WARNINGs/NOTEs discovered in the process.

[nevrome]

Sounds good to me :+1: