Reintegrate with distill package

mitchelloharawild commented 2 years ago

This hard fork is motivated by the need for some structural changes needed in the distill package. With support from the RStudio distill developers, it should be possible to generalise the distill package in a way that allows The R Journal to extend it rather than fork it.

This issue summarises the changes made in rjdistill which would need to (or should) be generalised in distill.

Arbitrary fields in article's d-byline metadata.
Add more optional date fields (submitted, modified). The existing date field can continue to satisfy the 'published' entry. (#1)
Parameterise scope of search utility. One idea that I was planning to implement was the inclusion of search in listing pages. (#3)
Arbitrary appendices (to contain custom entries for things like CRAN/Bioc packages used, supplementary files for the article, renv output). (#7)
Allow badges for article types in listings (#4)
~~Option for seal=FALSE to remove generated titles from info pages?~~ It is probably better to use a different output for this (like how postcards is used).

Changes that would still occur in rjdistill/rjtools as an extension of distill:

Dual rendering of distill HTML and rjournal PDF output (#2)
Mapping of YAML variable names from rticles::rjournal_article() to distill::distill_article(). For example, orcid_id -> orcid.
Generation of article DESCRIPTION
Generation of article supplementary resources archive (files specified YAML)
Handling of RJournal article package macros \pkg{*}, \CRANpkg{*}, \BIOpkg{*}

mitchelloharawild commented 2 years ago

Hi @yihui, thanks for chatting about these changes with us. An ordering of priority for changes needed in R Journal articles to use distill are:

Arbitrary fields in article's d-byline. We would like to use this section to link articles to their respective issue. It can also be used to provide additional date fields, however specific support for additional date fields would still be useful (but less necessary) in article metadata - for example the Google Scholar's online date may differ from the publication date: https://github.com/rjournal/rjdistill/blob/6392af7a62b4893e4df96fdf2d17af4106727630/R/metadata.R#L481-L482
Arbitrary appendices. We would like to achieve information parity between the original landing page (https://journal.r-project.org/archive/2021/RJ-2021-050/) and the distill article (https://journal.r-project.org/dev/articles/RJ-2021-050/). Things that are currently missing (CRAN/Bioc packages, task views, supplementary material) can be put in sections of the appendix.

The other items are more of a wishlist.

Parameterised search scope (or some guidance / extensibility support for an extension package to use the search utility). Instead of a global search in the navbar, we would prefer search functionality in the listings page to search through items in that given listing.
Article badging. Much like how listings in https://distill.pub/ have a small badge for identifying 'peer-reviewed' and 'editorial' articles, something similar would be useful for for R Journal articles.
More date fields for articles (as mentioned for article indexing via metadata). For purposes of The R Journal we would like to be able to specify a submitted date, published date, and modified/updated date.

yihui commented 2 years ago

Thanks @mitchelloharawild! I'll defer the response to other items to @cderv. For arbitrary appendices, does the .appendix class work? https://rstudio.github.io/distill/basics.html#appendices Or are you looking for a way to specify appendices in the YAML metadata instead of the end of the article body?

mitchelloharawild commented 2 years ago

Appendices would be added by the output format, possibly based on the YAML (CRAN / Bioc packages and task views): https://github.com/rjournal/rjournal.github.io/blob/8038aab22b6211488e388602e19ada47f03d2b7d/_articles/RJ-2021-050/RJ-2021-050.Rmd#L31-L56

Or possibly without a YAML entry (for example, renv.lock).

Maybe it's possible for the output format to add contents to the document's body to provide arbitrary appendices, but it'd be nicer if there was a simpler method.

yihui commented 2 years ago

Okay got it. Yes, I agree that the YAML approach (or renv.lock) is a lot simpler and preferable.

cderv commented 2 years ago

Hi @mitchelloharawild !

Know that I will look at the two main points describe in https://github.com/rjournal/rjdistill/issues/9#issuecomment-929676795 tomorrow.

Hopefully we'll get something working for you for next week. I'll ping you in the relevant PR for you to test and adapt in here in a companion PR.

does that sound ok to you ?

We could try to find a time to meet also if you want us to discuss the implementation. That would be morning for me and probably evening for you. (I am in France)

cderv commented 2 years ago

Hi @mitchelloharawild , I worked this by working on an example for a custom format that would do its best not to change distill. It is a first try so that we can identify what and how move some stuff into distill.

Example repo is here: https://github.com/cderv/rjdistill

Most of the tweaks happens in the output format definition. Then I do a lot using JS directly. Specifically for some parts hard to tweak. I am having a result close to what you are looking I believe.

I need to show that to you, and test it in a website. I only tested the article format on a single doc.

Some insights on what we could do or not in distill after that to try generalize. I'll update this when finding new ones.

Google scholar date

https://github.com/rjournal/rjdistill/blob/6392af7a62b4893e4df96fdf2d17af4106727630/R/metadata.R#L481-L482

Maybe we need two fields in the YAML :

date: 
date_online:

We could map to both google scholar dates if date_online is not set then both would be the same date. But this would allow to tweak

Tags on Listing page

Like in https://distill.pub/

Currently categories are used to create tags on listing page; Ex: https://blogs.rstudio.com/ai/

We would need to a second set of tags shown on listing page. Maybe a field like

type:
 - tag 1
 - tag 2

or more precisely

listing_categories:
  - tag 1
  - tag 2

If defined it would be shown on listing page. We could also use listing: fields on post

listing:
  tags:
    - tag 1
    - tag 2

Custom appendix

The current generic behavior works quite well and can be used in a custom formats. https://rstudio.github.io/distill/index.html#appendices

I used it in a pre_processor step in the output format to append custom markdown content to the intermediary file. See the demo package;

custom d-byline

Adding arbitrary fields in by-line is not as easy as I thought. This is hard to make generic because of the CSS grid and how it set currently (by the code).

I think we could make a similar system as appendices by having some content like this

### My new byline header {.byline}

Content

which would be moved to byline part.

By I recognize this could be weird to write in the document. 🤔 Maybe it could be set in YAML ?

byline:
  - name: New header
    content: value

but really the styling will be off and this would require a custom CSS anyway. And even JS to change the order if necessary.

Abstract

Do we really need to add a abstract ? Isn't the description field enough ?

description:
abstract:

We could easily add this new part with <d-abstract> but it would duplicate description. What is the difference for you ?

rjournal / rjdistill