Manubot for awesome lists

[agitter]

In the meta review, we discussed how GitHub is used for collaborative curation. One example is using GitHub readme files to curate a list of topic-specific manuscripts as an "awesome list". Typically the reference metadata is added manually, making maintenance harder than it needs to be.

Is there are good way to use Manubot to help with this? In the short term, one could make a manuscript that has sections for different sub-topics. Each section would have an unordered list (or table) that contains manuscript titles and citations. Is that the best option?

In the longer term, if manubot cite could produce a csv file with metadata fields as columns (title, authors, URL, etc.) then perhaps something like https://github.com/derekeder/csv-to-html-table could be used to create a sortable list of references. Ideally, it would also be possible to add extra columns to that table to annotate the references. That would require much more work though.

[dhimmel]

In the short term, one could make a manuscript that has sections for different sub-topics. Each section would have an unordered list (or table) that contains manuscript titles and citations. Is that the best option?

I think that is the best option.

In https://github.com/manubot/rootstock/issues/215, we briefly touch on interactive tables. Would the current Manubot design with the addition of a frontend plugin for interactive tables address the need here? The interactive tables could be sortable, searchable, etcetera.

[agitter]

The tables in #215 would be great for the front end.

I've been thinking about this more, and the real functionality needed to support what I had in mind might be more direct access to the information in references.json. Currently, I would create a list or table in the markdown file as:

- Paper title 1 [@doi:1]
- Paper title 2 [@doi:2]
- etc.

The paper title and perhaps other information like the date have to be manually specified next to the citation.

It would be cool to be able to template more of this

{% for ref in references %}
- {{ref.title}} {{ref.issued}} [@{{ref.id}}]

That is fictional syntax but hopefully conveys the idea about accessing citation components in templates. I'm not yet sure where the standard citations would come from if they aren't in the markdown. In this scenario, they might be in an separate list of references and annotations.

Would that be useful anywhere outside of this specific niche use case?

[slochower]

That is fictional syntax but hopefully conveys the idea about accessing citation components in templates. I'm not yet sure where the standard citations would come from if they aren't in the markdown. In this scenario, they might be in an separate list of references and annotations.

This is a good point. I wonder if you could associate different tags with each citation and then have tables templated like: citations with tag 1, citations with tag 2, etc. Then having the tables be interactive & collapsible would be a bonus. This might be out of scope of Manubot proper, but I definitely see the utility in this type of work. (Just think about how much easier review papers would be! Or collecting and sorting through related research! Or amassing a series of citations, going backwards in time, to figure out where ideas were first published!) In my mind, I imagine the standard citations would have to be specified in some other syntax--YAML(?)--with tags. That could get ugly without precedent.

[agitter]

I imagine the standard citations would have to be specified in some other syntax--YAML(?)--with tags.

Yes, I was thinking similarly and debating YAML versus tsv. We had similar discussions about manuscript metadata and decided on YAML for the metadata file. However, here I see some appeal to having a simple two column tsv file (like citation-tags.tsv) with a citation column and an annotation column.

For a specific example of what initiated this issue, see https://github.com/yangkky/Machine-learning-for-proteins. What @slochower refers to as tags above would be the categories in this list. I've been calling these "annotations" because we already use "tags" in the citation tag sense. "Categories" would also make sense.

This might be out of scope of Manubot proper

I agree with this as well. Even if this isn't supported by the rootstock, it could still be set up as another downstream use case of Manubot. I linked to https://github.com/derekeder/csv-to-html-table before with the idea that Manubot plus another simple script could generate and push csv files to gh-pages with continuous integration. A front end would display the tabular data in a nice way.

[slochower]

Yes, I was thinking similarly and debating YAML versus tsv. We had similar discussions about manuscript metadata and decided on YAML for the metadata file. However, here I see some appeal to having a simple two column tsv file (like citation-tags.tsv) with a citation column and an annotation column.

The simplicity is probably worthwhile.

I agree with this as well. Even if this isn't supported by the rootstock, it could still be set up as another downstream use case of Manubot. I linked to https://github.com/derekeder/csv-to-html-table before with the idea that Manubot plus another simple script could generate and push csv files to gh-pages with continuous integration. A front end would display the tabular data in a nice way.

Definitely agree. This could be bundled with other improvements on table display: sorting, downloading as CSV/TSV, and maybe pagination (like @dhimmel's blog tables) would be nice.

[dhimmel]

That is fictional syntax but hopefully conveys the idea about accessing citation components in templates.

I think we could make references available as metadata / variables like this. Will keep in mind when continuing work on https://github.com/manubot/manubot/pull/99.

[agitter]

I put together a quick proof of concept to see how manubot cite could be used for this outside of a full manuscript. Awesome lists typically list their resources in the main GitHub README. I wanted to generate the README by using Manubot to append formatted lists of references to a base Markdown file.

The first draft at https://github.com/agitter/manubot-awesome-list works. It is currently very fragile and would likely lead to many merge conflicts in practice because everything is done on master.