Create community on Zenodo to list research compendia

[nuest]

Just like the GitHub tag (see #7) we could create a community on Zenodo to collect research compendia published on Zenodo.

Help needed

• [x] identifier
  • research-compendium :1st_place_medal:
  • reproducible-paper
• [x] title
• [x] description
• [x] curation policy
  • require a corresponding journal submission?
  • require data + paper + code?
  • must have clear licenses
• [x] logo

Required information

[cboettig]

:+1: Creating GitHub topic and a Zenodo community for compendia is a brilliant idea.

I'm not good at naming things, but I'd go with research-compendium over reproducible-paper, as the latter could be confused with the notion that someone has reproduced the paper (a la recently reproducibility studies), and does not necessarily imply that the paper includes additional content (code + data). Of course, compendium is a considerably more niche concept that most people may not recognize, but in some ways that is a strength as it's a relatively unique term that is thus more likely to mean what we want it to mean...

[noamross]

I agree with research-compendium, but we probably need a pretty robust guidance/reference in the description. There's https://github.com/ropensci/rrrpkg, but that is an R-specific description. Is there a resource that describes research compendia in a more tool-agnostic way?

[nuest]

@noamross I am not aware of any other description than the one you posted. Could the Zenodo community be a place to publish one, as part of the curation policy?

I went ahead and created the community, just to "save" the identifier (but nothing more): https://zenodo.org/communities/research-compendium/

While this technically makes me the curator :blush: BUT I don't think I can get this to work alone... so here is my first draft for the required texts, edits welcome: https://epad.ifgi.de/p/zenodo-community-research-compendium

As a sidenote, on Github the "topic" compendium seems to be used by a few, but if we could convince @benmarwick to change to research-compendium we would already have most of it :-).

[cboettig]

@nuest looks like you created the community in the sandbox? was that intentional or do we want to create https://zenodo.org/communities/research-compendium ?

[benmarwick]

We've expanded on the https://github.com/ropensci/rrrpkg essay in our recent AmStat article (& OA preprint), which although R-focused, also has some generic principles for organising a compendium that should work for any language. That's the simplest account that I know of, other generic treatments are more elaborate and suit a more computationally-literature reader,, such as Stodden and Miguez and Wilson et al.

research-compendium sounds like a great label to me, I've just now updated my GitHub repos to use this also. If you can start the Zenodo community in the main namespace at https://zenodo.org/communities/research-compendium I'll help to populate it also. Happy to help with curation when they get multiple curators enabled!

[nuest]

My bad, did not check the URL properly, here it is in the non-sandbox: https://zenodo.org/communities/research-compendium

@benmarwick Thanks for the pointers. I know all of the mentioned articles but will re-check from the perspective of defining a research compendium.

I'll work on an update for the manuscript tomorrow and send it around for feedback.

[cboettig]

@benmarwick stupid question here - how do I add an existing Zenodo project to the research-compendium community?

[benmarwick]

This is what I'm doing:

1. Go to my uploads page, where I can see items I've previously deposited: https://zenodo.org/deposit?page=1&size=20

2. Choose one, click on it, then click 'edit'

3. Scroll down to 'communities' and type "research compendium", it should autocomplete, and then click on that

Once it's added:

4. Scroll to the very top or bottom and click 'save' then 'publish', and watch for the confirmation

5. Wait for Daniel to approve it? I'm not sure what the curation policy is, but if it's similar to https://zenodo.org/communities/archaeology/ that I curate, I get an email when someone wants their deposit to be listed in the archaeology community, and I have to approve each item before it appears in the community. It's usually not more than one item per month, more if @ekansa is archiving stuff from Open Context. But it's not onerous, usually it's very quick to determine if it's a good fit.

[nuest]

I actually got a few emails :-) Thank you! :bouquet:

I'll briefly evaluated them following the draft for a curation policy: https://epad.ifgi.de/p/zenodo-community-research-compendium I personally want the collection to be open/permissive, but also make sure there is "enough compendium"

@benmarwick @cboettig Since you are the first to be listed in the community, would you be so kind to review the curation policy? The etherpad also includes my notes on the first candidates.

[cboettig]

Thanks @benmarwick , great instructions! Maybe we can include those somewhere too.

@nuest curation policy looks good to me; I think the basic criteria you list there are spot-on. Might be worth referring to @noamross et al's work on a research-compendium reviewer guide and the checkers package?

[nuest]

I published the policy and other information on Zenodo: https://zenodo.org/communities/research-compendium

Please let me know if you're fine with being listed on https://zenodo.org/communities/research-compendium/about as community curators @noamross @cboettig @benmarwick - In case of doubt it would be great if I could get back to you.

Remaining open item: A logo :framed_picture: - anyone good at that?

[cboettig]

Please let me know if you're fine with being listed on https://zenodo.org/communities/research-compendium/about as community curators

👍

[benmarwick]

The curation policy looks really solid, great work! I've updated my other repos, so I think they should qualify now.

I hadn't noticed the "Related identifiers" before, and I think that is a great requirement to help create rich metadata about these deposits. I like the 'cited by' field, that will help to normalise data and code as a first class research product. I'm happy to be listed as a community curator, thanks.

Perhaps we could merge the first two criteria, from this:

The research compendium must be associated with a manuscript or talk slides published with a globally unique identifier (i.e. a DOI). This can be a journal, pre-print platform, file repository, or the manuscript may be included in the compendium directly.

If an external document exists, it must be referenced under the "Related identifiers" in the deposit metadata with one of these relations: "is supplemented by this upload", "is identical to this upload", "has this upload as part", "is part of this upload", "documents this upload", or "is compiled/created by this upload".

into this:

The research compendium must be associated with a manuscript or talk slides published with a globally unique identifier (i.e. a DOI). This can be a journal, pre-print platform, file repository, or the manuscript may be included in the compendium directly. This external document must cite the compendium by its DOI, and be referenced under the "Related identifiers" section in the Zenodo deposit metadata with the relation: "is cited by"

This will nudge people close the loop and cite their data in the paper, and their cite paper in their data.

Another thought, can I suggest that we move this text from the bottom to the top of the about page?:

Preparing a compendium for submission

The curation policy only requires the bare minimum. If you want to create excellent research compendia to boost your research and collaborations, either with others or your future self, we recommend diving into these resources:  

Packaging data analytical work reproducibly using R (and friends)
rOpenSci Analysis Best Practice Guidelines
"How to read a research compendium"
DataONE Reproducible Research Compendia Onboarding

That will help to communicate the expectations for the community, since the full list of criteria is a little hidden in the sidebar. Perhaps we could edit the first sentence like "The curation policy is described in the sidebar to the right of this page, it only requires the bare minimum." I think that is important to help people browsing find the info they need, and should save us from having to reject many unqualified submissions because people couldn't see the criteria.

Also, I just remembered this chapter Assessing Reproducibility which is good because it is totally language agnostic. Perhaps it could be added to the resource list?

[nuest]

@benmarwick @cboettig Thanks for the :+1: on the curating

@benmarwick I updated the first part a little bit: I now ask for two relations, the "cited by" you suggest (great idea to require people to cite their compendium!) and recommend an additional semantically more meaningful relation ("is supplemented by this upload" or "is compiled by this upload"). The about page is updated as you suggest, and the chapter added.

[nuest]