search

coq / platform

Multi platform setup for Coq, Coq libraries and tools
Creative Commons Zero v1.0 Universal
190 stars 50 forks source link

Citing Platform releases #329

Open palmskog opened 1 year ago

palmskog commented 1 year ago

How can one cite Coq Platform releases in a stable way? (GitHub URLs might change.)

Could for example each release be uploaded to Zenodo, possibly automatically? This would give a stable DOI that can be cited.

Example for Coq:

MSoegtropIMC commented 1 year ago

I can certainly do this and it certainly would add to the reproducibility of research artefacts. I think one should have a DOI for the combination of release, coq version and package pick, but I would a dd a DOI only for the latest version/pick of each release (the others are intended for porting).

palmskog commented 1 year ago

Unfortunately Zenodo's DOI system is not all that flexible (one general DOI and one for each new upload). If you do a different upload for each package pick of a Platform release, this will quickly lead to a proliferation of "versions" and DOIs, e.g., Coq Platform 2023.03.0 will need something like 7 uploads/DOIs for all picks, 2023.09.0 will need 8, and so on.

Isn't it more clear and convenient to just live with one Zenodo version per 20XX.YY.Z?

MSoegtropIMC commented 1 year ago

That's what I meant - do exactly one Coq version + pick version per platform version. What I meant to say is that the DOI should not reference a Coq Platform release, but a specific combination of Coq Platform release, Coq release and pick. Otherwise the DOI would point to something arbitrary.

It will then be confusing though, that the pick and the platform release will usually be the same (say 2022.09). But some versions of Coq come with different picks in one version of Coq Platform (usually the second latest Coq release has the original pick and a pick adjusted to the pick of the latest version).

palmskog commented 1 year ago

But the "artifact" on Zenodo here would be the tarball/zip of the Platform repo, right? So what a version DOI points to is not arbitrary, but more like: "has the possibility to be ambiguous", since scripts in the archive can choose different Coq versions / package picks.

MSoegtropIMC commented 1 year ago

since scripts in the archive can choose different Coq versions / package picks.

yes - and this conflicts with the identifier of a unique identifier for a specific Coq Platform version.

In the end it should point to spcific installers and specific installation instructions for a specific version/pick.

MSoegtropIMC commented 1 year ago

@palmskog : do you have a solution? I still don't think it makes a lot of sense to reference a Coq Platform release without a pick. Can you imagine a good way to reference a pick?

One imaginable way would be that one creates files which download / install a specific coq platform pick in a cross platform way and reference this.

palmskog commented 1 year ago

@MSoegtropIMC I don't have any good solution. I still think you should upload "Platform release 20XX.YY.Z" as a single artifact in a repository like Zenodo, and then we can cite a specific pick inside this artifact, e.g., "[PLATFORM-RELEASE-20XX-YY-ZZ, 8.14 package pick]".

MSoegtropIMC commented 1 year ago

And this single artifact would be a source tar ball?

palmskog commented 1 year ago

Right. For example, you could use the one on GitHub (e.g., https://github.com/coq/platform/archive/refs/tags/2022.09.1.tar.gz) or generate one of your own tarballs from the Git tag. If I understand correctly, the source code would be enough to install the Platform manually, and in theory allow reproducing the installers. You can link to the GitHub release/tag on Zenodo for the artifact (to allow people to find ready-built installers). Here is an example we did recently that points to the GitHub tag: https://zenodo.org/record/6997534

MSoegtropIMC commented 1 year ago

Coq Platform depends heavily on Opam. It did happen that changes in Opam break existing Coq Platform versions. So for real reproducibility, we should create a tar ball which has all opam packages actually used in Coq Platform internalised (which might anyway be a good idea).

See e.g. https://github.com/coq/platform/issues/337

palmskog commented 1 year ago

Better reproducibility is really nice to have, but I don't think it should be a blocker for uploading Coq Platform releases to a service like Zenodo to make them citeable. You could just use the regular tarball for Zenodo for 2023.03.0 and then create the more complete tarball for 2023.03.1 or 2023.09.0 or later.

MSoegtropIMC commented 4 months ago

Triage note: need feedback from the Coq core team on this.

@Zimmi48 : since reproducibility of research artefacts is one of your topics, can you please comment?