Pipeline stabilization

[kltm]

Project link

https://github.com/orgs/geneontology/projects/149

Project description

The purpose of this project is to increase the number of releases (snapshot and release) made and decrease the amount of time per release attempt. While a complete pipeline refactor may be in the cards in the future, this project is aimed at, more or less, "working with what we have".

To this end, the focus will be:

• buy new machine, to:
  • try and ameliorate https://github.com/geneontology/pipeline/issues/316
  • increase overall throughput
• look at blessing snapshots to release, to:
  • help with things like https://github.com/geneontology/pipeline/issues/349
  • prevent conflicting run timings when snpashot or release not going well
• drop Solr indices from the Zenodo archive bundle, to:
  • make the final stages a little more "stable"
  • help deal with recent Zenodo changes https://github.com/geneontology/pipeline/issues/345

PI

Chris

Product owner (PO)

Pascale

Technical lead (TL)

Seth

Other personnel (OP)

TBD

Technical specs

No new libraries or technologies. The only "interesting" additions would likely be:

• an extension of the current Zenodo scripts to assemble an upload package from a given snapshot and push it
• snapshots to get an associated date and location
• figure out what to do about testing

Other comments

N/A

[kltm]

@pgaudet I have a note that says that this can be considered under way (lit. "move to working"); does that sound right to you?

[kltm]

@pgaudet confirms above.