Redo NEO pipeline - Githubissues

geneontology / project-management

Tracking project metadata in the GO as issues.

2 stars 0 forks source link

Redo NEO pipeline #52

Open kltm opened 2 years ago

kltm commented 2 years ago

Project link

https://github.com/orgs/geneontology/projects/TBD

Project description

Redo NEO pipeline: more entities, scalable, and folded into main pipelines.

PI

TBD

Project owner (PO)

TBD

Technical lead (TL)

TBD

Other personnel (OP)

TBD

Technical specs

TBD

Other comments

See https://github.com/geneontology/pipeline/issues/35

[ ] get tests in
[ ] auto-deploy to noctua-amigo
[ ] fold into main pipeline
[ ] https://github.com/geneontology/neo/issues/84
[ ] https://github.com/geneontology/neo/issues/58

pgaudet commented 2 years ago

Could we also add:

[ ] ad hoc addition of TrEMBL entries (see https://github.com/geneontology/noctua/issues/799)
[ ] more frequent NEO updates (which possibly includes automation of the process)

kltm commented 2 years ago

@pgaudet I'm not really sure of the scope of the first bullet. If it's another not-too-large list, it would be easy to add and could be bundled. If it was larger and approached scalability problems, it might be better added to an eventual (discussed elsewhere) rewrite of the very silly NEO infrastructure that we're using. The second bullet is more of a Noctua and outage question than a technical or software question--we can produce NEO loads however past we want, but we specifically cut back to meet the cadence of the Noctua outages to keep messaging clear (tagging @vanaukenk). I was imagining this pre-project, lifted from a larger "super" ticket from a while ago, as more about cutting back on the proliferation of independent pipelines and the associated overhead.