Roll back (and then forward) "problematic" upstream files from selected resources

geneontology / pipeline

Declarative pipeline for the Gene Ontology.

https://build.geneontology.org/job/geneontology/job/pipeline/

BSD 3-Clause "New" or "Revised" License

5 stars 5 forks source link

Roll back (and then forward) "problematic" upstream files from selected resources #373

Closed kltm closed 2 months ago

kltm commented 5 months ago

We have four files that we want to roll back to the last "good" source, as represented in the last release.

[x] tair: many issues, should be fixed week of May 1st
[x] mgi (see https://github.com/geneontology/pipeline/issues/372#issuecomment-2045194118)
[x] dicty: bad dictybase-unirpt mapping
[x] pombase: to check
[x] wormbase [EDIT: added]

Fastest path to get out fix for https://github.com/geneontology/amigo/issues/716

Tagging @pgaudet @dustine32

kltm commented 5 months ago

Blocked by https://github.com/geneontology/pipeline/issues/374 for the moment.

kltm commented 5 months ago

snapshot started.

kltm commented 5 months ago

I have confirmed with @vanaukenk that the FTP (and downloads HTTP) are down for WB. Adding them to the list above.

kltm commented 5 months ago

Talking to @pgaudet last week, we'll be holding like this while she researches the issues (delta WB).

kltm commented 5 months ago

Noting that WB FTP still seems to be out of action. @vanaukenk, could I ask you to update here if that status changes?

kltm commented 5 months ago

Note from @vanaukenk that WB has recovered.

kltm commented 5 months ago

https://github.com/geneontology/go-site/pull/2302

kltm commented 5 months ago

@vanaukenk WB failing again:

01:10:56  Download of ftp://ftp.wormbase.org/pub/wormbase/species/c_elegans/PRJNA13758/annotation/gene_association/c_elegans.PRJNA13758.current.gene_association.wb.gz failed:  - Trying again

Reverting to our "safe" version.

kltm commented 5 months ago

Ah, I missed that the GPI is grabbed later for (the now unavailable) WB GPI. Reverting that as well.

vanaukenk commented 5 months ago

@kltm - I just tried the WB ftp site again and can manually download all of our files in the wb.yaml, so I'm not sure what's going on and if this is an intermittent thing.

If you want to look at this together in case I'm missing something, let me know.

kltm commented 5 months ago

@vanaukenk I can confirm that it's still not working for me and the pipeline machine, but I cannot confirm if there are firewall rules at my end or your end (yet). I'm happy to explore this with you at some point; hopefully we can deal with it at our leisure as your release frequency is low?

vanaukenk commented 5 months ago

Okay, thanks @kltm and very sorry for the continued trouble with the WB files.

Let me know when you'd like to meet and I could try to wrangle one of the WB developers, too, as that might also be a help.

kltm commented 2 months ago

I think MGI is good because we are now testing with our "internal" "upstream" source.