[manualpipe] Asklepian 20210810

[BioWilko]

Brief description

See https://github.com/COG-UK/dipi-group/issues/121#issuecomment-896080983 by @SamStudio8 for background and explanation.

---

Executed by: @BioWilko - BIRM

• [x] I am sure this pipeline is not already running

---

• [ ] Reconcile input
• [x] Engage pipeline from a clean environment
• [ ] Disable NXF cache destruction (if applicable)
• [ ] Reconcile output
• [ ] Check (or send) mqtt message (if applicable)
• [ ] Re-enable cache destruction (if disabled)

---

• [x] Manual intervention complete
• "Normal" service disrupted? NO
• If yes, describe impact:

[SamStudio8]

Investigating an issue with generating the Asklepian manifest.

[SamStudio8]

Seems to be an integrity error as it should be impossible for a PAG to be missing a published date!

File "/var/www/majora/majora2/tasks.py", line 273, in task_get_pag_v2
    pags[pag]["published_date"] = pags[pag]["published_date"].strftime("%Y-%m-%d")
AttributeError: 'NoneType' object has no attribute 'strftime'

There is indeed one such case:

>>> from majora2 import models
>>> models.PublishedArtifactGroup.objects.filter(published_date__isnull=True)
<PolymorphicQuerySet [<PublishedArtifactGroup: COG-UK/MILK-1AE4D95/SANG:210807_A00518_0179_AHGYK7DRXY (591c342c-7063-4a58-9b88-cb1596ae7ec9)>]>

[SamStudio8]

Quite possible a race condition wherein a PAG was created but failed to reach this line.

[SamStudio8]

PAGs are created as a side effect of the api.artifact.file.add endpoint when --publish-group is set. Two files are added for each PAG (the FASTA and BAM) in quick succession. The conclusion I'll make is that in the case of this PAG, both calls were involved in an incredibly narrow window write skew:

• Call 1 created and saved the PAG with get_or_create
• Call 2 fetched the new PAG via get_or_create
• Call 1 assigned the published_date as pag_created was True
• Call 2 did not assign the published_date (as pag_created would be False)
• Call 1 saved its object
• Call 2 saved its object (without the published_date set)

I'm happy enough with this to manually set the correct published date on the PAG (today). @BioWilko Please re-raise Asklepian

[SamStudio8]

Finally Github is working again!

Going forward, to mitigate, Majora will set the published_date of a PAG being updated with api.artifact.file.add if it is not set (regardless of whether it is new).

Additionally, the error code emitted by Ocarina briefly confused the source of the problem (as exit 2 has typically been the exit code in the well known case of an OAuth token refresh while polling Majora for a task). Ocarina 42.1 updates the error codes to follow BSD sysexits. This would have returned exit 69 (EX_UNAVAILABLE) and although OAuth token trouble should be a thing of the past (#114), expired OAuth tokens (and other permission problems) will generate an exit 77 (EX_NOPERM).

Updated ocarina in the samstudio8, nicholsz-asklepian, and elan-ocarina envs to https://github.com/SamStudio8/ocarina/commit/bb6047ab0253893e4b918e41474c935fa7ec5aa5.