9/12 Automation Update

[KELSEYDOWLING7]

• [x] Weekly Biospecimen Report [succeeded]
• [ ] Weekly Biospecimen CSVs [failed]
• [x] Weekly Biospecimen QC- automated (broke this week) [succeeded]
• [x] Weekly Biospecimen Custom Rules + output [succeeded- CSV did not post to Box]
• [x] Weekly Module Metrics Report [succeeded]
• [x] Operations report new filename.. [succeeded]

[jacobmpeters]

@lorszag testing :)

[KELSEYDOWLING7]

Can't seem to assign Brittany here. I'll try tagging her directly @BrittanyCrawford405

[jacobmpeters]

@KELSEYDOWLING7 The module metrics ran successfully.. The Weekly Biospecimen report is running now..

[jacobmpeters]

Can't seem to assign Brittany here. I'll try tagging her directly @BrittanyCrawford405

I added Brittany after you tagged her. I think she'll see this after she accepts the invitation.

[jacobmpeters]

Kelsey, I think this could be an issue with the new docker container I set up. I'm going to revert back to the old one and try this again tomorrow. I can also test your other biospecimen reports then as well. Usually we use xelatex rather than pdflatex so I think I can fix this on my end..

[KELSEYDOWLING7]

Ok! I did add back "latex_engine: xelatex" to the Weekly Module Metrics YAML and that worked successfully yesterday. Would it be helpful to add that back in this report? Or do you think that makes no difference?

[jacobmpeters]

Yeah let's add it back. I think it does make a difference. I remember configuring the docker container to work with xelatex rather than pdflatex. Might as well try the easy solution first.

[KELSEYDOWLING7]

Ok a new PR has been submitted and latex_engine: xelatex added to both the weekly biospecimen metrics and the custom biospecimen qc report

[jacobmpeters]

The Weekly Biospecimen Report succeeded!

The Weekly Biospecimen CSVs failed:

[KELSEYDOWLING7]

Yay!! Well one down. I'll run the csv report again and see if anything comes up on my end

[jacobmpeters]

The Custom Biospecimen QC worked but I'm not sure if it was delivered to Box.

[KELSEYDOWLING7]

Yup! It posted to Box

[KELSEYDOWLING7]

Oh, but the csv files that should have been created by that report did NOT post to Box....

[jacobmpeters]

Oh, but the csv files that should have been created by that report did NOT post to Box....

Ok. I'll look into that in a bit. Should be an easy fix..

[jacobmpeters]

Ops report succeeded!!

[KELSEYDOWLING7]

The Weekly Biospecimen Report succeeded!

The Weekly Biospecimen CSVs failed:

I don't get this error locally, even after clearing my environment so I'm not quite sure what's needed here

[jacobmpeters]

The Automated QC is failing due to one of those annoying bigrquery page size issues.. I'll try to run that again after an adjustment.

[jacobmpeters]

The Automated Biospecimen QC succeeded.

[KELSEYDOWLING7]

@jacobmpeters Good news and bad news. Good news it that the Biospecimen Custom Rule CSV files are now successfully posting to Box. Bad news is the operations report and Metrics report is failing. I'm trying to revert the Operations report back to the code from 9/13 but am having several issues so far...

[KELSEYDOWLING7]

@jacobmpeters Actually the Weekly Metrics report said fail on the GCP status report, and then posted anyways so that one's good. And the Ops report is up and running again, and a PR has been pushed. It just needs your approval for the api and Docker file merge conflicts

[jacobmpeters]

Ok. That is good news. I hate that those statuses on the scheduler are not always accurate.

I'll look at that PR now and try to merge it..