search

CDCgov / cfa-epinow2-pipeline

https://cdcgov.github.io/cfa-epinow2-pipeline/
Apache License 2.0
10 stars 2 forks source link

Set up job on deployment #48

Open zsusswein opened 2 months ago

zsusswein commented 2 months ago

This PR is now tightly focused on getting jobs linked to the pools in Azure Batch. It needs to have the required environment variables injected into the pool to interact with the Azure Blob API.

https://github.com/CDCgov/cfa-epinow2-pipeline/blob/67727c6565542e998b03e79f3dc62116dbb6d005/R/azure.R#L93-L99

This issue is likely gated by #59

We should also evaluate how to kick this off. I'm thinking probably on a schedule and a workflow dispatch trigger? I'm on the fence about including this functionality in this issue vs. a related one. Feel free to punt it! EDIT: Let's do workflow_dispatch only here and handle scheduling in or after #68

Currently there's a missing (2) action based on @gvegayon's refactor in #54. We're leaving that spot to be filled in this PR. If this action is instead appended to a different action (maybe (1)?) then this PR should renumber the actions. EDIT: #86

From @jkislin in #43 as the original list of to-dos:

To-do:

  1. There are likely parameters buried in the cfa-nnh-pipelines create_expt_pool.py module that I haven't fully ported in my zeal to simplify. I can get that validated tomorrow. EDIT: Now #59
  2. Job submission is very rudimentary and buggy here, and I have it commented out for now. It will take some time to port from cfa-nnh-pipelines.
  3. Curious if you want git long shas or git short shas for versioning pools, containers, and jobs. I've settled on short shas for now for readability. EDIT: @gvegayon has moved us over to a naming syntax tagging the pools and images with latest and <branch-name> in #54. I think that's fine but let's revisit if you have concerns Jon.
  4. Take a look at the autoscale formula text file I've put in the top level of the repo. I know we wanted to change things. I'm new to the syntax, so haven't changed anything yet from cfa-nnh-pipelines, but I can dig in tomorrow. There's probably also a way to get this incorporated directly into the Github Actions yaml, but this seems to be a fair approach. EDIT: Now #61
  5. Code Coverage is now failing. I'll give this a look over tomorrow as well.
  6. We need to determine a schedule to run the jobs. Do we still want to go with Wednesday mornings? Apologies if this is obvious!

Only (2) is included in this issue as in-scope. Everything else has been moved to separate issues.

jkislin commented 2 months ago

5 and 6 have been handled in PR 43, 1-4 will be in the next PR. Thanks Zach!

zsusswein commented 1 month ago

@dylanhmorris had issues setting this up on the VAP via the SP.

It looks from a prior thread where Arik found a fix that this is a WSL-specific networking thing and hopefully won't reproduce on a linux runner

jkislin commented 1 month ago

@zsusswein , yeah, that's totally a python/cfa_azure thing specific to logging in with regular users. We're using a service principal directly logged in with the Azure CLI here, so shouldn't be an issue. Let me know if you find out otherwise.

dylanhmorris commented 1 month ago

^ WSL-networking-workaround-specific, in particular