49 fix dependency caching in docker image build

[gvegayon]

• Puts all the content of workflows 1-3 into workflow 2.
• Deletes workflow 1 and renames workflow 2 as workflow 1.
• Sets the dockerimage label to be either branch name if it is a PR or latest if it is merging to main.

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Additional details and impacted files

:loudspeaker: Thoughts on this report? Let us know!

[gvegayon]

I think the docker file needs to point to the tag dynamically. Will figure it out later today (about an hour). Unless you want to give it a try!

[zsusswein]

I won't have time to get to it today, but I can try to look tomorrow if needed!

[gvegayon]

Did we ever figure out if the pool was actually linked to the pushed image in ACR? If no, I'd like to keep it out of scope here but I want to know to make it an issue and tag it for Jon.

I don't think it is linked, as I don't see any line passing that information to the pool. I could create an issue out of it.

I flagged the scheduling inline, but does the scheduling still make sense here? I think no, but would like a second pair of eyes.

Just commented on that. We could comment that out for the moment and put it back as soon as we have a few runs in production.

Do we want to renumber the workflows? I think it's fine if the answer is no and we'll plan to slot the job creation in (2)

Probably, I would renumber them (a separate PR).

What are the other issues to create? ACR image deletion on PR close? Pool deletion on PR close? Polling for pool status before rebuild (and figuring out the strategy)?

From the top of my mind:

• Re-numbering workflows.
• Automatic pool deletion, which would imply having the R/python script running in batch writing a file with "success + message" or something like that + having a separate workflow that connects to azure and checks for that file. Since jobs could take hours running, I would have a workflow that automatically closes pools that have finished, unless there's a message or metadata of the pool indicating "do not terminate" or something like that.
• Automatic deletion of the branch-based image for PRs.

For the future, I suggest having a workflow (or a job within existing workflows) that creates a GitHub issue for the run (e.g., weekly runs) and uses that for posting information. That could be the way in which Azure batch communicates it is done or whatnot. That could be an action for https://github.com/CDCgov/cfa-actions!

[gvegayon]

Also, important: pool creation fails when there's a pool with that id/name. This could be either addressed here or somewhere else. I suggest opening an issue for doing this, as you can still stop/delete pools manually.