search

tl-its-umich-edu / remote-office-hours-queue

Virtual queuing tool supporting Zoom video conferencing and/or in person meetings.
https://officehours.it.umich.edu/
Apache License 2.0
12 stars 28 forks source link

Investigate and implement additional fixes for prod service deployment #524

Closed jonespm closed 1 week ago

jonespm commented 4 months ago

Things we need to investigate/fix for next release:

TODO: Need to think about test/release plan for this change

lsloan commented 2 months ago

@jonespm, can you elaborate on the 2nd and 3rd bullet items?

jonespm commented 2 months ago

So 1) I think this is controlled here https://github.com/tl-its-umich-edu/remote-office-hours-queue/blob/177caed90e68cfaf1f150d43d303e3e9f5529201/service/base/web-deployment.yaml#L35 And is fine for test/dev but should be switched off in prod via overlay. (Or switched on in dev/test and off by default)

2) All of our other projects deploy from an imagestream in the test project. For this project you have to build and deploy from the prod project.

I think that's because this is defined here in the overlay https://github.com/tl-its-umich-edu/remote-office-hours-queue/blob/177caed90e68cfaf1f150d43d303e3e9f5529201/service/overlays/prod/deployment.yaml#L3

I haven't ran this kustomize on every deploy though, if nothing hasn't changed. However I'd also prefer it deployed from a tag rather than latest so maybe this would need to be updated and ran every-time unless there's some other way to do it. Like for instance still create a tagged image like 1.10.0 but also just have another one linked to this called "release" or something and just update that?

So maybe when the next tag is getting ready this might have to be changed to something like

value: image-registry.openshift-image-registry.svc:5000/officehours-test/officehours:release but this is actually image-registry.openshift-image-registry.svc:5000/officehours-test/officehours:1.11.0. In any case this shouldn't be in officehours-prod.

Ideally this would reference an image built on Github via #454 also like our other apps rather than just built on Openshift.

3) I think that's referring to this like that still says officehours, and should be officehours-prod. https://github.com/tl-its-umich-edu/remote-office-hours-queue/blob/177caed90e68cfaf1f150d43d303e3e9f5529201/service/overlays/prod/deployment.yaml#L12

This is what it shows on the test overlay https://github.com/tl-its-umich-edu/remote-office-hours-queue/blob/master/service/overlays/test/deployment.yaml

So I think I might have had to manually change this but if this kustomize file is ever run on prod again it will mess it up.

I'm not sure how to test this prod overlay except on "release day" and it was a headache when things were breaking and I had to manually make modifications to get things working.

jonespm commented 1 month ago

We discussed this at the meeting. The second item related to using a test image stream isn't required and if it can't be completed for this release should be moved to it's own issue for the next release. This should be committed by Thursday so we can release next week.

lsloan commented 1 month ago

It's tricky and I need more time to figure that out, so I'll just make a new issue for that remaining item so we can call this one good.

jonespm commented 3 weeks ago

We won't be able to fully QA the prod overlay change until this goes to prod but I confirmed the base and test changes apply so marking as done.