pcrespov commented 1 year ago

Release version

1.51.0

Commit SHA

b91f3442b25888e3997d20af24fb769e80f53921

Previous pre-release

https://github.com/ITISFoundation/osparc-simcore/releases/tag/staging_Mithril4

Did the commit CI suceeded?

[X] The commit CI succeeded.

Motivation

Permalink feature in production ready before SCM

Changes

Devops check 👷

[x] Review changes above with (⚠️ devops)
- [x] catalog environ missing (check issue opened by @sanderegg )
[ ] Add changes (if any) and release tags in https://github.com/ITISFoundation/osparc-ops-environments/tags
[ ] Add changes (if any) and release tags in osparc ops configurations

Tests assessment: e2e testing check 🧪

[ ] vtk links failing staging
[ ] gc in aws-staging. @mrnicegyu11 looking into it.

Test assessment: targeted-testing 🔍️

[x] Check changes by @GitHK
[x] Check changes by @sanderegg
[x] Check changes by @pcrespov
[x] Check with @elisabettai

Test assessment: user-testing 🧐

No response

Summary 📝

[x] Prepare release link

make release-prod version=1.51.0  git_sha=b91f3442b25888e3997d20af24fb769e80f53921

[x] Draft release changelog
[x] Announce maintenance. Tue, Mar.28 from 10:30 - 12:00 am :

redis

  { 
    "start": "2023-03-28T08:30:00.000Z", 
    "end":   "2023-03-28T10:00:00.000Z", 
    "reason": "Release v1.51.0"
  }

[x] aws
[x] dalco
[x] tip

status page
- [x] osparc
- [x] s4l
mattermost channels
- [x] maintenance
- [x] power users

Releasing 🚀

[x] Maintenance page up 👷
[x] Release by publishing draft
[ ] Check release CI
[ ] Check deployed
- [x] aws deploy
- [ ] dalco deploy
- [ ] tip deploy
[x] Delete announcement
[x] Check e2e runs

[ ] Announce

:tada: https://github.com/ITISFoundation/osparc-simcore/releases/tag/v1.51.0

pcrespov commented 1 year ago

WARNING: see time!

mrnicegyu11 commented 1 year ago

BLOCKED

Issue https://github.com/ITISFoundation/osparc-simcore/issues/4027 until understood / fixed is a blocker for me to release to prod, please provide some comforting words for poor mrnicegyu11 or fix the bugs. thanks :)

Update: We released anyway.

mrnicegyu11 commented 1 year ago

Even though issue https://github.com/ITISFoundation/osparc-simcore/issues/4027 impacts production, we released. For now, dynamic services that dont close need to be monitored constantly to prevent resource derivation. The procedure is as follows:

- monitor matermost channel osparc-batman for alert
- open portainer for deployment that caused alert
- open services tab in portainer
- search for service with the $NODE_UUID from the mattermost alert
- if no sidecar is shown: do nothing. all is fine.
- if sidecar is there: open a shell in the director-v2 container
- run command: simcore-service-director-v2 close-and-save-service $NODE_UUID
- if the command printed above cause errors: write in osparc core team chat about it, a backend dev needs to check
- if the message "Finished cleanup for service $NODE_UUID" is printed, the hanging sidecar was successfully cleaned up

Aftermath:

Forever hanging sidecars CC @GitHK @pcrespov @sanderegg :

osparc.io

dy-sidecar_f6299542-3d23-5067-8ae0-e6addd6b33a0
dy-sidecar_eb71c1e7-94db-48c7-9eec-011511bafddf
dy-sidecar_b1c8ad49-d862-53a4-9f7e-4fba678d0b78
dalco
dy-sidecar_32116d2d-cdda-5d0b-8145-146f5a621d14
tip
dy-sidecar_969c4d64-788f-5c79-b189-851b6eb30857

Issues

Due to a human error, a docker node placement constraint was briefly set wrong on tip.itis.swiss, and resolved after 10min.

ITISFoundation / osparc-simcore

🚀 Release v1.51.0 #4023