search

cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
29.9k stars 3.78k forks source link

upgrades: failures during an auto upgrade do not leave observable artifacts #90148

Open fabiog1901 opened 1 year ago

fabiog1901 commented 1 year ago

Describe the problem

I initiated the upgrade of the cluster from 21.2.14 to 22.1.8 and the finalization part failed.

Upon research, there was a log entry with the description of the error shows a severity level of INFO (fixed in #90165).

We would also like to see some notification that it failed, for example as a failed job or some other item in DB Console.

@dt explains:

i guess auto_upgrade is less obvious; we don't want to just spam failed jobs. But at least for explicit sets, it does feel like there should be a durable record of when we checked and what failed (edited)

Details on Slack

Jira issue: CRDB-20615

knz commented 1 year ago

doing the severity part in #90165.

For db console changes, we'll need to retitle the issue (after #90165 merges)

github-actions[bot] commented 5 months ago

We have marked this issue as stale because it has been inactive for 18 months. If this issue is still relevant, removing the stale label or adding a comment will keep it active. Otherwise, we'll close it in 10 days to keep the issue queue tidy. Thank you for your contribution to CockroachDB!