jbrown-heroku
commented
1 year ago Thank you @kreintjes for bringing this up. Although we must work with some constraints on the Essential tier, having better communication of the behavior, including notification, will be helpful for our customers. We will consider improvements.
Required Terms
What service(s) is this request for?
Postgres
Tell us about what you're trying to solve. What challenges are you facing?
We use Essential tier databases (Mini and Basic) for our staging/acceptance environments and for some small/non-critical production apps. Recently we noticed these databases were suddenly upgraded from Postgres 13 to Postgres 14 during regular maintenance. This maintenance was largely unannounced (we only got an e-mail 7 minutes before it started), took quite some time (1.5 hours) and it was unclear the Postgres major version would or had been updated (no details what happened, we discovered this ourselves later). We also didn't want to use Postgres 14 yet and were unsure if you app would work with it.
After some discussion with Heroku support, we understand that this expected behaviour for Essential tier databases and unannounced Postgres major version upgrades can happen at any moment. We believe this severely decreases the usability of these databases. They become less usable for staging environments, since you want to match the Postgres (major) version with that of production, but instead it can change any time and unnoticed (causing acceptance tests to be less accurate). Older major versions are still supported though, so you can change it back (by creating a new database and specifying the older version), but Heroku can decide to upgrade the major version again unannounced and at any moment, which is quite unexpected.
For small and even non-critical production apps they become unusable, since a Postgres major upgrade could likely break things. Even if some unexpected downtime (e.g. an hour or less) for these apps is acceptable, the downtime and risk caused by sudden major version upgrades, is just too high (at least to us). We understand that the Standard database tier does not have this problem and we upgraded these three apps to that tier now, but the cost increase is quite steep (from $9 per month per database to $50).
We propose several solutions/improvements:
1. Do not perform unexpected Postgres major version upgrades
This would probably make these databases suitable again for both our goals (staging and non-critical production apps), since the downtime for regular maintenance is quite short and the risk much less. However, we understand from Heroku support that the sudden version upgrades are caused by design limitations of the Essential tier (which uses shared resources for these databases), so this might be difficult to fix.
2. Clearly notify about a Postgres major version upgrade
At least afterwards, but preferably also before-hand. This would make the tier usable for our staging databases again, since we can then downgrade our databases to the previous version (matching production) so our testing remains accurate. It would of course be slightly cumbersome, especially if these automatic upgrades happen a lot, but we expect we can manage (they did not happen that frequently in the past).
If the notifications happen before-hand and a lot earlier than currently (e.g. a week before-hand instead of just several minutes), than we might even be able to start using these databases for our non-critical production apps again. That way we can test the new version before-hand and maybe even announce the downtime about to happen.
3. Add a cheaper Standard database option
As said before, the cost increase from Basic to Standard-0 is now quite steep ($9 p/m to $50 p/m). It would help a lot if there was a cheaper option in between (e.g. around $25 p/m). Than we could just use that new Standard database for all our non-critical production apps as well and better schedule maintenance and have even less downtime. While some (unexpected) downtime for these apps is no problem, less is of course better.
The features we now miss the most in the Essential tier database for our use-case are in order: no sudden Postgres major upgrades, better maintenance control (i.e. scheduling maintenance) and Encryption at Rest. The other extra's such as more RAM/storage/rows/connections, Forks, Followers, Rollbacks and more Backups are less important for us. You could take this into consideration when designing the new plan. Probably the hardest part will be coming up with a good name ("Standard Minus 1"/"standard--1" 😄 ), although I guess you could rename the current standard-0 to standard-1 and use standard-0 for the new plan.
4. Better documentation about unexpected major version upgrades
For us it was quite unexpected that the Essential tier databases behave this way and sudden major version upgrades are normal behaviour. In case the other suggestions are too much work, then I guess it would at least be better if the Heroku docs and add-ons page are more clear about this behaviour. We know there is some documentation about it, for example the sentence "Unannounced maintenance and database upgrades" here, but it is not very clear. It would help to change this to something like "Unannounced maintenance which could involve automatic Postgres major version upgrades". UPDATE: I see this has now changed to "Unannounced maintenance and automatic Postgres version upgrades", which is better, but I would still suggest to explicitly add "major version" as well since that's quite different (or use my previous suggestion).
It would help even more to be clear about this on the add-on page (e.g. in the specs table). There is nothing about the missing maintenance control and the possibility of sudden major version upgrades now.