Update Heroku deployment process

[ehmatthes]

Heroku only auto-provisions a Postgres db for users whose accounts were created before 5/15/23. This means deployments will fail for anyone with a new Heroku account:

Task list

• [x] Simplify logging first. It needs to be done, and it will simplify this work as well. #260
• [x] Update Heroku process to create a Postgres db.
• [x] Test, including integration test with poetry.
• [x] Release.
• [x] Test PyPI version.
• [ ] Share, and tell people that all three platforms should be working. If not, file an issue, please!

[ehmatthes]

Notes 12/23

This should be simpler to fix after working through Fly deployments, #263. We'll call heroku create, and look for a db. If it doesn't exist, we'll create one.

• [x] Create a test project, call heroku create, then run heroku addons to see what's created for me (I'm guessing a mini postgres, because of my older heroku account that still qualifies for auto provisioning a db).

% heroku create
Creating app... done, ⬢ desolate-shore-27436
https://desolate-shore-27436-3a63a9188e3c.herokuapp.com/ | https://git.heroku.com/desolate-shore-27436.git
(.venv) eric@Erics-Mac-Studio dsd-dev-project_pg1o3 % heroku addons
No add-ons for app desolate-shore-27436.

This looks like I'm not getting a db auto-provisioned.

• [x] Try integration test; expect it to fail, unless a db gets created on the git push heroku main.

remote: -----> Discovering process types
remote:        Procfile declares types -> web
remote: 
remote: -----> Compressing...
remote:        Done: 33.6M
remote: -----> Launching...
remote:  !     The following add-ons were automatically provisioned: heroku-postgresql. These add-ons may incur additional cost, which is prorated to the second. Run `heroku addons` for more info.

Postgres database was auto-provisioned:

% heroku addons

Add-on                                      Plan  Price         Max Price  State  
──────────────────────────────────────────  ────  ────────────  ─────────  ───────
heroku-postgresql (postgresql-curly-63375)  mini  ~$0.007/hour  $5/month   created
 └─ as DATABASE

The table above shows add-ons and the attachments to the current app (pacific-sands-48013) or other apps.

[ehmatthes]

Notes 12/23 (cont)

• [x] Try calling heroku create, then create a db. Will it auto-provision a second, redundant db? (Do this with a manual push from a dev project.)

remote: -----> Compressing...
remote:        Done: 33.6M
remote: -----> Launching...
remote:  !     The following add-ons were automatically provisioned: . These add-ons may incur additional cost, which is prorated to the second. Run `heroku addons` for more info.
remote:        Released v8

Does not auto-provision a db if it finds an existing one. This is confirmed by running heroku addons again after the push, and finding just the one db that I manually created.

[ehmatthes]

Implementation

• [ ] Is heroku addons:create heroku-postgresql idempotent? Try this. (It is not idempotent; makes sense, can use two dbs for a single app.)

 % heroku create
Creating app... done, ⬢ polar-ocean-42915
https://polar-ocean-42915-4c1ec37e8ebe.herokuapp.com/ | https://git.heroku.com/polar-ocean-42915.git
(.venv) eric@Erics-Mac-Studio dsd-dev-project_pg1o3 % heroku addons
No add-ons for app polar-ocean-42915.
(.venv) eric@Erics-Mac-Studio dsd-dev-project_pg1o3 % heroku addons:create heroku-postgresql:mini
Creating heroku-postgresql:mini on ⬢ polar-ocean-42915... ~$0.007/hour (max $5/month)
Database has been created and is available
 ! This database is empty. If upgrading, you can transfer
 ! data from another database with pg:copy

Created postgresql-lively-93073 as DATABASE_URL
Use heroku addons:docs heroku-postgresql to view documentation
(.venv) eric@Erics-Mac-Studio dsd-dev-project_pg1o3 % heroku addons:create heroku-postgresql:mini
Creating heroku-postgresql:mini on ⬢ polar-ocean-42915... ~$0.007/hour (max $5/month)
Database has been created and is available
 ! This database is empty. If upgrading, you can transfer
 ! data from another database with pg:copy

Created postgresql-clear-50368 as HEROKU_POSTGRESQL_BLACK_URL
Use heroku addons:docs heroku-postgresql to view documentation
(.venv) eric@Erics-Mac-Studio dsd-dev-project_pg1o3 % heroku addons

Add-on                                       Plan  Price         Max Price  State  
───────────────────────────────────────────  ────  ────────────  ─────────  ───────
heroku-postgresql (postgresql-clear-50368)   mini  ~$0.007/hour  $5/month   created
 └─ as HEROKU_POSTGRESQL_BLACK

heroku-postgresql (postgresql-lively-93073)  mini  ~$0.007/hour  $5/month   created
 └─ as DATABASE

The table above shows add-ons and the attachments to the current app (polar-ocean-42915) or other apps.

[ehmatthes]

Implementation (cont.)

• [x] In configuration-only run, tell people to run heroku create and heroku addons:create heroku-postgresql:mini.
• [ ] Parse apps:info, look for postgres db; create one if it doesn't exist. (Assume this is the only add-on?)
• [x] Note that if you don't create one, simple_deploy will create one for you. (Be explicit if possible.)
• [x] In automate-all run, make this call in addition to the create call, if a database doesn't already exist.
• [x] Test manually:
  • [x] config only, without having called create
  • [x] config only, after create and db
  • [x] second config only run, without destroying first project
  • [x] config only, without prior db
  • [x] automate-all
• [x] Run unit tests. (all pass, except poetry)
• [x] Run integration tests.
  • [x] req_txt
  • [x] poetry
  • [x] req_txt on Windows
• [x] Review documentation.
• [x] Review messages.

[ehmatthes]

Running on Ubuntu

I haven't tested any of these on Linux in a while. Let's start here.

• [x] Build an Ubuntu VM.
• [x] Set up for Python development, with Python 3.12.
• [x] Snapshot.
• [x] Clone project.
• [x] Build dev test project, and try running without Heroku CLI.
  • [x] Fixed.
• [x] Run heroku unit tests. (All pass except poetry.)
• [x] Use simple_deploy manually to make a deployment. (heroku open doesn't work, but maybe a configuration issue on my end?)
• [x] Run Heroku integration tests.
  • [x] First attempt: migration failed. (Was able to issue migration command from tmp project directory, and db was usable but sometimes got 500 errors.)
  • [x] Second attempt: Migrations succeeded, some tests failed. (Was able to manually create accounts and enter data, but got intermittent 500 errors. Who knows root cause. I have not heard great things about Heroku databases lately.)
• [x] Try an automate-all run from a dev project. (Worked, but got an out of memory error during the migration run. Running migrations manually, it seems only some ran originally. Odd that I don't see these issues when pushing from macOS. If persist only on Linux, perhaps diff a project configured on macOS with one configured on Linux?)
• [x] Run integration test on macOS one more time. (Some db issues during testing as well; probably an issue on Heroku's end.)
• [x] Push a project after only calling create manually. What does my legacy heroku app generate? A postgres mini, or something larger? (It's a mini.)

Notes

• Heroku CLI requires node. These instructions helped me install node.
• Needed Chromium for heroku login to work.
• Also, had better luck setting up Python 3.11 than 3.12.