Simplify logging

[ehmatthes]

Originally, I didn't want to start writing a log file until we knew that the deployment was going to work. In retrospect, this was an unnecessary complexity. We're taking significant steps such as creating a project, and creating a db before we start logging. These are really important things to have a record of, especially for troubleshooting.

It's perfectly reasonable to create a log file as soon as we have something to log. People can pass the --no-logging arg if they need, and they can delete log files or directories before making a commit to their project if needed.

• [x] Begin logging immediately during the inspection and configuration process.
• [x] Make sure we're not logging anything sensitive such as db passwords.
  • [x] Platform.sh
  • [x] Fly.io
  • [x] Heroku
• [x] Update tests
• [x] Tests pass on Windows
  • [x] unit tests
  • [x] integration tests
• [x] Update documentation (nothing to update)
• [x] Merge this branch.
• [x] Set up PyPI 2FA.
• [x] Make new release
• [ ] Run integration test against PyPI version.
  • [x] fly_io
  • [ ] platform_sh (worked manually, as an end user would, but CLI having authentication issues from script.) See #265
  • [x] heroku
• [x] Move any significant open tasks to another issue, or roadmap doc.

[ehmatthes]

Working notes

• [x] Run build_dev_env.py, create fresh dev env. Use sys.exit() calls to ensure each step is being logged appropriately.
• [x] Create log file immediately.
• [x] build_dev_env.py should write reset_project.py.
  • [x] Should be cross-platform.
  • [x] Should run git reset as well.
  • [x] Add reset_project.py to gitignore. (No, this kept reset_project.py out of project repo. Having it tracked by git is no issue, as it doesn't change at any point.)
• [x] Log early steps that weren't previously being logged.
  • [x] platformsh.confirm_preliminary()
  • [x] fly_io.confirm_prelminiary()
  • [x] inspect_system()
  • [x] inspect_project()
    • [x] _check_git_status() should not flag simple_deploy_logs/ being in uncommitted changes.
  • [x] _confirm_automate_all()
  • [x] platform_deployer.validate_platform()
    • [x] platform_sh
      • [x] _validate_cli()
      • [x] deployed_project_name
      • [x] org_name
    • [x] fly_io
      • [x] Consider subclassing CommandError to log each error and then reraise.
      • [x] Consider a log_info() method in simple_deploy.py.
      • [x] _validate_cli()
      • [x] deployed_project_name
      • [x] region
      • [x] _create_db()
    • [x] heroku
      • [x] validate_platform()
      • [x] handle_poetry()
      • [x] other changes, ie skip_logging
  • [x] prep_automate_all()
  • [x] _add_simple_deploy_req()
  • [x] deploy()
  • [x] Search entire project for skip_logging=True.
• [x] Update conditionals in later logging steps.
• [x] Search codebase for raise CommandError. Should be any left.
• [x] Unit tests: add assertions about what we should find from early steps
• [ ] Integration tests: only check for sections of log that don't show up in unit tests?

[ehmatthes]

git status notes

• If there are changes that haven't been added, git status is not clean, but git diff shows no output.
• _check_git_diff() may need output of git status.
  • If git diff has no output, should only find simple_deploy_logs/ in output?
• Want to pass the git status check when:
  • git status lists only presence of simple_deploy_logs/
  • AND git diff is empty.
• [x] git status --porcelain is fantastic! Is that flag available on all commands? No, see Git - Plumbing and Porcelain.
• [x] log the output of git status?

[ehmatthes]

More tasks

• [ ] Deploy to fly_io without cli installed; does _validate_cli() work?
• [ ] Try heroku deployment without CLI as well.
• [x] Revisit fly_io _create_db(); should we log the output of fly postgres create? Probably not, because I believe it shows db crews. Maybe scrub them with and log the rest of the output. I imagine that would be quite useful.
• [ ] Probably post-1.0, may want to print commands being run to console as well as output. ie for Fly.io, show command fly apps create --generate-name as well as output of that command. This is clear in logs, but should be just as clear on CLI output?
• [ ] Note for later that much of this logging could be moved to ie execute_command(), or execute_subp_run(). Would probably need skip_logging parameter.

[ehmatthes]

prep_automate_all()

• [x] platform_sh
• [x] fly_io
• [x] heroku

[ehmatthes]

Final changes before testing

• [x] Skim through all files:
  • [x] platform_sh/deploy.py
  • [x] fly_io/deploy.py
  • [x] heroku/deploy.py

[ehmatthes]

First pass at testing

Start with integration tests first, because they are most important and don't check log output. If those pass, unit tests should be mostly updating log assertions?

Make sure integration tests pass, and skim through log after each deployment to address any glaring issues.

• [x] integration tests
  • [x] platform_sh (start here, because these almost always pass)
  • [x] fly_io (next, because just ran these)
  • [x] heroku (last, because least likely to pass based on staleness)
• [x] unit tests
  • [x] platform agnostic
  • [x] platform_sh
  • [x] fly_io
  • [x] heroku

[ehmatthes]

platform_sh integration test

• [x] Hangs at login required.
  • [x] Catch this, or otherwise surface it to user. (Note, this is an issue with the test, not simple_deploy.)
• [ ] CLI logged in test should happen in conftest.py.check_valid_call().
• [x] Don't log output of platform project:info --yes. It includes a bunch of identifying information, and the repository client ssh key.

[ehmatthes]

fly_io integration test

• [x] Need to update fly CLI.
• [ ] Functionality tests failed, but passed when run manually.
• [ ] Catch assertion error from functionality test, print message to consider running the test manually, perhaps with temp project location (maybe full command to run test), then re-raise assertion exception?
• [ ] Log info could be formatted better:

2023-11-04 01:32:11,742 INFO: Looking for Fly.io app to deploy against...
2023-11-04 01:32:12,552 INFO: fly apps list
2023-11-04 01:32:12,552 INFO: NAME                          OWNER       STATUS      PLATFORM    LATEST DEPLOY 
2023-11-04 01:32:12,552 INFO: fly-builder-wild-thunder-9106 personal    suspended   machines                    
2023-11-04 01:32:12,553 INFO: snowy-sea-2928                personal    pending     machines                    
2023-11-04 01:32:12,553 INFO: 
2023-11-04 01:32:12,553 INFO:   Found Fly.io app: snowy-sea-2928
2023-11-04 01:32:12,553 INFO: Looking for Fly.io region...

[ehmatthes]

Update unit tests

• [ ] Look at test_project_inspection.py.

[ehmatthes]

build_dev_env.py cross-platform

• [x] subprocess.run() needs to use cmd, not cmd.parts().
• [x] Is it committing the venv?
  • [x] Sample project needs to ignore .venv!
• [x] Address -a issue.
• [x] Address line endings issue.
• [x] Run unit tests on Windows.
  • [x] gitignore test is failing, because I added .venv/ to ignore.
• [x] Unit tests still work on macOS.
• [x] Run integration tests on Windows.
  • [x] platform_sh
  • [x] fly_io
  • [x] heroku
• [x] Snapshot the Windows vm
• [x] Make sure integration tests still work on macOS.

[ehmatthes]

Platform.sh integration test (Windows)

• [x] git status shows .platform/ directory.
  • [x] Why is this being added before the git status check? platform create creates a .platform/local/ directory, with .gitignore, project.yaml, and README.txt files.
  • [x] Is this happening on macOS as well? 1) It's not causing a failed deployment. 2) Is that same directory being created???
  • [x] Resolve on both platforms.
  • [x] SSH issue on Windows after a couple deploy attempts.
  • [x] Oof, this is all because the platform CLI must be run in a bash environment? My god, they should detect when they're not running in a bash environment and exit with a clear message!

[ehmatthes]

Moving remaining platform.sh issues to #265.