Update workflow

[Mandera]

I want to support [LINUX] and [WINDOWS] for when I only want to test on a single quick instance. Hmm maybe being able to write [3.8] could be good too. What about any combination?

• [LIN 3.8 3.9]
  • Linux 3.8
  • Linux 3.9
• [3.9]
  • Windows 3.9
  • Linux 3.9
• [LIN]
  • Linux 3.8
  • Linux 3.9
  • Linux 3.10
  • Linux 3.11
  • []
  • No run (New [CI SKIP])

[Mandera]

I also want to an initial step to halt simultaneous actions, somehow prioritize the first one, race condition is fine

[Mandera]

Also stop creating so many commit when not needed, only commit if something changed

• Sort mermaid in readme
• Disable using sha in readme
• Remove writing time in readme

[Mandera]

I think I should take an even bigger step back -- The entire workflow is convoluted. I'm trying to do everything at once, so let's break it down.

Abstract

When changed:

1. Ensure it still works
2. Update information
3. Publish

Change can be

• Direct commits to repo
• Dependency being updated
  • PyPI
  • RSS
    • Would require a function to first get dependencies (indirect too?)
  • libraries.io (only direct deps)
  • NPM
  • Can't find RSS feed
  • libraries.io (only direct deps)
  • GitHub Action
  • Perhaps overkill, only for setup-python, it's not even on libraries.io
• New language version (Probably just manual?)
  • Python
  • Node
• New OS version (Manual too?)
  • Windows
  • Linux

1. Ensure it still works

• Run unit tests
  • No errors
  • Code itself can be broken
  • A dependency could have broken it
  • Or both

2. Update information

• Updating files in repo itself such as readme, setup, license
• Updating files in my profile repos
• Updating GitHub metadata

3. Publish

If the version is bumped and tests pass then we publish.

• PyPI for Python
• npmjs for JS
• Coda for packs
• SSH into servers for server repos

Current setup

Currently, these 2 of these steps happen for all 5 of my general packages (only Python) with every commit. Step 3 happens if the version is bumped. Workflow takes a long time and scales poorly. Doesn't handle external dependencies.

Special treatment

I think it's a mistake to treat my dependents differently. Workflow should run when any dependency is updated.

Re-sync Library when Packager is published

Since Packager isn't a dependency for Library, Library won't be triggered when Packager is published. This is where we can make high-level functions to manually re-sync everything. Oh but wouldn't this require a bump and push, to test and publish? Which would trigger everything. Perhaps a simple re-sync would never warrant a bump on its own.

Version locking

Option 1 -- Commit on success:

• All dependencies are locked to versions we know work: dep <= x
• If tests fail we simply don't commit and notify

Option 2 -- Commit on fail:

• All dependencies are unlocked
• If tests fail we need to lock to the last known working version
  • It would be nice if this could all happen within 1 workflow run
  • How do we determine last known working version?
  • GitHub Artifacts?
  • Could perhaps dynamically downgrade one version at a time until it works? Probably very complicated

Behavior in my environment on publish:

C depends on B B depends on A

Option 1 -- Commit on success:

• Push to A with bumped version
  • Sync and publish on test success
• B is triggered and updates
  • B publishes on test success with new locked version
• C is triggered...

Recursive workflows, not ideal

Option 2 -- Commit on fail:

• Push to A with bumped version
  • Sync and publish on test success
• B is triggered
  • Test success
  • Nothing happens
    • C is never triggered... Should it be?
  • Test fail
  • Locking version to previous working one and publish
    • C is triggered

If we can make indirect dependencies trigger then this seems better! I guess we could also rely on manual and scheduled triggers.

Lock indirect dependencies?

Probably don't bother, would mess up the dependency chain cause it would look like my package directly requires all indirect dependencies, it would "flatten" the chain.

Private dependencies

I'll need to support private dependencies in the future, perhaps wait with this for now

Plan

• Only install trigger repo
• Sync non-aesthetic
• Run tests
• If triggered by direct commit, and version bump on the master branch, publish
• If triggered by dependency update, only publish on test fail to lock version