Split the build and publish steps into two separate jobs. The 'build' job builds and checks the distributions and then uploads them as a job artifact. The 'publish' job downloads the required artifact from the 'build' job and the publishes them to TestPyPI or PyPI if the typical publishing requirements are met.
Use the OpenID Connect (OIDC) standard to publish to PyPI and TestPyPI using PyPI's "Trusted Publisher" implementation to publish without using API tokens stored as GitHub Actions secrets. Use an optional GitHub Actions environment to further restrict publishing to selected branches ('main', 'release/', 'v') for additional security.