Prep + Publish to PyPI

[ric-evans]

wipac-dev-tools is our guinea pig. It also has no requirements which will make this transition less painful.

Roadmap: Pre-Merge

1. Replace SetupShop usage in setup.py with setup.cfg section(s)
   • update setup.cfg with [wipac:cicd_setup_builder] section

     [wipac:cicd_setup_builder]
     pypi_name = wipac-dev-tools
     python_min = 3.6
     keywords_spaced = python tools utilities

   • requirements go in [options] section

     [options]
     install_requires =
         package-a
         package-b

2. Use copy-pasteable GH Action for generation of standardized setup/requirements (this is in wipac-cicd.yml)
   • ~Source of Truth TBD: requirements.txt/setup.py/setup.cfg/etc.~
   • Source of Truth: setup.cfg ([options] section)
3. Turn on the PyPI-publishing trigger within semantic_release's GH Action (this is in wipac-cicd.yml)
   • see https://github.com/WIPACrepo/wipac-dev-tools/pull/29

Roadmap: Post-Merge

Proceed to update WIPAC repos in descending order of complexity. All updates:

• follow backward compatibility, unless there's a good reason not to
• publish to PyPI only what is needed by a client, if any
  • prefix all PyPI packages with wipac-
• can be updated in time, there is no immediate necessary rush

Updates, in order:

1. Add wipac-telemetry to PyPI (https://github.com/WIPACrepo/wipac-telemetry/pull/53)
   • call it wipac-telemetry w/ v1.0.0 (no longer prototype)
   • has requirements, and requires wipac-dev-tools (and no other wipac packages)
   • follow Roadmap: Pre-Merge workflow for this repo
2. Add rest-tools to PyPI (https://github.com/WIPACrepo/rest-tools/pull/62)
   • publish on PyPI as wipac-rest-tools
   • requires wipac-telemetry (and wipac-dev-tools in turn, but no other wipac packages)
     • TBD: Will this be an optional/extra pip feature: wipac-rest-tools[wipac-telemetry]?
       • Is tracing "core" to rest-tools? No.
       • But is it worth separating the functionality at the package level? Maybe.
   • follow Roadmap: Pre-Merge workflow for this repo
3. Add other utility repos to PyPI
   • Packages:
     • MQClient (+ child repos) (https://github.com/WIPACrepo/MQClient/pull/40)
     • File Catalog (required by FC Indexer) (https://github.com/WIPACrepo/file_catalog/issues/127)
     • KRS (https://github.com/WIPACrepo/keycloak-rest-services/issues/49)
     • IceProd (required by FC Indexer) (https://github.com/WIPACrepo/iceprod/issues/307)
       • probably doesn't need the wipac- prefix
     • FC Indexer (https://github.com/WIPACrepo/file-catalog-indexer/issues/42)
   • fix requirements
   • publish on PyPI with wipac- prefixes
   • if/where PyPI is not appropriate for entire repo, then spinning-off utility repos may be wanted
     • Ex: wipac-file-catalog-types, wipac-krs-utils
   • follow Roadmap: Pre-Merge workflow for each repo
4. Fix requirements for each end-stream app
   • Apps:
     • MOU Dash (https://github.com/WIPACrepo/mou-dashboard/issues/58)
     • Skymap Scanner (https://github.com/icecube/skymap_scanner/issues/2)
     • LTA (https://github.com/WIPACrepo/lta/issues/230)
   • these apps are not required by any "downstream" repo/client (not-published on PyPI)
   • pre-merge: there may be some overlap with section above, see "spinning-off utility repos" point above
5. Fix requirements for each script
   • Scripts:
     • ...
   • these are not required by any "downstream" repo/client (not-published on PyPI)

[dsschult]

If you want to save some duplication efforts, you could use the tool cog along with a github action to render text into files programmatically.

An example of someone using cog to edit their build files: https://github.com/hynek/structlog/commit/5e6ea9c04e070f7ef0106ff1256c58ef54c03686

[ric-evans]

If you want to save some duplication efforts, you could use the tool cog along with a github action to render text into files programmatically.

An example of someone using cog to edit their build files: hynek/structlog@5e6ea9c

I'm looking into https://github.com/jazzband/pip-tools