StackStorm / community

Async conversation about ideas, planning, roadmap, issues, RFCs, etc around StackStorm
https://stackstorm.com/
Apache License 2.0
8 stars 3 forks source link

Testing Wheels generated by Pantsbuild #119

Open cognifloyd opened 1 year ago

cognifloyd commented 1 year ago

To generate all of the st2 wheels, run this from the root of the StackStorm/st2 repo:

pants package ::

Here, :: is like a recursive glob so that pants will package everything it knows about (based on BUILD metadata).

To build a subset of the wheels, replace :: with the path where it is defined. To build just st2common and the orquesta_runner, do:

pants package st2common contrib/runners/orquesta_runner

Pants puts all packaged artifacts under the dist/ directory.

The package goal

The package goal (where goals refer to the thing(s) you want to do with pants: pants <goal(s)> <path(s)>) documentation is here:

https://www.pantsbuild.org/2.18/docs/python/goals/package

The package goal makes various kinds of installable or distributable artifact. The only packageable artifacts in st2 are python_distribution artifacts, which includes sdists and wheels. In particular, it will generate wheels for all the components (defined in st2*/ like st2commom and st2api) and the runners (defined in contrib/runners/*_runner) when given the :: recursive target glob.

python_distribution

We use python_distribution in BUILD metadata to tell pants about each wheel we want it to generate. Documentation about building python distributions is here:

https://www.pantsbuild.org/2.18/docs/python/overview/building-distributions

Defining this BUILD metadata for st2 was the focus of these PRs:

Dependencies defined in other PRs also have an affect on what gets included in the wheel.

Testing the wheels

Open or install the sdists and/or wheels. Possible things to check:

As you're looking at it, what kinds of tests would be good for the wheels? Maybe even write some tests for pants to run via pytest (make the python_test target depend on the relevant python_distribution target to get the wheel in the test sandbox.

What to do when the wheel is missing something

When something is missing from a wheel, look for something else that is included in the wheel that might need a manual dependency on whatever is missing. If there is nothing obvious inside the wheel that needs the dependency (eg because it is a public API in st2common) then add the dependency to the python_distribution metadata.

What to do when files are in the wrong place in the wheel

I suspect this happens when the files are owned by a resources or files target when they should probably be the other one. We might also need to use relocated_files to move them similar to how I added LICENSE to all the wheels.