search

paritytech / parity-publish

A tool to manage publishing Parity's crates
Apache License 2.0
4 stars 1 forks source link

Integration Tests #32

Open ggwpez opened 4 months ago

ggwpez commented 4 months ago

We should probably have some regression end-to-end tests that take a historical set of Polkadot-SDK commit hashes and then run some commands. It should then assert on the output of the command and exit code.

I do something similar in Zepter to ensure that historic behaviour remains unchanged.

The command that we want to check the most is the prdoc command that is used extensively in the SDK CI.
First step would be to write some small unit tests that has some fixed crates. The checks for the prdoc command focus on semver changes, so there should be a test for at least each case: Major change, minor and patch. The test must assert that SemVer violations in the Minor and Patch category are flagged correctly and that Major changes can do anything.
Additionally it should have different prdoc files, some stating the correct changes and some stating incorrect changes.

Next step could be to use real crates like frame-support and use historic versions and commit hashes and check that the output+return value is as expected.

We can then use these tests to prevent regressions (situations where we break something that already worked).

ggwpez commented 3 months ago

Alternatively it is also possible to just start with simple unit tests about any functionality. Problem is that this project currently has zero tests.