Closed TimoGlastra closed 6 months ago
berendsliedrecht
commented
7 months ago @berendsliedrecht do you have an idea why this suddenly fails?
Hmmm, not too sure. Seems like gyp broke something again... maybe a lockfile update or newer node version?
genaris
commented
7 months ago @berendsliedrecht do you have an idea why this suddenly fails?
Hmmm, not too sure. Seems like gyp broke something again... maybe a lockfile update or newer node version?
It seems that the CI image is now installing python 3.12, where distutils has been removed (and that one is referenced by node-gyp). When I ran into this issue locally I solved it by installing setuptools python package (e.g. python3 -m pip install setuptools) but it seems that it might be also fixed by updating the node-gyp version (currently it is in 9.x where there are 10.x that apparently are compatible)
swcurran
commented
7 months ago We’ve asked @amanji on the BC Gov team to understand the release process for the shared components. How do we do a release such that all the components are released — crate and wrappers. Right now we don’t understand it, and we have a number of people frustrated when a new release of the Rust component does not include the wrappers.
Is there any documentation for how this is done? It should include what the triggers are for the release process, and what manual steps (testing/updating) are needed during the process.
@andrewwhitehead @TimoGlastra @berendsliedrecht @genaris — would love to get your help in documenting this — and in getting as much of it automated as possible.
Thanks!
berendsliedrecht
commented
6 months ago (apologies for the late response!)
We’ve asked @amanji on the BC Gov team to understand the release process for the shared components. How do we do a release such that all the components are released — crate and wrappers. Right now we don’t understand it, and we have a number of people frustrated when a new release of the Rust component does not include the wrappers.
Currently, we have 4 different releases. A crates.io release (rust), a library release on GitHub (used for the wrappers), A PyPi release (Python) and an npm release (JavaScript).
In my opinion, these should be separate events and not combined to release everything. It is possible to create an action that releases all the packages, but as the versions are not tied to each other, it does not make sense to release all of them at the same time all the time. Versions not being tied to each other is done on purpose, so wrappers can evolve individually and bug fixes do not cause useless increments in versions of other packages.
Is there any documentation for how this is done? It should include what the triggers are for the release process, and what manual steps (testing/updating) are needed during the process.
There is no official documentation, but I can write out of we have done it in the past and if there is anyone in the BC Gov team, @amanji :), that could streamline this, that would be awesome and I'd be happy to help. (all this also applies to anoncreds-rs and indy-vdr).
Step 0: create a pull request with the correct version increments for the libraries that need releasing. (this commit)[https://github.com/hyperledger/aries-askar/pull/218/files] gives an idea of how we do this. (Automating this would be great, but since there is no version-sync it is very very difficult and not worth the effort I'd say).
First, for the crates.io release there is no CI step. This must be added and is relatively simple. Right now, I think @andrewwhitehead, or someone else, publishes it locally.
Second, to release the libraries on GH, a workflow can be started manually in the GitHub UI with the checkbox Publish Binaries to Release (will create a release if no release exists for branch or tag) enabled. This makes a release on GitHub and adds all the libraries required for the wrappers or manual installation.
Third, the same (or a new workflow), can be manually started with the Publish Python Wrapper to Registries checkbox enabled. This publishes the Python wrapper. Make sure that the version is incremented. If I am not mistaken, this will always release with the latest state of main for the libraries of aries-askar, it is not tied to a version.
Lastly, for the JavaScript wrappers, the same (or a new workflow), can be manually started with the Publish JavaScript Wrapper to Registries enabled. Same as with the Python wrapper, make sure that the version (and library version of aries-askar) is incremented as shown in the PR linked above.
If we want to automate as much as possible, I would look into tools that do version increments for Rust, Python and JavaScript individually. With this, manually start each workflow and when selecting to publish the JS wrapper for example, we also specify to update the package version on the major, minor or patch version. Also, we should be able to specify a new askar library version. This can just be manual text input and we would just type in a tag released on github (v3.0.1 for example). If nothing is specified, the same library version as before will be used. For the version increment, there are many tools out there, for the library version increment, we would likely need to create some sed/awk/grep/jq script to update it.
For Python, crates.io and GitHub library releases it would all be same, but skipping the library version increase. We would just have a dropdown menu with: major, minor and patch and based on that the version will increment.
I hope my rambling makes some sense! Feel free to bombard me with questions, we have also noticed that the release process is not really smooth but we just never had the time to go through and fix it all.
swcurran
commented
6 months ago @berendsliedrecht — thanks for the summary. Awesome.
I’m going to try to summarize your summary. This applies to each of the Rust based shared components (Askar, VDR, AnonCreds-RS and Shared-RS while we have it:
Is that right? We really need to take this into an issue and a document.
@berendsliedrecht do you have an idea why this suddenly fails?