search

paritytech / srtool

A fork of chevdor's srtool
MIT License
64 stars 18 forks source link

It would be amazing to be able to run unit tests using the build produced by srtool. #55

Closed brandonpille closed 1 year ago

brandonpille commented 1 year ago

It would also be very handy to be able to run the build inside a container. This now requires docker inside docker which is not always wanted. It would be amazing to have some scripts that we can use to build.

chevdor commented 1 year ago

Hello @brandonpille, could you expand ?

Your proposal sounds out of scope for srtool and nothing prevents you from using the runtime that was produced for anything you like, inlcuding running some tests. Testing a runtime however requires a network and upgrading the runtime on chain.

You may want to look at https://github.com/paritytech/zombienet

brandonpille commented 1 year ago

Our pipeline first builds the runtime, calls cargo test on it and finally uploads the runtime when we want to release it. We would want to use the srtool to generate a deterministic binary. We are currently doing all these steps in a docker container (custom one) so using the srtool would require us to do some steps outside of the container. But it is not allowing us to run the unit tests on them. The srtool is running cargo build with some arguments right? In the same way it would be amazing to be able to run cargo test too (in the container that was used to build the deterministic build).

chevdor commented 1 year ago

Our pipeline first builds the runtime, calls cargo test on it and finally uploads the runtime when we want to release it.

OK

We would want to use the srtool to generate a deterministic binary.

Yep, that's what it is for.

We are currently doing all these steps in a docker container (custom one) so using the srtool

srtool is primarily a docker image and a cli srtool-cli to help calling it.

would require us to do some steps outside of the container.

Yes, anything beyong building the deterministic image would be another topic, where in docker or not.

The srtool is running cargo build with some arguments right?

yes

In the same way it would be amazing to be able to run cargo test too (in the container that was used to build the deterministic build).

Oh I see what you are thinking of but you really don't need and should not use srtool for the tests.

By design, srtool is slow (no caching etc...). You should run your unit tests using standar methods and if you want to do integrationm tests of the runtime, you should do an upgrade and using tests frameworks such as zombienet.

brandonpille commented 1 year ago

Alright. Thank you for the info. We will create a second pipeline that will build using the srtool.