Proper black-box integration tests MVP

hyperledger-iroha / iroha

Iroha - A simple, enterprise-grade decentralized ledger

https://wiki.hyperledger.org/display/iroha

Apache License 2.0

443 stars 278 forks source link

Proper black-box integration tests MVP #4500

Closed mversic closed 1 month ago

mversic commented 7 months ago

We've long had an issue that our client/tests/integration are not proper black box tests. This means the have iroha as dependency when they shouldn't.

The main goal is to remove iroha and test_network as dev-dependencies of iroha_client. However, the scope of this task is to produce an MVP with only several tests migrated to this new scheme of running integration tests. Later we can migrate all of the tests, but it's important to provide a framework for new tests first.

We should use this to bring up dockers running iroha for every test.

nxsaken commented 6 months ago

Questions on the Dockerfile I struggle to find answers to:

Why do we use archlinux as the base image for the build stage?
- this probably isn't that simple, but can't we use something that supports arm64, like alpine?
What difference does adding rust-src to the toolchain make?
Can we get away with only installing the minimal toolchain (without rustfmt, clippy and rust-docs)?
- asking because specifying the image to be amd64/archlinux (for testing reasons) makes rustup fail to install rust-docs for some reason

Erigara commented 6 months ago

Why do we use archlinux as the base image for the build stage?

Afaik it was chosen just because it's easy to install rust, so imo we can replace it.

0x009922 commented 6 months ago

For the reference, iroha-javascript has an internal utility for end-to-end testing: test-peer. It sets up actual iroha binary & clean environment for each test, and tests with it work real fast, surprisingly. It has direct library interface and even a simple stateful HTTP server to control peer execution state.

0x009922 commented 5 months ago

We should use this to bring up dockers running iroha for every test.

Do we really need testcontainers for this purpose? While trying to integrate it in tests of iroha-javascript, I found it quite heavy and excessive in comparison to plain child-process execution. Using Docker in tests also opens up an infinite hole of possible Docker-related problems which are usually very hard to debug.

dmitrivenger commented 1 month ago

@0x009922 can we move this task to "Done"?