Restructure Parachains Integration Tests (Emulated)

muharem commented 1 year ago

Context:

The current structure for the tests based on xcm-emulator under the parachains/integration-tests/emulated/ directory, ./common - setup for all networks (polkadot, kusama, etc) hence imports all runtimes ./%parachain% - tests crate per parachain which imports common

Problem:

every tests imports all the runtimes which results a long compilation process while developing locally
one common for all networks setup results long names and inconvenient navigation through the xcm-emulator setup

Proposal 1

Per crate per network with all tests cases structured via modules, the xcm-emulator setup located in crate's lib.rs, something like parachains/integration-tests/emulated/: ./polakdot ./polakdot/src/lib.rs - xcm-emulator setup moved from common, only the part concerning Polkadot network ./polakdot/src/assethub - parachain based module ./polakdot/src/treasury - feature based module ... ./kusama ...

Proposal 2

Split parachains/integration-tests/emulated/common into parachains/integration-tests/emulated/network/[polkadot|kusama|...].

muharem commented 1 year ago

@bkontur @NachoPal @gilescope @joepetrowski

NachoPal commented 1 year ago

I would prefer Proposal 2. It would solve the problem without touching anything else. I like our current directory tree structure which follows parachains/runtimes directory.

In Proposal 1 you are overlooking we will have BridgesHub integrations tests and therefore we'll need runtimes from different networks. If we have one crate per network we would have to import anyway other network crate with its runtimes.

There is also another alternative which is this one: https://github.com/paritytech/cumulus/pull/2752 I would like to see it merged. It could be used also for integration tests.

muharem commented 1 year ago

@NachoPal good point about Bridges

NachoPal commented 1 year ago

If you agree, what I can do is to split parachains/integration-tests/common into:

parachains/integration-tests/common/common/ - integration-tests-common (stuff common to all networks)
parachains/integration-tests/common/polkadot/ - integration-tests-common-polkadot
parachains/integration-tests/common/kusama/ - integration-tests-common-kusama
parachains/integration-tests/common/westend/ - integration-tests-common-westend
parachains/integration-tests/common/rococo/ - integration-tests-common-rococo

Then, each parachain will have to import only integration-tests-common and its respective common network crate.

Only BridgeHub parachains will have the need of importing more than one common network crate.

muharem commented 1 year ago

yeah, sounds good to me

bkontur commented 1 year ago

If you agree, what I can do is to split parachains/integration-tests/common into:
* `parachains/integration-tests/common/common/` - `integration-tests-common` (stuff common to all networks)

* `parachains/integration-tests/common/polkadot/` - `integration-tests-common-polkadot`

* `parachains/integration-tests/common/kusama/` - `integration-tests-common-kusama`

* `parachains/integration-tests/common/westend/` - `integration-tests-common-westend`

* `parachains/integration-tests/common/rococo/` - `integration-tests-common-rococo`
Then, each parachain will have to import only integration-tests-common and its respective common network crate.

Only BridgeHub parachains will have the need of importing more than one common network crate.

I am finally coming to adding some integration tests, but this looks reasonable

@NachoPal are you going to change this in your big PR?

NachoPal commented 1 year ago

@NachoPal are you going to change this in your big PR?

No, I will open a new one.

paritytech / cumulus