Speed up Tests

[jsdw]

Tests (particularly the integration tests I think) take >20mins to complete; can we speed this up? Does cargo nextest help at all (did we try it and find it didn't?). Any other way to parallelise tests? Worth running them on Gitlab or something instead? It'd be amazing if we could get this to under 5 mins :)

[jsdw]

https://github.com/paritytech/subxt/pull/928 really helped and has sped things up a fair bit.

Other things we can investigate:

• Fewer UI tests; we generate a UI test to generate every pallet individually. Do we need them all? Might be better to test specific pallets instead. This should be a fairly easy win at the mo, since UI tests can take longest.
• We use the #[subxt] macro a lot. How long does it take per invocation? Can we speed up most of our tests/examples by pointed at a stripped down version? Maybe a artifacts/polkadot_metadata_tiny.scale one with no pallets in that can be used anywhere we don't need any pallet logic. Maybe also a artifacts/polkadot_metadata_small.scale for examples that only need a few pallets in. Maybe rename the other to polkadot_metadata_full.scale for consistency, if it's still actually used somewhere. (add readme to artifacts folder to describe exactly what is in each file).
• Try cargo nextest again; with faster machines, maybe we can get more speedup running tests with this? Maybe we can merge test runs and rely more on this (plus possibly a more powerful machine) to just run everything in one place in parallel?

Probably more painful and less wins:

• Is there a way to cache the output from the #[subxt] macro when inputs are the same?
• Is there any win from sharing a Substrate instance in some of the integration tests rather than spinning up a new one for every test? Probably a fair difficult thing to try; need to ensure tests use unique accounts for instance (not just alice any bob) and only applicable to tests that can be run alongside others without any conflict.

[tadeohepperle]

@jsdw I found out that invocation of the subxt macro takes about 110ms on my machine for the polkadot.scale metadata that is 346KB in size. I created a stripped down version of the default substrate metadata (without any pallets) that is just 6.43KB in size. It reduces the time of macro invocation to about 2.5ms. This is a pretty linear speed up (50x smaller -> 50x faster). Hoverever this is just the macro invocation which is, (just my guess) probably a lot times faster than the actual compilation of the generated code.