Stream blocks during import

[Alenar]

Content

This PR changes the way the blocks are retrieved from the BlockScanner. Instead of giving the full list of blocks from the given cardano database it yields now a BlockStreamer trait that allow to poll the blocks at a pace given by it's implementation (the ImmutableBlockStreamer yield them immutables per immutables).

This reduce the memory load needed during the import process.

The BlockStreamer trait has been designed with the transition to the chain_sync protocol using pallas in mind. As such it sadly don't implement the rust Iterator trait since it needs to be async.

Note: Since mithril-common/src/cardano_block_scanner.rs would become quite huge with the new types introduced, it has been promoted to a module directory instead and it's types spread out to dedicated files.

Pre-submit checklist

• Branch
  • [x] Tests are provided (if possible)
  • [x] Crates versions are updated (if relevant)
  • [ ] CHANGELOG file is updated (if relevant)
  • [x] Commit sequence broadly makes sense
  • [x] Key commits have useful messages
• PR
  • [x] No clippy warnings in the CI
  • [x] Self-reviewed the diff
  • [x] Useful pull request description
  • [x] Reviewer requested
• Documentation
  • [ ] Update README file (if relevant)
  • [ ] Update documentation website (if relevant)
  • [ ] Add dev blog post (if relevant)

Comments

This PR should be merged after #1650 since its branch is based on that PR code.

Issue(s)

Closes #1646

[github-actions[bot]]

Test Results

    3 files  ±0     43 suites  ±0   8m 23s :stopwatch: -1s   976 tests +4    976 :white_check_mark: +4  0 :zzz: ±0  0 :x: ±0  1 070 runs  +4  1 070 :white_check_mark: +4  0 :zzz: ±0  0 :x: ±0 

Results for commit e32d61b5. ± Comparison against base commit c0a7b67b.

This pull request removes 7 and adds 11 tests. Note that renamed tests count towards both.

``` mithril-common ‑ cardano_block_scanner::tests::test_instantiate_parser_with_allow_unparsable_block_should_log_warning mithril-common ‑ cardano_block_scanner::tests::test_instantiate_parser_without_allow_unparsable_block_should_not_log_warning mithril-common ‑ cardano_block_scanner::tests::test_parse_expected_number_of_transactions mithril-common ‑ cardano_block_scanner::tests::test_parse_from_lower_bound_until_upper_bound mithril-common ‑ cardano_block_scanner::tests::test_parse_should_error_with_unparsable_block_format mithril-common ‑ cardano_block_scanner::tests::test_parse_should_log_error_with_unparsable_block_format mithril-common ‑ cardano_block_scanner::tests::test_parse_up_to_given_beacon ``` ``` mithril-common ‑ cardano_block_scanner::block_scanner::tests::test_instantiate_parser_with_allow_unparsable_block_should_log_warning mithril-common ‑ cardano_block_scanner::block_scanner::tests::test_instantiate_parser_without_allow_unparsable_block_should_not_log_warning mithril-common ‑ cardano_block_scanner::block_scanner::tests::test_parse_from_lower_bound_until_upper_bound mithril-common ‑ cardano_block_scanner::block_scanner::tests::test_parse_up_to_given_beacon mithril-common ‑ cardano_block_scanner::dumb_block_scanner::tests::dumb_scanned_construct_a_streamer_based_on_its_stored_blocks mithril-common ‑ cardano_block_scanner::dumb_block_scanner::tests::polling_with_multiple_sets_of_blocks_returns_some_once mithril-common ‑ cardano_block_scanner::dumb_block_scanner::tests::polling_with_one_set_of_block_returns_some_once mithril-common ‑ cardano_block_scanner::dumb_block_scanner::tests::polling_without_set_of_block_return_none mithril-common ‑ cardano_block_scanner::immutable_block_streamer::tests::if_allowed_reading_unparsable_block_should_log_an_error mithril-common ‑ cardano_block_scanner::immutable_block_streamer::tests::if_disallowed_reading_unparsable_block_should_fail … ```

:recycle: This comment has been updated with latest results.