celestiaorg / celestia-app

Celestia consensus node

https://celestiaorg.github.io/celestia-app/

Apache License 2.0

342 stars 281 forks source link

Refocusing big block tests on 8MB block sizes and simplifying maintenance #3865

Open staheri14 opened 3 weeks ago

staheri14 commented 3 weeks ago

Problem

Initially, for the big block test, the plan was to run e2e benchmark tests for block sizes of 8, 32, and 64MB. However, the focus later shifted to 8MB blocks. While the current constants in celestia-app are compatible with 8MB blocks, modifying those constants was necessary to produce larger blocks, which led to this PR, based on which the e2e big block tests are run. Since the current focus is on 8MB blocks, maintaining that branch is no longer necessary, so

Acceptance Criteria

[ ] I'd suggest closing this https://github.com/celestiaorg/celestia-app/pull/3261
[x] The e2e benchmark tests will be updated to use the latest from the main branch. This simplifies maintenance and reduces the workload.
[x] As a result, other benchmark tests for block sizes of 32 and 64MB will also be removed until a concrete plan is developed for those, and the relevant constants are introduced to celestia-app.

evan-forbes commented 2 weeks ago

atm, knuu isn't suitable for reliable network experiments. There was discussion of adding this functionality, but nothing has been planned yet. Presumably this would look very similar to what congest does.

If this functionality is being planned for, then I think we can confidently continue to maintain these experiment setups. If we don't have plans to add this functionality to knuu, it might be better to focus on infrastructure that can do this so that we have a reliable network setup.

alternative to using congest, we could alternatively abstract some of the functionality of the network test setup code here so that it works both for things like knuu and pulumi

staheri14 commented 2 weeks ago

@evan-forbes Thanks for your insights, then are you proposing to hold off on removing the e2e benchmark tests targeting the larger block sizes i.e., this issue?

evan-forbes commented 2 weeks ago

are you proposing to hold off on removing the e2e benchmark tests targeting the larger block sizes i.e., this issue?

If we are going to have network tests, then I think that we need the option for bigger block sizes, but in that comment I was unsure if we were having knuu based network tests.

we had a sync today to discuss this, and we're investigating how much effort it would be to develop a pulumi / ansible esque backend for knuu so that we could have network tests that use the same code as our e2e tests