posvyatokum
commented
7 months ago 2024-01-31 Meeting notes CC: @marcelo-gonzalez @gmilescu @posvyatokum
We are again focusing our attention on the forknet. Forknet MVP would support:
- CI correctness testing of nearcore master
- correctness and load testing of custom binaries on demand
We have developed two toolboxes for mocknet: Marcelo's mirror tools, and Vlad's forknet tools. We need to combine them into a working solution that hides away most of the complexity from the users, while providing enough flexibility.
To achieve this we will:
- utilize Vlad's scripts to make creation of a custom network with a slice of mainnet traffic and arbitrary additional load an easy and cheap process
- utilize Marcelo's neard_runner.py and mirror.py scripts to handle different test scenarios
For a feature developer the flow of using forknet will look like this:
- Run github action to create an image with desired traffic slice, if needed.
- Run github action to create a network with desired setup, slice of traffic, and additional load.
- Implement supporting functions in
neard_runner.py, if needed (start/stop/switch binary/update binary/update config are already implemented) - Implement and execute test scenarios using
mirror.py. This script uploads and startsneard_runner.pyon your forknet cluster. You then can manipulate nodes however you want.
CI testing using forknet will use the same flow automatically. New traffic slice image will be created monthly. New forknet instance will be created/destroyed weekly. Flow of the test will simply start every node with the latest master binary.
Our first goal is a stable setup for forknet CI with one of existing traffic slices.
marcelo-gonzalez
Goals
The toolbox of infrastructures we have for creating and managing test mock networks is large and versatile, with Forknet and mirror tool standing out as being the most powerful. The goal of this project is to continue the development of Forknet by unifying it with the mirror test infrastructure. The two have a lot in common, but some parts (e.g. control plane) are done differently due to the different design philosophies. By merging them together we will simplify toolbox and will better utilise it.
Background
We had several tests (regular betanet, and on-demand spoon test) that were creating mocknet with simple predefined traffic. We wanted to have a way to make traffic more meaningful, so we developed a mirror toolset – a way to test binary (or binary release) via mocknet with a real traffic slice from mainnet. This tool was utilized only during releases, and only by the Node team. And it was not very easy to do.
Independently, we also developed a toolbox to speed up creation of mainnet forks for testing (Forknet). Now, we want to combine fast test setup with working traffic mirroring, to create a better testing system. In order for it to be as useful as it can be, we are also focusing on user-friendliness.
Context
The project is split into three pillars: Correctness, Performance and Infrastructure.
Why should NEAR One work on this
Short term value:
Medium term value:
Long term value:
What needs to be accomplished
Correctness goal:
How we will do that:
Performance goals:
How we will do that:
Infrastructure goals:
How we will do that:
Links to external documentations and discussions
Assumptions
N/A
Pre-requisites
N/A
Out of scope
Custom test flow development is out of scope. If some feature need specific network orchestration to be properly tested, we expect the feature engineer to write the orchestration script using provided tools and examples.
Task list:
Roadmap
Real life progress