lni / dragonboat

A feature complete and high performance multi-group Raft library in Go.
Apache License 2.0
5.09k stars 542 forks source link

Will there be any support for Mock Dragonboat Server to be used in the integration test ? #344

Closed abhishektakale closed 6 months ago

abhishektakale commented 9 months ago

So the problem we are facing with dragonboat is that, whenever we are running integration tests that require a Dragonboat instance to be running, we have to start the dragonboat nodehost at the start of each test. That results in around 4 minutes of execution time.

We tried using another approach for the same, i.e., writing a TestMain test and running Setup() and Breakdown(). But we observed that previously tests that took around 3s of time, now they were taking around 10s.

It would be very helpful if a mock server is there that will make things easy. Similar to what Etcdserver provides. In case we are missing something and it is already available, please guide me on the correct path.

Thank you so much in advance.

kevburnsjr commented 9 months ago

My integration tests run in < 20 seconds when creating, deleting and joining multiple shards and replicas across a dozen hosts. https://github.com/logbn/zongzi/blob/main/agent_integration_test.go

It sounds like you need to refactor your tests to share a single cluster rather than creating a new cluster for each test.

lni commented 6 months ago

@abhishektakale

thanks for raising the issue. there is already good infrastructure available in dragonboat for what you want. in short, the long delay you experienced is caused by disk IOs, or fsync operations more specifically. There is already a memory based virtual fs available to allow you to bypass all those slow physical IOs in your tests, basically it is a pebble vfs implementation.

please have a look at internal/vfs/memfs.go and see how it is used.