scylladb / gocql

Package gocql implements a fast and robust ScyllaDB client for the Go programming language.
https://docs.scylladb.com/stable/using-scylla/drivers/cql-drivers/scylla-go-driver.html
BSD 3-Clause "New" or "Revised" License
190 stars 59 forks source link

Implement single connection benchmark #308

Closed sylwiaszunejko closed 2 weeks ago

sylwiaszunejko commented 1 month ago

API for hooking to Write() and Read() of Conn struct added. New struct TrafficRecorder added for enabling recording and mocking single Connection flow.

Fixes: #296

sylwiaszunejko commented 1 month ago

Adding this as a draft, because I wanted to make sure this part looks like intended in the issue, before pushing the rest.

sylwiaszunejko commented 1 month ago

@dkropachev could you take a look?

sylwiaszunejko commented 1 month ago

@dkropachev I pushed some changes, it is clearly wip, but I want to make sure the direction is right

sylwiaszunejko commented 1 month ago

@dkropachev I pushed some changes, it is still not a complete solution, I am not sure how to replay the traffic properly, but it is in such a state that it should be possible to state if the direction is good

dkropachev commented 2 weeks ago

@sylwiaszunejko , we need to disable these guys for NewSingleHostQueryExecutor

        s.hostSource = &ringDescriber{session: s}
        s.ringRefresher = newRefreshDebouncer(ringRefreshDebounceTime, func() error { return refreshRing(s.hostSource) })
dkropachev commented 2 weeks ago

@sylwiaszunejko , I spot another problem, standard benchrmark library does not collect goroutine stats, which means that some parts are not shown in results. Unfortunately due to the library limitations, there is only one way to fix it is to come up with custom test library, which is a bit cumbersome. Given all that I think it is better to merge it and look for solution that would show everything later, WDYT?

sylwiaszunejko commented 2 weeks ago

@sylwiaszunejko , I spot another problem, standard benchrmark library does not collect goroutine stats, which means that some parts are not shown in results. Unfortunately due to the library limitations, there is only one way to fix it is to come up with custom test library, which is a bit cumbersome. Given all that I think it is better to merge it and look for solution that would show everything later, WDYT?

I agree

sylwiaszunejko commented 2 weeks ago

I created an issue to collect goroutine stats https://github.com/scylladb/gocql/issues/330