This patch adds an "accurate" replay mode.
In this mode we try to preserve the relative start time of every query as best as possible - we will not replay as fast as possible but instead preserve the pauses between queries.
This is useful when one does not want to test peak throughput but instead the real load the database had.
Unfurtunately this mode requires us to preprocess the slowlog and keep the queries in memory.
I reduced the memory consumption by read only mmaping the slow query log (so that the OS can page it/out as required)
and keeping only minimal data about every query in memory and reading most data lazily from the memory mapped file.
The query start time is calculated by subtracting the "Query_time" from the end timestamp we find inside the "# Time:" metadata.
We then order all queries based on the timstamp (and for queries from the same connection we preserve the order in which they appear inside the slowlog).
After that we calculate the difference between the current time and the one of the first recorded query and use it as an offset for all further queries.
Before executing any query we wait until the recorded timestamp rebased by the offset is reached.
In addition this patch makes sure that every connection/thread gets shutdown when the last query of the connection got executed (in thread-per-connection mode).
Enabling percona servers "slow_query_log_timestamp_precision" helps making the replay more accurate.
This patch adds an "accurate" replay mode. In this mode we try to preserve the relative start time of every query as best as possible - we will not replay as fast as possible but instead preserve the pauses between queries. This is useful when one does not want to test peak throughput but instead the real load the database had.
Unfurtunately this mode requires us to preprocess the slowlog and keep the queries in memory. I reduced the memory consumption by read only mmaping the slow query log (so that the OS can page it/out as required) and keeping only minimal data about every query in memory and reading most data lazily from the memory mapped file.
The query start time is calculated by subtracting the "Query_time" from the end timestamp we find inside the "# Time:" metadata. We then order all queries based on the timstamp (and for queries from the same connection we preserve the order in which they appear inside the slowlog). After that we calculate the difference between the current time and the one of the first recorded query and use it as an offset for all further queries. Before executing any query we wait until the recorded timestamp rebased by the offset is reached.
In addition this patch makes sure that every connection/thread gets shutdown when the last query of the connection got executed (in thread-per-connection mode). Enabling percona servers "slow_query_log_timestamp_precision" helps making the replay more accurate.