search

spaghettidba / WorkloadTools

A collection of tools to collect, analyze and replay SQL Server workloads, on premises and in the cloud
MIT License
234 stars 53 forks source link

01/01/0001 00:00:00 oldest command date #49

Closed ghotz closed 5 years ago

ghotz commented 5 years ago

During the replay sometimes an error similar to the following is logged:

2019-04-12 01:33:14.4887 - Info - WorkloadTools.Consumer.Replay.ReplayConsumer : 01/01/0001 00:00:00 oldest command date

The error is risen periodically and after a while the tool completely stops replaying and just logs this error (as if there's nothing to replay).

More info:

  1. 16 CPUs on the replaying server all at 100% usage
  2. 500 workers configured in Workloadtools, around 300 used
  3. many errors during replay (e.g. trying inserting NULLs, missing procedures)
spaghettidba commented 5 years ago

Seems like the target server can't keep up with the workload. Have you tried capturing sp_whoIsActive to see whether some queries are being replayed and got stuck?

ghotz commented 5 years ago

Yes, queries are regularly stuck waiting for CPUs or PFSs on tempdb, but is expected since the destination system run constantly at 100% with 8 cores (the original workload has 32 cores). Still, the message should output a proper date and the real problem is that it stops sending commands to the Replay Consumer even tough it's collecting them and writing them to the Analysis Consumer.