spaghettidba / WorkloadTools

A collection of tools to collect, analyze and replay SQL Server workloads, on premises and in the cloud
MIT License
230 stars 51 forks source link

Replay on AzureDB hangs #154

Open marceloadade opened 1 week ago

marceloadade commented 1 week ago

Thank you for writing this tool and making it available to us. I have been able to Replay workloads with no issues on on Prem Instances and on VMs in Azure.

When I try to replay the workloads in an Azure DB, I get connected with no issues and some commands get executed. However, it reaches a point where the Replay consumer simply hangs and stops producing outputs. In one execution I left it executing for more than 12 hours with no success. When executing against an exact copy of the Database but on premisses, it completes in less than 5 minutes.

Any suggestions, ways to log in which command the tool is at, parameters that could be changed?

image

A few examples are in the images below.

Thank you again.

image

image

spaghettidba commented 1 week ago

Hi Marcelo. Do you have any logs you can share with me? You can also email the logs to me if you don't want to upload them here publicly.

marceloadade commented 1 week ago

Thanks for the response Gianluca! I sent you an e-mail with more details. We can add any other relevant information here afterwards to help other folks that may run into a similar issue as well. (used the spaghettidba@sqlconsulting.it e-mail).

marceloadade commented 1 week ago

Hi Gianluca, adding more details to this issue: In all executions of the Replay I did have the same XEvents running. After all of them, I created SQLITE files for further analysis.

It looks like all commands are executed about twice in Azure, compared to the source events files (which is an Azure SQL DB BTW), the number of events is the same when executing in a local Instance (no issues).

Executing the following query in SQLITE:

select sql_text,count (*) from events 
where sql_text not like 'exec sp_reset_connection%'
group by sql_text
order by 2 desc;

Replay execution in Azure: image

Replay execution Local: image

Source: image

spaghettidba commented 1 day ago

Hey Marcelo, this is really weird. I'm going to have to take a look at the code. I really can't see how queries are executed twice, but of course that could be a bug. Let me get back to you on that.