spaghettidba / WorkloadTools

A collection of tools to collect, analyze and replay SQL Server workloads, on premises and in the cloud
MIT License
230 stars 51 forks source link

WorkloadFileWriterConsumer not capturing expected data #152

Open JljHook opened 3 months ago

JljHook commented 3 months ago

Hi, We have extended events listener and a SQLlite consumer: { "Controller": {

    "Listener":
    {
        "__type": "ExtendedEventsWorkloadListener",
        "Source": "C:\\Workloadtools\\sqlworkload_filtered_internal.sql",
         "ConnectionInfo":
        {
            "ServerName": "[server.com](http://server.com/),3342",
            "UserName": "xxxx",
            "Password": "yyyyy"
        },
        "DatabaseFilter": "xxxxxx"
    },
    "Consumers":
    [
        {
            "__type": "WorkloadFileWriterConsumer",
            "OutputFile": "C:\\Workloadtools\\sqlworkloads3.sqlite"
        }
    ]

}

}

I noticed that the extended event session is capturing a lot of queries and the tool is capturing thousands of events(screenshot). When I close the capture with Ctrl+c I get a pink error which is something new. Also when I open SQLlite database there are only few queries in the events usually under 100 events stored. I've tried to capture queries for 10-20 minutes periods. Additionally file size is very small(60-85KB). image image

spaghettidba commented 2 months ago

Hi Joona, sorry for the delay, I had quite a few health issues. It looks very similar to a problem that was fixed recently. Are you using the latest available release?

JljHook commented 2 months ago

Hi, I was using latest version. I tried the capture multiple times and most of the capture data was missing.

JljHook commented 2 months ago

Here's the error: 2024-03-25 19:13:45.8878 - WorkloadTools.Listener.ExtendedEvents.ExtendedEventsWorkloadListener - The shutdown workflow generated a warning: Microsoft.SqlServer.XEvent.Linq.EventEnumerationException: An exception occurred during event enumeration. Examine the inner exception for more information. ---> System.Data.SqlClient.SqlException: The Extended Events session named "sqlworkload" has either been stopped or dropped and can no longer be accessed. at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action1 wrapCloseInAction) at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose) at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady) at System.Data.SqlClient.SqlDataReader.TryHasMoreRows(Boolean& moreRows) at System.Data.SqlClient.SqlDataReader.TryReadInternal(Boolean setTimeout, Boolean& more) at System.Data.SqlClient.SqlDataReader.Read() at Microsoft.SqlServer.XEvent.Linq.Internal.XEventSqlStreamProvider1.ExecuteReader() --- End of inner exception stack trace --- at Microsoft.SqlServer.XEvent.Linq.Internal.XEventEnumerator.MoveNext() at WorkloadTools.Listener.ExtendedEvents.StreamXEventDataReader.ReadEvents() at WorkloadTools.Listener.ExtendedEvents.ExtendedEventsWorkloadListener.ReadEvents() 2024-03-25 19:19:10.2662 - WorkloadTools.Listener.ExtendedEvents.ExtendedEventsWorkloadListener - The shutdown workflow generated a warning: Microsoft.SqlServer.XEvent.Linq.EventEnumerationException: An exception occurred during event enumeration. Examine the inner exception for more information. ---> System.Data.SqlClient.SqlException: The Extended Events session named "sqlworkload" has either been stopped or dropped and can no longer be accessed. at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action1 wrapCloseInAction) at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose) at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady) at System.Data.SqlClient.SqlDataReader.TryHasMoreRows(Boolean& moreRows) at System.Data.SqlClient.SqlDataReader.TryReadInternal(Boolean setTimeout, Boolean& more) at System.Data.SqlClient.SqlDataReader.Read() at Microsoft.SqlServer.XEvent.Linq.Internal.XEventSqlStreamProvider1.ExecuteReader() --- End of inner exception stack trace --- at Microsoft.SqlServer.XEvent.Linq.Internal.XEventEnumerator.MoveNext() at WorkloadTools.Listener.ExtendedEvents.StreamXEventDataReader.ReadEvents() at WorkloadTools.Listener.ExtendedEvents.ExtendedEventsWorkloadListener.ReadEvents()