Container dies in simultaneously queries

[zehmigueljr]

Hello,

I have a scenario of an application that executes some queries simultaneously within a panel. Running the application in a container, after a few requests, the container dies. When debugging locally, the debug does not die, but some exceptions from the DuckDB lib occur. The error message is "The external component threw an exception."

Here are some stacks of the error I captured locally:

in DuckDB.NET.NativeMethods.PreparedStatements.DuckDBExecutePrepared(DuckDBPreparedStatement preparedStatement, DuckDBResult& result) in DuckDB.NET.Data.PreparedStatement.Execute(DuckDBParameterCollection parameterCollection) in DuckDB.NET.Data.PreparedStatement.PrepareMultiple(DuckDBNativeConnection connection, String query, DuckDBParameterCollection parameters) --- End of stack trace from previous location --- in DuckDB.NET.Data.PreparedStatement.PrepareMultiple(DuckDBNativeConnection connection, String query, DuckDBParameterCollection parameters) in DuckDB.NET.Data.DuckDbCommand.ExecuteNonQuery() in UpFlux.Data.Olap.EventLogOlapRepository.CreateTempDataTable(String processId, SqlBuilder builder) em C:\Users\MiguelSilveiraJunior\Desktop\source\upflux\upflux-platform-v4\src\UpFlux.Data\Olap\EventLogOlapRepository.cs:linha 124 in UpFlux.Data.Olap.EventLogOlapRepository.ExecuteHistogramLoader(SqlBuilder parentQuery, KpiComponent component, ProcessConfiguration config) em C:\Users\MiguelSilveiraJunior\Desktop\source\upflux\upflux-platform-v4\src\UpFlux.Data\Olap\EventLogOlapRepository.cs:linha 401 in UpFlux.Service.Implementation.OLAP.SqlAnalysisViewService.ProcessComponent(SqlBuilder builder, KpiContext kpiContext, KpiComponent component, ProcessConfiguration processConfiguration, String notificationGroupName, String tableName) em C:\Users\MiguelSilveiraJunior\Desktop\source\upflux\upflux-platform-v4\src\UpFlux.Service\Implementation\OLAP\SqlAnalysisViewService.cs:linha 131

Second stack:

 in DuckDB.NET.NativeMethods.PreparedStatements.DuckDBExecutePrepared(DuckDBPreparedStatement preparedStatement, DuckDBResult& result)

in DuckDB.NET.Data.PreparedStatement.Execute(DuckDBParameterCollection parameterCollection) in DuckDB.NET.Data.PreparedStatement.PrepareMultiple(DuckDBNativeConnection connection, String query, DuckDBParameterCollection parameters) --- End of stack trace from previous location --- in DuckDB.NET.Data.PreparedStatement.PrepareMultiple(DuckDBNativeConnection connection, String query, DuckDBParameterCollection parameters) in DuckDB.NET.Data.DuckDbCommand.ExecuteDbDataReader(CommandBehavior behavior) in DuckDB.NET.Data.DuckDbCommand.ExecuteScalar() in UpFlux.Data.Olap.EventLogOlapRepository.ExecuteScalar[T](String processId, SqlBuilder builder) em C:\Users\MiguelSilveiraJunior\Desktop\source\upflux\upflux-platform-v4\src\UpFlux.Data\Olap\EventLogOlapRepository.cs:linha 884 in UpFlux.Data.Olap.EventLogOlapRepository.GetMaxValue(String processId, SqlBuilder query, KpiHistogram histogram) em C:\Users\MiguelSilveiraJunior\Desktop\source\upflux\upflux-platform-v4\src\UpFlux.Data\Olap\EventLogOlapRepository.cs:linha 928 in UpFlux.Data.Olap.EventLogOlapRepository.ExecuteHistogramLoader(SqlBuilder parentQuery, KpiComponent component, ProcessConfiguration config) em C:\Users\MiguelSilveiraJunior\Desktop\source\upflux\upflux-platform-v4\src\UpFlux.Data\Olap\EventLogOlapRepository.cs:linha 403 in UpFlux.Service.Implementation.OLAP.SqlAnalysisViewService.ProcessComponent(SqlBuilder builder, KpiContext kpiContext, KpiComponent component, ProcessConfiguration processConfiguration, String notificationGroupName, String tableName) em C:\Users\MiguelSilveiraJunior\Desktop\source\upflux\upflux-platform-v4\src\UpFlux.Service\Implementation\OLAP\SqlAnalysisViewService.cs:linha 131

Can you tell me what could be happening?

Initially, the DuckDBExecutePrepared method is being called on both stacks and is the central point of the exception.

Thanks.

[zehmigueljr]

Hi, I created a code example of the problem you are experiencing. First it throws a SHEException exeption, we handle this error with retry, and then it throws an AggregateException exeption, locally it does not kill the debug instance, which is a web instance, but if it is in a container it kills the container.

Link -> https://github.com/zehmigueljr/DuckDbError

[Giorgi]

The connection object isn't thread safe. You should create a new connection for each thread.

[zehmigueljr]

Ok, I tested this code I made for testing and it really worked, but it raised a question, should this exception that it throws in this situation kill the container?

[Giorgi]

If the container is killed by unhandled exception, I can't do anything about it.

[zehmigueljr]

Ok, Thanks @Giorgi