The MARS TDS header contained errors using ASP.NET Core and EF Core connecting to Azure SQL Server

AfsanehR-zz commented 6 years ago

Copied from here:

I have an ASP.NET Core app [Microsoft.AspNetCore.App 2.1.4] with EF Core 2.1.4 [DbContext pooling enabled] and data stored on an Azure SQL database.

Occasionally [once in 1-2 days] I get unrecoverable error stating System.Data.SqlClient.SqlException (0x80131904): The incoming tabular data stream (TDS) protocol stream is incorrect. The MARS TDS header contained errors.

Once I restart the app, everything resumes working until it happens again.

Probably import to note is that I don't have MultipleActiveResultSets enabled in my connection string at all, which makes the issue even more strange.

Has anyone else noticed something similar? Are there any ways I can trace the problem?

Stack trace:

System.Data.SqlClient.SqlException (0x80131904): The incoming tabular data stream (TDS) protocol stream is incorrect. The MARS TDS header contained errors.
   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at System.Data.SqlClient.SqlDataReader.TryConsumeMetaData()
   at System.Data.SqlClient.SqlDataReader.get_MetaData()
   at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
   at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, SqlDataReader ds)
   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean asyncWrite, String method)
   at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior)
   at System.Data.SqlClient.SqlCommand.ExecuteDbDataReader(CommandBehavior behavior)
   at Microsoft.EntityFrameworkCore.Storage.Internal.RelationalCommand.Execute(IRelationalConnection connection, DbCommandMethod executeMethod, IReadOnlyDictionary`2 parameterValues)
   at Microsoft.EntityFrameworkCore.Storage.Internal.RelationalCommand.ExecuteReader(IRelationalConnection connection, IReadOnlyDictionary`2 parameterValues)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable`1.Enumerator.BufferlessMoveNext(DbContext _, Boolean buffer)
   at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteImplementation[TState,TResult](Func`3 operation, Func`3 verifySucceeded, TState state)
   at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable`1.Enumerator.MoveNext()
   at Microsoft.EntityFrameworkCore.Query.Internal.LinqOperatorProvider.ExceptionInterceptor`1.EnumeratorExceptionInterceptor.MoveNext()
   at System.Collections.Generic.List`1.AddEnumerable(IEnumerable`1 enumerable)
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Inviton.Web.Portal.Controllers.EventListController.GetEvents(BL bl, Int32 venueId, Int32 promoterId)

Wraith2 commented 5 years ago

sure, but it's hard to type the word build wrong and they look like real errors to me...

E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\Microsoft.Data.Sqlite.Core\Debug\netstandard2.0\Microsoft.Data.Sqlite.Core.AssemblyInfo.cs(10,12): error CS0579: Duplicate 'System.Reflection.AssemblyCompanyAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\Microsoft.Data.Sqlite.Core\Microsoft.Data.Sqlite.Core.csproj]
obj\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(11,12): error CS0579: Duplicate 'System.Reflection.AssemblyCompanyAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\EFCore.Analyzers\Debug\netstandard1.3\EFCore.Analyzers.AssemblyInfo.cs(14,12): error CS0579: Duplicate 'System.Reflection.AssemblyCompanyAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\EFCore.Analyzers\EFCore.Analyzers.csproj]
obj\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(12,12): error CS0579: Duplicate 'System.Reflection.AssemblyConfigurationAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\Microsoft.Data.Sqlite.Core\Debug\netstandard2.0\Microsoft.Data.Sqlite.Core.AssemblyInfo.cs(11,12): error CS0579: Duplicate 'System.Reflection.AssemblyConfigurationAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\Microsoft.Data.Sqlite.Core\Microsoft.Data.Sqlite.Core.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\EFCore.Analyzers\Debug\netstandard1.3\EFCore.Analyzers.AssemblyInfo.cs(15,12): error CS0579: Duplicate 'System.Reflection.AssemblyConfigurationAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\EFCore.Analyzers\EFCore.Analyzers.csproj]
obj\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(13,12): error CS0579: Duplicate 'System.Reflection.AssemblyCopyrightAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\Microsoft.Data.Sqlite.Core\Debug\netstandard2.0\Microsoft.Data.Sqlite.Core.AssemblyInfo.cs(12,12): error CS0579: Duplicate 'System.Reflection.AssemblyCopyrightAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\Microsoft.Data.Sqlite.Core\Microsoft.Data.Sqlite.Core.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\EFCore.Analyzers\Debug\netstandard1.3\EFCore.Analyzers.AssemblyInfo.cs(16,12): error CS0579: Duplicate 'System.Reflection.AssemblyCopyrightAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\EFCore.Analyzers\EFCore.Analyzers.csproj]
obj\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(14,12): error CS0579: Duplicate 'System.Reflection.AssemblyDescriptionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\Microsoft.Data.Sqlite.Core\Debug\netstandard2.0\Microsoft.Data.Sqlite.Core.AssemblyInfo.cs(13,12): error CS0579: Duplicate 'System.Reflection.AssemblyDescriptionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\Microsoft.Data.Sqlite.Core\Microsoft.Data.Sqlite.Core.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\EFCore.Analyzers\Debug\netstandard1.3\EFCore.Analyzers.AssemblyInfo.cs(17,12): error CS0579: Duplicate 'System.Reflection.AssemblyDescriptionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\EFCore.Analyzers\EFCore.Analyzers.csproj]
obj\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(15,12): error CS0579: Duplicate 'System.Reflection.AssemblyFileVersionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\EFCore.Analyzers\Debug\netstandard1.3\EFCore.Analyzers.AssemblyInfo.cs(18,12): error CS0579: Duplicate 'System.Reflection.AssemblyFileVersionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\EFCore.Analyzers\EFCore.Analyzers.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\Microsoft.Data.Sqlite.Core\Debug\netstandard2.0\Microsoft.Data.Sqlite.Core.AssemblyInfo.cs(24,12): error CS0579: Duplicate 'System.Reflection.AssemblyFileVersionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\Microsoft.Data.Sqlite.Core\Microsoft.Data.Sqlite.Core.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\EFCore.Analyzers\Debug\netstandard1.3\EFCore.Analyzers.AssemblyInfo.cs(19,12): error CS0579: Duplicate 'System.Reflection.AssemblyInformationalVersionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\EFCore.Analyzers\EFCore.Analyzers.csproj]
obj\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(16,12): error CS0579: Duplicate 'System.Reflection.AssemblyInformationalVersionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
obj\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(11,12): error CS0579: Duplicate 'System.Reflection.AssemblyCompanyAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
obj\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(17,12): error CS0579: Duplicate 'System.Reflection.AssemblyProductAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\Microsoft.Data.Sqlite.Core\Debug\netstandard2.0\Microsoft.Data.Sqlite.Core.AssemblyInfo.cs(25,12): error CS0579: Duplicate 'System.Reflection.AssemblyInformationalVersionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\Microsoft.Data.Sqlite.Core\Microsoft.Data.Sqlite.Core.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\EFCore.Analyzers\Debug\netstandard1.3\EFCore.Analyzers.AssemblyInfo.cs(20,12): error CS0579: Duplicate 'System.Reflection.AssemblyProductAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\EFCore.Analyzers\EFCore.Analyzers.csproj]
obj\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(12,12): error CS0579: Duplicate 'System.Reflection.AssemblyConfigurationAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\Microsoft.Data.Sqlite.Core\Debug\netstandard2.0\Microsoft.Data.Sqlite.Core.AssemblyInfo.cs(26,12): error CS0579: Duplicate 'System.Reflection.AssemblyProductAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\Microsoft.Data.Sqlite.Core\Microsoft.Data.Sqlite.Core.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\EFCore.Analyzers\Debug\netstandard1.3\EFCore.Analyzers.AssemblyInfo.cs(21,12): error CS0579: Duplicate 'System.Reflection.AssemblyTitleAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\EFCore.Analyzers\EFCore.Analyzers.csproj]
obj\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(13,12): error CS0579: Duplicate 'System.Reflection.AssemblyCopyrightAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
obj\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(18,12): error CS0579: Duplicate 'System.Reflection.AssemblyTitleAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\Microsoft.Data.Sqlite.Core\Debug\netstandard2.0\Microsoft.Data.Sqlite.Core.AssemblyInfo.cs(27,12): error CS0579: Duplicate 'System.Reflection.AssemblyTitleAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\Microsoft.Data.Sqlite.Core\Microsoft.Data.Sqlite.Core.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\EFCore.Analyzers\Debug\netstandard1.3\EFCore.Analyzers.AssemblyInfo.cs(22,12): error CS0579: Duplicate 'System.Reflection.AssemblyVersionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\EFCore.Analyzers\EFCore.Analyzers.csproj]
obj\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(14,12): error CS0579: Duplicate 'System.Reflection.AssemblyDescriptionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\Microsoft.Data.Sqlite.Core\Debug\netstandard2.0\Microsoft.Data.Sqlite.Core.AssemblyInfo.cs(28,12): error CS0579: Duplicate 'System.Reflection.AssemblyVersionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\Microsoft.Data.Sqlite.Core\Microsoft.Data.Sqlite.Core.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\EFCore.Analyzers\Debug\netstandard1.3\EFCore.Analyzers.AssemblyInfo.cs(23,12): error CS0579: Duplicate 'System.Resources.NeutralResourcesLanguageAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\EFCore.Analyzers\EFCore.Analyzers.csproj]
obj\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(19,12): error CS0579: Duplicate 'System.Reflection.AssemblyVersionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
obj\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(15,12): error CS0579: Duplicate 'System.Reflection.AssemblyFileVersionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\Microsoft.Data.Sqlite.Core\Debug\netstandard2.0\Microsoft.Data.Sqlite.Core.AssemblyInfo.cs(29,12): error CS0579: Duplicate 'System.Resources.NeutralResourcesLanguageAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\Microsoft.Data.Sqlite.Core\Microsoft.Data.Sqlite.Core.csproj]
obj\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(20,12): error CS0579: Duplicate 'System.Resources.NeutralResourcesLanguageAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
obj\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(16,12): error CS0579: Duplicate 'System.Reflection.AssemblyInformationalVersionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
obj\x86\Debug\net461\ef.AssemblyInfo.cs(11,12): error CS0579: Duplicate 'System.Reflection.AssemblyCompanyAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
obj\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(17,12): error CS0579: Duplicate 'System.Reflection.AssemblyProductAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
obj\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(18,12): error CS0579: Duplicate 'System.Reflection.AssemblyTitleAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
obj\x86\Debug\net461\ef.AssemblyInfo.cs(12,12): error CS0579: Duplicate 'System.Reflection.AssemblyConfigurationAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
obj\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(19,12): error CS0579: Duplicate 'System.Reflection.AssemblyVersionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
obj\x86\Debug\net461\ef.AssemblyInfo.cs(13,12): error CS0579: Duplicate 'System.Reflection.AssemblyCopyrightAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
obj\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(20,12): error CS0579: Duplicate 'System.Resources.NeutralResourcesLanguageAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
obj\x86\Debug\net461\ef.AssemblyInfo.cs(14,12): error CS0579: Duplicate 'System.Reflection.AssemblyDescriptionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
obj\x86\Debug\net461\ef.AssemblyInfo.cs(11,12): error CS0579: Duplicate 'System.Reflection.AssemblyCompanyAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
obj\x86\Debug\net461\ef.AssemblyInfo.cs(15,12): error CS0579: Duplicate 'System.Reflection.AssemblyFileVersionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
obj\x86\Debug\net461\ef.AssemblyInfo.cs(12,12): error CS0579: Duplicate 'System.Reflection.AssemblyConfigurationAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
obj\x86\Debug\net461\ef.AssemblyInfo.cs(16,12): error CS0579: Duplicate 'System.Reflection.AssemblyInformationalVersionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
obj\x86\Debug\net461\ef.AssemblyInfo.cs(13,12): error CS0579: Duplicate 'System.Reflection.AssemblyCopyrightAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
obj\x86\Debug\net461\ef.AssemblyInfo.cs(17,12): error CS0579: Duplicate 'System.Reflection.AssemblyProductAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
obj\x86\Debug\net461\ef.AssemblyInfo.cs(14,12): error CS0579: Duplicate 'System.Reflection.AssemblyDescriptionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
obj\x86\Debug\net461\ef.AssemblyInfo.cs(18,12): error CS0579: Duplicate 'System.Reflection.AssemblyTitleAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
obj\x86\Debug\net461\ef.AssemblyInfo.cs(15,12): error CS0579: Duplicate 'System.Reflection.AssemblyFileVersionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
obj\x86\Debug\net461\ef.AssemblyInfo.cs(19,12): error CS0579: Duplicate 'System.Reflection.AssemblyVersionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
obj\x86\Debug\net461\ef.AssemblyInfo.cs(16,12): error CS0579: Duplicate 'System.Reflection.AssemblyInformationalVersionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
obj\x86\Debug\net461\ef.AssemblyInfo.cs(20,12): error CS0579: Duplicate 'System.Resources.NeutralResourcesLanguageAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
obj\x86\Debug\net461\ef.AssemblyInfo.cs(17,12): error CS0579: Duplicate 'System.Reflection.AssemblyProductAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\ef\Debug\net461\ef.AssemblyInfo.cs(10,12): error CS0579: Duplicate 'System.Reflection.AssemblyCompanyAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
obj\x86\Debug\net461\ef.AssemblyInfo.cs(18,12): error CS0579: Duplicate 'System.Reflection.AssemblyTitleAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\ef\Debug\net461\ef.AssemblyInfo.cs(11,12): error CS0579: Duplicate 'System.Reflection.AssemblyConfigurationAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
obj\x86\Debug\net461\ef.AssemblyInfo.cs(19,12): error CS0579: Duplicate 'System.Reflection.AssemblyVersionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\ef\Debug\net461\ef.AssemblyInfo.cs(12,12): error CS0579: Duplicate 'System.Reflection.AssemblyCopyrightAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
obj\x86\Debug\net461\ef.AssemblyInfo.cs(20,12): error CS0579: Duplicate 'System.Resources.NeutralResourcesLanguageAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\ef\Debug\net461\ef.AssemblyInfo.cs(13,12): error CS0579: Duplicate 'System.Reflection.AssemblyDescriptionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\ef\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(10,12): error CS0579: Duplicate 'System.Reflection.AssemblyCompanyAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\ef\Debug\net461\ef.AssemblyInfo.cs(14,12): error CS0579: Duplicate 'System.Reflection.AssemblyFileVersionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\ef\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(11,12): error CS0579: Duplicate 'System.Reflection.AssemblyConfigurationAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\ef\Debug\net461\ef.AssemblyInfo.cs(15,12): error CS0579: Duplicate 'System.Reflection.AssemblyInformationalVersionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\ef\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(12,12): error CS0579: Duplicate 'System.Reflection.AssemblyCopyrightAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\ef\Debug\net461\ef.AssemblyInfo.cs(16,12): error CS0579: Duplicate 'System.Reflection.AssemblyProductAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\ef\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(13,12): error CS0579: Duplicate 'System.Reflection.AssemblyDescriptionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\ef\Debug\net461\ef.AssemblyInfo.cs(17,12): error CS0579: Duplicate 'System.Reflection.AssemblyTitleAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\ef\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(14,12): error CS0579: Duplicate 'System.Reflection.AssemblyFileVersionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\ef\Debug\net461\ef.AssemblyInfo.cs(18,12): error CS0579: Duplicate 'System.Reflection.AssemblyVersionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\ef\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(15,12): error CS0579: Duplicate 'System.Reflection.AssemblyInformationalVersionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\ef\Debug\net461\ef.AssemblyInfo.cs(19,12): error CS0579: Duplicate 'System.Resources.NeutralResourcesLanguageAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\ef\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(16,12): error CS0579: Duplicate 'System.Reflection.AssemblyProductAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\ef\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(17,12): error CS0579: Duplicate 'System.Reflection.AssemblyTitleAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\ef\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(18,12): error CS0579: Duplicate 'System.Reflection.AssemblyVersionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\ef\Debug\netcoreapp2.0\ef.AssemblyInfo.cs(19,12): error CS0579: Duplicate 'System.Resources.NeutralResourcesLanguageAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\ef\ef.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\EFCore.Abstractions\Debug\netstandard2.1\EFCore.Abstractions.AssemblyInfo.cs(14,12): error CS0579: Duplicate 'System.Reflection.AssemblyCompanyAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\EFCore.Abstractions\EFCore.Abstractions.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\EFCore.Abstractions\Debug\netstandard2.1\EFCore.Abstractions.AssemblyInfo.cs(15,12): error CS0579: Duplicate 'System.Reflection.AssemblyConfigurationAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\EFCore.Abstractions\EFCore.Abstractions.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\EFCore.Abstractions\Debug\netstandard2.1\EFCore.Abstractions.AssemblyInfo.cs(16,12): error CS0579: Duplicate 'System.Reflection.AssemblyCopyrightAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\EFCore.Abstractions\EFCore.Abstractions.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\EFCore.Abstractions\Debug\netstandard2.1\EFCore.Abstractions.AssemblyInfo.cs(17,12): error CS0579: Duplicate 'System.Reflection.AssemblyDescriptionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\EFCore.Abstractions\EFCore.Abstractions.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\EFCore.Abstractions\Debug\netstandard2.1\EFCore.Abstractions.AssemblyInfo.cs(19,12): error CS0579: Duplicate 'System.Reflection.AssemblyFileVersionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\EFCore.Abstractions\EFCore.Abstractions.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\EFCore.Abstractions\Debug\netstandard2.1\EFCore.Abstractions.AssemblyInfo.cs(20,12): error CS0579: Duplicate 'System.Reflection.AssemblyInformationalVersionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\EFCore.Abstractions\EFCore.Abstractions.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\EFCore.Abstractions\Debug\netstandard2.1\EFCore.Abstractions.AssemblyInfo.cs(21,12): error CS0579: Duplicate 'System.Reflection.AssemblyProductAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\EFCore.Abstractions\EFCore.Abstractions.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\EFCore.Abstractions\Debug\netstandard2.1\EFCore.Abstractions.AssemblyInfo.cs(22,12): error CS0579: Duplicate 'System.Reflection.AssemblyTitleAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\EFCore.Abstractions\EFCore.Abstractions.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\EFCore.Abstractions\Debug\netstandard2.1\EFCore.Abstractions.AssemblyInfo.cs(23,12): error CS0579: Duplicate 'System.Reflection.AssemblyVersionAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\EFCore.Abstractions\EFCore.Abstractions.csproj]
E:\Programming\csharp7\EntityFrameworkCore\artifacts\obj\EFCore.Abstractions\Debug\netstandard2.1\EFCore.Abstractions.AssemblyInfo.cs(24,12): error CS0579: Duplicate 'System.Resources.NeutralResourcesLanguageAttribute' attribute [E:\Programming\csharp7\EntityFrameworkCore\src\EFCore.Abstractions\EFCore.Abstractions.csproj]

smitpatel commented 5 years ago

@Wraith2 - Please do git clean before running build. It seems that you are using existing repo from past and VS had issues like that if you don't clean solution. I would recommend starting afresh.

Wraith2 commented 5 years ago

ok, clean did nothing so I razed the directory with fire and started again. test run now (without me specifying a connection string which is very neat) so how do I test only the sql server tests?

smitpatel commented 5 years ago

Make sure that test is enabled. https://github.com/aspnet/EntityFrameworkCore/blob/edd48434948dc97a6f5b53efe388405e14339c9f/test/EFCore.Specification.Tests/Query/AsyncSimpleQueryTestBase.cs#L138-L150

run build.cmd on command line on repo root. then cd into test\EFCore.SqlServer.FunctionalTests run ..\..\.dotnet\dotnet test

ErikEJ commented 5 years ago

@smitpatel how do we configure the tests to run against SQL Azure DB?

cheenamalhotra commented 5 years ago

Found some info: https://github.com/aspnet/EntityFrameworkCore/pull/6625

smitpatel commented 5 years ago

The errors we are seeing now are on localdb, so SQL Azure is not a requirement for this repro anymore.

To configure connectionstring in EF tests (Sql Azure or not) just update connectionstring in this file https://github.com/aspnet/EntityFrameworkCore/blob/release/3.1/test/EFCore.SqlServer.FunctionalTests/config.json

Wraith2 commented 5 years ago

You know what isn't fun? trying to pick apart a complex build system. I can replicate the error but trying to get rid of the 14000 tests I don't care about is annoying me. I can't open the project because the solution is pinned to a dotnet version I don't have available so I just keep having to try and remove a file then do a full rebuild to see if anything breaks. This doesn't feel even remotely productive.

smitpatel commented 5 years ago

@Wraith2 - If you followed building wiki guide for EF Core and still facing any issue then let us know. We have a wiki page describing steps for specific reason that we use pre-release dependency to build a pre-release product. I can understand frustration of having to pick apart 14,000 tests but at the same time, if the repro does not occur otherwise what choice do we have.

Wraith2 commented 5 years ago

How do I turn off the strong name requirement for the references so I can use my local SqlClient build? I've got it narrowed to 2 tests and now I want to debug into what's going on. I've added my local sqlclient package directory to the nuget config and changed the EFCore.SqlServer.csproj file to have an explicit version of my package, but my package isn't signed.

E:\Programming\csharp7\EntityFrameworkCore\test\EFCore.SqlServer.FunctionalTests>..\..\.dotnet\dotnet.exe test
CSC : error CS8002: Referenced assembly 'Microsoft.Data.SqlClient, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' does not have a strong name. [E:\Programming\csharp7\EntityFrameworkCore\src\EFCore.SqlServer\EFCore.SqlServer.csproj]

smitpatel commented 5 years ago

@bricelam - Can you advice on strong naming issue above?

Wraith2 commented 5 years ago

The cause of the problem is in the test ToListAsync_can_be_canceled, it breaks something which then shows the next time you run a query. The Average_on_nav_subquery_in_projection is just a victim because it happens to be next to the other test in the definition order and presumably the test runner walks backwards in vtable order, not that It matters.

A single breaking test is this:

[ConditionalFact]
public async Task BreakTDSStream()
{
    for (var i = 0; i < 10; i++)
    {
        // without fix, this usually throws within 2 or three iterations

        using (var context = CreateContext())
        {
            var tokenSource = new CancellationTokenSource();
            var query = context.Employees.AsNoTracking().ToListAsync(tokenSource.Token);
            tokenSource.Cancel();
            try
            {
                await query;
            }
            catch (Exception)
            {
            }
        }
    }
    using (var context = CreateContext())
    {
        var results2
            = await context.Customers.Select(
                    c => new { Ave = c.Orders.Average(o => o.OrderID) })
                .ToListAsync();
    }
}

Which it seems pretty likely the cancellation is causing a problem somehow and the query outside the loop pulls an invalid connection from the pool and tries to use it.

Observations:

if you try to run ExecuteNonQueryAsync with a cancelled token it doesn't even go as far as checking if there's a valid open connection ( I forgot a connection string in a test project and was surprised it didn't throw) so I think it's unlikely that it's the read, I think it's the OpenAsync with cancelled token but I can't find where the connection is actually created/opened in entity framework.
- it is speed/timing related because if you attempt to step through it sometimes it won't occur. This suggests that an unfinished connection is returned to the pool somehow to me.
- We're absolutely going to need the EF team to help by removing all these layers of abstraction to pare it down to a sequence of operations directly on SqlClient. Async lifecycle management is bad enough without all the extra baggage on top of it.
I believe the query being run is irrelevant to the problem

smitpatel commented 5 years ago

@Wraith2 - Just to confirm, the test you posted would generate the error when running alone? I can help out removing EF from the picture for that.

Wraith2 commented 5 years ago

Yes. That test alone reproduced the tds error we're looking for 100% of the time against my local sql instance.

bricelam commented 5 years ago

The simplest way to address the strong name issue is to strong name your local build of Microsoft.Data.SqlClient (any key will do). You could also try setting <SignAssembly>False</SignAssembly> in EF Core's root Directory.Build.props, but you may have to change a few other things too (e.g. InternalsVisibleTo attributes)

Wraith2 commented 5 years ago

I can help out removing EF from the picture for that.

Any movement on this?

smitpatel commented 5 years ago

It will be a while before I can get back to this. Give me about a week.

RamBoddapati commented 5 years ago

Can any body help me finding the resolution for this issue?

Nick-Madden commented 5 years ago

We're experiencing this same issue with one of our .NET Core APIs as well. We've had two incidents in the last 5 days. As others have previously noted, this puts the application in an unrecoverable state and the only solution is to recycle its app pool.

Host (useful for support): Version: 2.2.5 Commit: 0a3c9209c0

.NET Core SDKs installed: No SDKs were found.

.NET Core runtimes installed: Microsoft.AspNetCore.All 2.1.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All] Microsoft.AspNetCore.All 2.1.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All] Microsoft.AspNetCore.All 2.2.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All] Microsoft.AspNetCore.All 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All] Microsoft.AspNetCore.App 2.1.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 2.1.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 2.2.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.NETCore.App 2.1.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 2.1.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 2.2.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]

cheenamalhotra commented 5 years ago

@smitpatel do you have any updates on the standalone repro?

smitpatel commented 5 years ago

@cheenamalhotra - I was able to get the error. But after it errored once, it was corrupting my localdb instance that I could not run any other query again. I had to delete and recreate the instance. I will try again soon.

cheenamalhotra commented 5 years ago

That's scary! You may try to wrap up a Dockerfile with sample app to avoid corrupting SQL instances.

bricelam commented 5 years ago

That's scary!

+1 Perhaps we should continue this conversation in private until we fully understand the impact of this bug...

Wraith2 commented 5 years ago

I was able to get the error. But after it errored once, it was corrupting my localdb instance

I ran my repo several dozen times and I've seen no adverse effect on my instance. So perhaps it's localdb only or can cause problems with very fast repro. Probably worth treating carefully as @bricelam suggests though.

michaeltnguyen commented 5 years ago

@Wraith2 do you know if this is still an issue on .NET Core 3.0? Will upgrading solve the problem?

mscrivo commented 5 years ago

@Wraith2 do you know if this is still an issue on .NET Core 3.0? Will upgrading solve the problem?

It is. I just finished upgrading our entire codebase to .net core 3 and EF core 3 and pushed it out to QA, and low and behold, got the error in the new Microsoft.Data.SqlClient. Was rather disappointed.

EDIT: After a few days of observing the EF Core 3/Microsoft.Data.SqlClient changes on our QA systems, we're seeing this error a lot. Averaging twice a day so far. Previously, we never saw it on QA system (because load was so low I presume) and only saw it in production maybe once every 2 months or so. The only saving grace with the new code is that it seems to be able to self recover without having to restart the app/reboot.

To give some more details about our environment:

This is a single console app targetting netcoreapp3.0 running on Windows Server 2019 that makes heavy use of async/await to run many "worker" type jobs. ie. grabbing messages from a message queue and processing them in parallel, which often involves work against our Azure SQL DB instance.
- So far we have not yet seen this happen on our web heads which are running IIS on Windows Server 2019 proxying requests to an asp.net core 3 app.

EDIT 2: After deploying EF Core 3 upgrade to production, we started getting 10's of thousands of these exceptions per hour. Luckily, it was contained to our worker tier, which is pretty resilient and we didn't have to do much to mitigate, it just made our monitoring a mess. I had to deploy @brunolau's wonderful solution and now we're only seeing 10's per hour before it resets the connection pool.

It's really curious that this became a much bigger problem with Microsoft.Data.SqlClient and EF Core 3.

JohnnyFun commented 5 years ago

Does it make any difference whether MultipleActiveResultSets is set to true or false?

David-Engel commented 5 years ago

Does it make any difference whether MultipleActiveResultSets is set to true or false?

It does not appear to make a difference if MARS is enabled or not.

yayayahei commented 4 years ago

I'm really sad😔 that the issue happened 2 times at our production environment, at today and Sep. 29.

I'm afraid that it will happen again. So I will give up EF Core, and use other technology to access DB, until this issue is resolved. Actually EFCore is convient, I want to use it at work. Please resolve this issue ASAP.

Wraith2 commented 4 years ago

@yaeldekel the issue isn't in EF, it's the database driver (which is why it's in the repository) so you'll see it whether you use EF or not. If EF is helpful to your work it would be pointless for you to drop it in favour of direct access.

simeyla commented 4 years ago

I just got this for the first time with EFCore 3 and WebAPI on the most innocuous query - to load a single user with Identity. Restarting my app fixed it but it is quite worrying.

I am using connection pooling and wonder if that has anything to do with this.

Failed executing DbCommand (79ms) [Parameters=[@__loginProvider_0='?' (Size = 450), @__providerKey_1='?' (Size = 450)], CommandType='Text', CommandTimeout='30']
      SELECT TOP(2) [a].[LoginProvider], [a].[ProviderKey], [a].[ProviderDisplayName], [a].[UserId]
      FROM [AspNetUserLogins] AS [a]
      WHERE (([a].[LoginProvider] = @__loginProvider_0) AND @__loginProvider_0 IS NOT NULL) AND (([a].[ProviderKey] = @__providerKey_1) AND @__providerKey_1 IS NOT NULL)
fail: Microsoft.EntityFrameworkCore.Query[10100]
      An exception occurred while iterating over the results of a query for context type 'RR.EF.Identity.ApplicationDbContext'.
      Microsoft.Data.SqlClient.SqlException (0x80131904): The incoming tabular data stream (TDS) protocol stream is incorrect. The MARS TDS header contained errors.

mscrivo commented 4 years ago

These errors have completely disappeared for us after upgrading to EF Core 3.1/Microsoft.Data.SqlClient 1.1.0.

cheenamalhotra commented 4 years ago

That is surprising and a happy news! 🎉

I'll close this issue for now, if anyone experiences this issue again, an upgrade to Microsoft.Data.SqlClient 1.1.0 would be helpful. If still issue occurs, please let us know with a reproducible app/code.

mscrivo commented 4 years ago

Sounds good to me, but to be fair, it may not be completely gone. This is only 5 days of logs now where we haven't seen the issue, whereas it was occurring thousands of times per hour on 3.0/1.0, so definitely a positive sign.

However, when were on EF Core 2.2 and the old SqlDataClient, we saw this issue maybe once every month or two. So that may still be the case, who knows, only time will tell.

michaeltnguyen commented 4 years ago

@cheenamalhotra I agree with @mscrivo. Closing this seems premature. Not observing it for a few days is not the same as proving that the issue is resolved. (especially because the issue is intermittent and hard to reproduce). When we were on SqlClient 1.0.0, we didn't observe it for 3 months until it suddenly became an issue.

cheenamalhotra commented 4 years ago

No problem!

alefranz commented 4 years ago

Reporting back that since updating to EFCore 3.1 and SqlClient 1.1, we haven't had any more occurrence as well. But still probably worth investigating it to confirm that is is fixed and not just less likely to happen with the risk to regress with future unrelated changes.

IvanAlekseev commented 4 years ago

This issue started appearing to us yesterday on .Net Core 3.0 and Microsoft.Data.SqlClient 1.0.19269.1 After upgrading to .Net Core 3.1 and Microsoft.Data.SqlClient 1.1.0 it is still there. It is deployed to Azure App Service connecting to Azure Sql Database. We use Dapper to run queries The service works fine for some time then starts throwing errors and only restart helps The error I see in Application Insights is:

Microsoft.Data.SqlClient.SqlException (0x80131904): The incoming tabular data stream (TDS) protocol stream is incorrect. The MARS TDS header contained errors.
   at Microsoft.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at Microsoft.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at Microsoft.Data.SqlClient.SqlDataReader.TryConsumeMetaData()
   at Microsoft.Data.SqlClient.SqlDataReader.get_MetaData()
   at Microsoft.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption, Boolean shouldCacheForAlwaysEncrypted)
   at Microsoft.Data.SqlClient.SqlCommand.CompleteAsyncExecuteReader(Boolean isInternal, Boolean forDescribeParameterEncryption)
   at Microsoft.Data.SqlClient.SqlCommand.InternalEndExecuteReader(IAsyncResult asyncResult, Boolean isInternal, String endMethod)
   at Microsoft.Data.SqlClient.SqlCommand.EndExecuteReaderInternal(IAsyncResult asyncResult)
   at Microsoft.Data.SqlClient.SqlCommand.EndExecuteReaderAsync(IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)

cheenamalhotra commented 4 years ago

@IvanAlekseev

Could you provide a repro if this gets reproduced consistently? We still haven't got a standalone repro that reproduces the error.

michaeltnguyen commented 4 years ago

@cheenamalhotra @Wraith2 wrote a test that was able to reproduce the issue 100% of the time in this comment

https://github.com/dotnet/SqlClient/issues/85#issuecomment-532403666

cheenamalhotra commented 4 years ago

@michaeltnguyen

As commented by Wraith:

We're absolutely going to need the EF team to help by removing all these layers of abstraction to pare it down to a sequence of operations directly on SqlClient. Async lifecycle management is bad enough without all the extra baggage on top of it.

I believe we have been waiting for a repro without any abstraction layers to understand the flow of APIs that cause this issue. There is not enough clarity with the test case from EF suite.

michaeltnguyen commented 4 years ago

gotcha. Does the EF team know that you're waiting for their help? (I missed that in Wraith's earlier comment.) Or should one of us try to create a repro using purely SqlClient?

cheenamalhotra commented 4 years ago

Does the EF team know that you're waiting for their help? (I missed that in Wraith's earlier comment.)

I would hope so, but I'm not sure if this is in top of their priorities.

Should one of us try to create a repro using purely SqlClient?

That will be awesome! Highly appreciate it!

michaeltnguyen commented 4 years ago

I ran @Wraith2 's test in #85 on the master branch and it no longer reproduces this issue. Unfortunately, it appears that the issue is still happening in the wild, so we're back at square one. =/

cheenamalhotra commented 4 years ago

@michaeltnguyen

Can you upload your app in zip as it is which I can test with different source references of Microsoft.Data.SqlClient? I'm not very used to working with EF Core, and it shall save time if you already have a workable code.

I'd like to give it a try if I can find something.

michaeltnguyen commented 4 years ago

@cheenamalhotra Sure, I've uploaded it to my google drive. (~600 MB)

https://drive.google.com/open?id=1n3D9RzSPv1nfLYfyPhXHXcVyBq_3tXP1

It's basically just this repository with Wraith2's test added, and Average_on_nav_subquery_in_projection no longer skipped. The new test is in NorthwindAsyncSimpleQueryTestBase.cs

I'm running it via Visual Studio 2019.

cheenamalhotra commented 4 years ago

Oh I guess you've not pulled out the test then.. I thought you've pulled out abstractions. nvm in that case I'll just clone EF Core and add this test there.

michaeltnguyen commented 4 years ago

I first tried to reproduce #85 using just SqlClient and SqlCommand, but I couldn't. So I figured the next step is to try to reproduce using wraith's code verbatim in EF Core. (but the test seems to pass)

Here's my attempt of translating #85 into SqlClient, in case it's useful.

https://gist.github.com/michaeltnguyen/e9710fe6530286ab6363ae5f5b245a5c

cheenamalhotra commented 4 years ago

I can reproduce this issue in release/3.0 branch of EF Core with the test from Wraith, but not in master branch. I'm wondering what changed in EF Core that this error can no longer be reproduced. (I guess I found a repro I can now investigate into, and the issue is reproducible with M.D.S v1.1.0)

https://gist.github.com/michaeltnguyen/e9710fe6530286ab6363ae5f5b245a5c

I somehow cannot reproduce in standalone repro with any driver version.

@smitpatel do you have any updates on this error from EF Core side?

cheenamalhotra commented 4 years ago

While debugging this issue, I just happened to change my branch today to @Wraith2's PR https://github.com/dotnet/SqlClient/pull/248 and the PR seems to fix this issue. Reverting my branch to master brings back failures.

The use case that fails gets it's trigger from concurrent runs of CancellationTokenSource().Cancel() where Attention signals are sent to SQL Server, and the PR does seem to work on the same paths by locking Send() to avoid race conditions.

Although the PR still needs some work as some test cases seem to fail, but it's also seems to be making a big difference leading to fix this issue.

@Wraith2 would you like to give it a try and confirm the same? As mentioned above, this error is reproducible with the test posted above and release/3.0 branch of EF Core. Also would appreciate if you can address pipeline failures!

dotnet / SqlClient

The MARS TDS header contained errors using ASP.NET Core and EF Core connecting to Azure SQL Server #85