Windows 10 (1803) - SqlCient error after updating Windows - “The incoming tabular data stream (TDS) remote procedure call > (RPC) protocol stream is incorrect”

[divega]

From @dotnetshadow on May 15, 2018 7:10

Hi there,

I found some of my queries are not working after updating to Windows 10 version 1803. I can confirm that the same code works on machines without this update.

The error I keep getting is this:

System.Data.Entity.Core.EntityCommandExecutionException: An error occurred while executing the command definition. See the inner exception for details. ---> System.Data.SqlClient.SqlException: The incoming tabular data stream (TDS) remote procedure call (RPC) protocol stream is incorrect. Parameter 2 (""): Data type 0x00 is unknown.

Interestingly if I make the query non async then everything seems to work. Has something changed since Windows 10?

Other users have also indicated having the same problem https://stackoverflow.com/questions/50340258/entity-framework-error-after-updating-windows-the-incoming-tabular-data-strea

Further technical details

EF version: 6.2 Database Provider: Azure SQL Server Operating system: Windows 10 (1803) - April 2018 update IDE: Visual Studio 2017 15.7

Copied from original issue: aspnet/EntityFramework6#537

[divega]

From @chainbridgetech on May 15, 2018 12:53

I can report that I received the same error after updating to 1803. A system restore back to 1709 has temporarily fixed the problem.

[divega]

From @ErikEJ on May 15, 2018 12:56

Sounds like a SqlClient bug, not an EF6 issue

[divega]

From @dotnetshadow on May 15, 2018 13:6

@ErikEJ Any idea where I can report this?

[divega]

From @ErikEJ on May 15, 2018 13:35

I am sure the EF team will know who to notify

[divega]

From @chainbridgetech on May 15, 2018 19:21

I have confirmed that this is specifically caused by .NET Framework 4.7.2. I manually installed .NET 4.7.2 on my Windows 10 1709 computer, and the error started up again.

[divega]

@dotnetshadow Thanks for reporting this. I am going to be moving this issue to https://github.com/Microsoft/dotnet/issues, which is where issues for SqlClient on .NET Framework are tracked. In the meantime, it would be great if you could provide a repro project.

[zachgreencbt]

I am still working on a simplified repro project, but I have found that if I change MultipleActiveResultSets from TRUE to FALSE in my connection string, the error stops.

[zachgreencbt]

I now have a repro project. I have confirmed that the error only happens when connecting to a SQL Azure db and when the connection string is configured with MultipleActiveResultSets=True. What is the preferred way to distribute the repro project? I will need to open up access to the db in Azure by IP for whoever is testing it.

[corivera]

@chainbridgetech Would you be able to make simple test tables in your repro project, or is this error actually data-dependent?

[zachgreencbt]

I created a local db in the project that exactly matches the Azure db (one connection string is commented out and can be flipped to test), but the error doesn't occur when connecting to the local db only when connecting to SQL Azure. It is not dependent on the data. The tables can be empty and the error occurs.

[corivera]

Would you be able to push up your repro project to Github then? If the tables can be empty, then I can just run the repro against my own Azure DB.

[zachgreencbt]

https://github.com/chainbridgetech/net472-rpc_error-repro -- I can send you the password if you want to try it against my test db (that doesn't have any private data).

[corivera]

No, that won't be necessary. Are there test table creation statements included in the project?

[zachgreencbt]

The models project has an mdf, Database1.mdf with the same schema as the Azure db. I can easily add a script in the next 10 minutes.

[corivera]

That would be very helpful, thanks.

[zachgreencbt]

https://github.com/chainbridgetech/net472-rpc_error-repro/blob/master/Models/db_create.sql

[corivera]

Great, I'll look into it.

[zachgreencbt]

Were you able to replicate the exception using my repro project?

[corivera]

Yes, I was able to repro the issue. I'm investigating potential fixes.

[leonardobaggio]

I'm also facing this exact same issue at dev environment in two different machines (both running Windows version 1803). If you guys need any further technical details, please let me know.

[demodav]

I am facing the same issue with version 1803. I had fixed my issue by converting from .Include to .IncludeOptimized

[Joshuapwilley]

Also experiencing this issue. Another temporary fix appears to be removing async from the queries.

[Joshuapwilley]

@demodav You should make others aware that the method you referenced as a solution is only available from a third-party package. It is not part of Entity Framework.

[joylemicrosoft]

An update from Azure SQL Engineering team: Engineers have developed a SQL DB server side fix and are rolling out through the safe deployment procedure. ETA 24-48 hours.

[dotnetshadow]

I can confirm that the issue seems to be resolved, thank you for all your efforts, great work

[Joshuapwilley]

I can also confirm that the issue appears to be resolved as well.

[zachgreencbt]

For the server where I discovered the issue, the problem is fixed. However, for the server I created for the purposes of creating a repro project, I am still getting the error. I am going to assume for now that it simply hasn't received the fix yet, and I will recheck it tomorrow.

[zachgreencbt]

Azure support was able to confirm that the fix has not been rolled out to the server where I am still experiencing the error.

[bwaldron]

@joylemicrosoft: Is there a time where this fix has been rolled out to all servers? We are still seeing the error with Windows 1803.

[petercarlin]

The fix has been rolled out to all SQL DB servers. If you are seeing it at this point, please contact Microsoft support.

[petercarlin]

My name is Peter Carlin and I work in the SQL Server team. I first want to apologize for this incident and the impact on users of .NET Framework 4.7.2. I also would like to thank all those who reported the issue. I would like to explain what happened and how Microsoft is fixing it in more detail. The issues is due to improvements to the Always Encrypted functionality in SQL. These improvements expand the set of operations that can be done in Always Encrypted, however they are not yet ready for applications to use. These improvements involve changes to both SqlClient and SQL server side. We introduced an error in .NET Framework 4.7.2 such that in some circumstances (related to MARS) SqlClient incorrectly thinks the added functionality is in use and sends invalid requests to SQL. SQL rejects those with the error messages seen in this thread. This only happens if connected to a SQL server that has support for the added functionality as well. SQL DB is the first to get the latest SQL changes, and recently deployed the added functionality. Our immediate fix is to ensure the SQL DB acts as if it does not have the added functionality, so the SqlClient side bug in 4.7.2 is not encountered. This is why we are able to fix the issue with a SQL DB side change. At this point the fix has been deployed to all SQL DB servers.

[leonardobaggio]

@petercarlin good to hear that these improvements for AE are going on ...

Sorry to write this here, probably this isn`t the right place. I'm currently in a migration of some columns to Always Encrypted in a already populated Azure SQL Database, and the main application that consumes it is using Entity Framework. The overall process is being quite painful.

A lot of unexpected "Operand clash" errors in SELECT operations required us to recode many of our existing queries, especially the ones with .Include () method that EF uses UNION ALL Operator to deal with.

Anyway, I noticed in the middle of the last week that some error messages have changed, and it now specifies the column that is causing the error, etc ... My question is, is there a place like github or other community that can I follow and be aware of this improvements that are being made at the Always Encrypted functionality in SQL?

Thanks, Leo

[petercarlin]

I do not know of a specific Always encrypted forum. But if send me email at p e t e r c a @ Microsoft I will connect you with the program managers working on Always Encrypted who can work to keep you up to date.