Column Encryption (Always Encrypted) support in SqlClient for .NET Core

[ghost]

When using a DbContext with a ConnectionString like

{
  "ConnectionStrings": {
    "DefaultConnection": "Server=.\\SQLEXPRESS;Database=EFGetStarted.AspNetCore.NewDb;Trusted_Connection=True;Column Encryption Setting=Enabled;MultipleActiveResultSets=true;"
  }
}

I get the following error StackTrace

{System.ArgumentException: Keyword not supported: 'column encryption setting'.
   at System.Data.Common.DbConnectionOptions.ParseInternal(Dictionary`2 parsetable, String connectionString, Boolean buildChain, Dictionary`2 synonyms)
   at System.Data.Common.DbConnectionOptions..ctor(String connectionString, Dictionary`2 synonyms)
   at System.Data.SqlClient.SqlConnectionString..ctor(String connectionString)
   at System.Data.SqlClient.SqlConnectionFactory.CreateConnectionOptions(String connectionString, DbConnectionOptions previous)
   at System.Data.ProviderBase.DbConnectionFactory.GetConnectionPoolGroup(DbConnectionPoolKey key, DbConnectionPoolGroupOptions poolOptions, DbConnectionOptions& userConnectionOptions)
   at System.Data.SqlClient.SqlConnection.ConnectionString_Set(DbConnectionPoolKey key)
   at System.Data.SqlClient.SqlConnection.set_ConnectionString(String value)
   at Microsoft.EntityFrameworkCore.Storage.Internal.SqlServerConnection.CreateDbConnection()
   at Microsoft.EntityFrameworkCore.Internal.LazyRef`1.get_Value()
   at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.Open()
   at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.BeginTransaction(IsolationLevel isolationLevel)
   at Microsoft.EntityFrameworkCore.Update.Internal.BatchExecutor.Execute(Tuple`2 parameters)
   at Microsoft.EntityFrameworkCore.Storage.Internal.SqlServerExecutionStrategy.Execute[TState,TResult](Func`2 operation, Func`2 verifySucceeded, TState state)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChanges(IReadOnlyList`1 entriesToSave)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChanges(Boolean acceptAllChangesOnSuccess)
   at Microsoft.EntityFrameworkCore.DbContext.SaveChanges(Boolean acceptAllChangesOnSuccess)
   at xBN.Repository.Repositories.EntityBaseRepository`1.Commit() in C:\!Checkouts\git checkouts\xBN\xBN.Service\Repositories\EntityBaseRepository.cs:line 98
   at xBN.API.Controllers.CompanyController.Get(Int32 id) in C:\!Checkouts\git checkouts\xBN\xBN.API\Controllers\CompanyController.cs:line 37
   at lambda_method(Closure , Object , Object[] )
   at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.<InvokeActionMethodAsync>d__27.MoveNext()}

The issue is about adding support for the Column Encryption Setting to the .NET Core ADO.NET SqlClient.

[jasondaicoder]

guys, has this been fixed or?

[corivera]

CoreFx SqlClient does not currently support column encryption

[jasondaicoder]

@corivera, do we have any plans on this feature?

[corivera]

We don't have any plans for it at the moment, we (SqlClient team) will have to discuss it.

[jasondaicoder]

Thanks for you info. I think it would be highly demanded feature as column encryption is widely used. Hope to see this feature being implemented soon.

[divega]

Related https://github.com/aspnet/EntityFramework/issues/9142.

[jdelrue]

Need this

[LarrySmith-1437]

The lack of Always Encrypted is a real drag on promoting .net Core where I am, because security is paramount.

[wizofaus]

Actually I had reported this separately, so technically my issue is a duplicate. But I've been looking at explicit server-based value encryption (using ENCRYPTBYKEY etc), and it seems there are no .net libraries out there that support this either?

[aspnerd]

What is going on with this? We really need this now!

[danmoseley]

@divega @corivera is this in plan? If not is it feasible for community to port?

[mattd73uk]

Just starting a new enterprise project, we had decided on .Net Core but are switching to .Net Framework over this issue - I just removed the .Net Core projects from the solution and am retargeting ASP.Net Core at the .Net Framework. It is a shame Core isn't yet mature enough for us, but at the end of the day you've got the resources you've got.

[JohnKoz]

@corivera : Similar to @mattd73uk, we've started an enterprise project and just found out always encrypted is not supported in .Net Core. Basically we will have to switch back to the .Net Framework unless we hear otherwise. Is there any indication on when this feature would likely be added?

[se7vanj]

We had the same demand during mid of 2017 in one of our enterprise project that uses AE feature from a .NET Core API, reached through the Microsoft premier support, but could not find any timelines for this functionality, I don't think its even present in their future milestones.

Wondering when they will add support for column encryption in CoreFX sql client.

Finally, we had to target the API to .NET Framework and use EF 6.x instead of Core.

[danmoseley]

@divega @corivera could you please share thoughts? Clearly it's a blocking issue for a number of folks here. Is it something community could help port?

[louislewis2]

Ouch, this has burned me today horribly. Tomorrow we are deploying to production. Which means that today we start migrating users across from an old system across to a new one.

Then I get hit by this......

[ld0614]

While I was looking forward to trying .NET Core out for a new project the lack of support for column encryption and the lack of support for AccessToken means I will have to build the project on .Net Framework instead.

[David-Engel]

Always Encrypted is definitely still on our radar. We don't have a timeline for it at the moment, though. There are a few items that we believe are higher on the list. We are open to prioritization changes, though (vote up comments on this issue).

[rudiv]

I've commented on another thread regarding this already - but with your recent push on GDPR it shocks me that this isn't available. It's fairly critical to stop people from rolling their own methods to encrypt data in columns (which as we know from experience never ends well).

Being forced to use .NET Framework for such "basic" (from an outside perspective) functionality isn't really acceptable, when there's such a huge push to use .NET Core.

[aspnerd]

@David-Engel We cannot use the latest SQL and .net Core due to this. We had hours and hours of rework which had to be done because of this and we aren't the only ones. This should have already been done imo.

[LarrySmith-1437]

Here we are at the midpoint of 2018. To find out now that this critical piece of functionality is even at this late date not under development is deeply disappointing.

Like these others, I've had to push back on timelines to move to .net core.

Prospective dates are in order here. When?

[aspnerd]

Always Encrypted came out in 2016. We are a complete newer version of SQL Server ahead and at least 4 versions of .net framework ahead since it came out and we still don't have support for core. This makes no sense!

[SariDev]

I hope to see this feature in the next version of System.Data.SqlClient. Support for Always Encrypted is required for enterprise applications that rely on SQL Server and it's security features. Releasing this together with .NET Core 2.2 later this year should be high priority.

[filippostefanello]

I share your hope as I need a solution to comply the GDPR's rules.

[armordba]

With all the data security push from GDPR and others, we really need this fix ASAP

[RJoseSilva]

we really need this fix ASAP. This feature will be release with .Net Core 2.2 ?

[erikrenaud]

Any news on this ? would be nice to not have to use .net framework for bigger projects...

[mjtpena]

Wow. Basically my whole Cryptography layer can't be upgraded to .NET Core because of this.

[houseofcat]

Would very much like this prioritized.

Understand you are busy, do you guys want help - this isn't really an optional feature anymore.

Rolling custom encryption is going to lead to insecurity, which is really bad PR for NetCore. Once you implement said feature- unless you never release - you are essentially locked to said implementation.

It's also been lower priority than features released and then re-released under NetCore 1.0 and then 2.0.

I simply dont understand the motivation to shelve this, even without GDPR. Security is paramount.

[David-Engel]

Always Encrypted is still very much a priority for the SqlClient team and we are hearing all the comments. Unfortunately, we still don't have a concrete timeline we can share yet.

[aspnerd]

We are going 3 versions deep in SQL Server without support. We really want to use .net core, but we can't because of this we have to stay full framework. When will it be priority enough to get it finished?

[houseofcat]

@David-Engel can we bribe you with cookies and beer?

[David-Engel]

@houseofcat I do like a good chocolate chip cookie. 😄 But some things are beyond my control. I wish I could share more. There are always other factors in play but we definitely are shooting for AE support in .NET Core as soon as we can.

[gulbanana]

I have a .NET Core application currently using SQL Server. In a future release, data is being added to the system which needs column encryption due to its nature - we’re going to have to port it to .NET Framework due to this missing capability.

[mentallabyrinth]

Was always told by pundits that .NET had best in class security which was the governing reason for selecting the platform. Surprised to find out that as of yet ".NET Core" doesn't support the Always Encrypted feature. Neither our team or the consultants we hired were aware of this 😆Learning experience, tenfold.

Anyway, throwing our hat into the ring to support fast-tracking this feature, it would be beneficial to have. Because conversion is out of the question, but we'll figure out something else in the interim.

And it's worth saying that despite this, thanks for the excellent product so far! 😀

[tomRedox]

Adding our name to the list of people not implementing new projects in .NET Core because of the lack of AE support.

[jxvalenz]

Work for a large Fortune 10 company that is also putting the brakes on many large projects due to this. Going with .NET Framework is not an option. We would really appreciate it if at least you could give us hope that this feature is coming in the near future, at least in beta form, so our management doesn't throw .NET Core out the window and go for a Java solution. We have already invested in .NET Core and was about to become our corporate standard. Please give us a date range at least.

[aspnerd]

I wonder if the holdup is the fact that the encryption is proprietary and they can't open source it. If that is the cast then they should just create an add on nuget package for .net core so we can use it!

[lsuarez5280]

@aspnerd that definitely isn't the problem. The existing SqlClient always encrypted support code is available right now in the Framework GitHub. There's nothing proprietary about that implementation.

[David-Engel]

@jxvalenz It's coming. Someone is actively working on the code. I don't have a timeline I can publicly share right now, though.

[houseofcat]

@David-Engel thank you for the update and I understand you can't give specifics, however, what we really could use are some abstracts. Will we have a preview/release this year (2018), or in v2.2, >v2.2, 3.0. etc.

Can you let us know if there is intention to release it this year?

This isn't asking for a concrete delivery date, this is so we can plan our future NetCore empires. We have to plan/prototype. Sell these ideas we are creating - and the technology that powers these ideas - to Project Managers who by-and-large eat q-tips. Being able to tell the q-tip eaters when this will be secure and viable is the horcrux for keeping these ideas alive.

Respectfully, and not above bribing with cookies and beer, HouseCat.

[kevbry]

Seconding @houseofcat . With full framework support going away in asp.net core 3.0 we need to start making plans.

[tellamon]

I also hope we could make a mid-term decision for our project with the plan of Column Encryption support.

[gulbanana]

This has definitely become more urgent with the news that ASP.NET Core will no longer run on netfx - the plan of porting back to netfx which we had is no longer feasible!

[aspnerd]

@divega I don't this is related to our issue above.

[divega]

@aspnerd they are related in the sense that they are both potentially competing priorities for the SqlClient team to tackle in the near future. You can follow the link to see in what context I mentioned it.

[mattd73uk]

Hi, as per my comment above I switched our enterprise project from Core to Framework over this, definitely the right decision, SQL Server 2017 with ASP.Net Core targeting the .Net Framework with EF is a great combination of secure & easy to maintain tech.

But taking a step back, my concern is how is it possible that the latest Microsoft technology in one area (.Net Core) has spent two years not supporting/interop with such a great feature in the current MS tech in another area (SQL Server 2016, 2017, 2019).

I don't mind using the .Net Framework - it's fantastic & served me well. But from its announcements the MS megalith (that's a good thing - I and my clients have relied on it for 20 years) wants me/my clients to use .Net Core going forward, but it hasn't able to put the resources in play to make what it wants happen.

@divega does that make sense as a concern? Yes, you are able to cite an issue that has received more upvotes, but why is there a need to pit two so important/requested issues against each other? Why has MS been unable to resource either for 2 years to support a flagship product like SQL Server?

[achancecwg]

We are developing a LOB application using UWP and just hit a roadblock because of the lack of support for this feature. I think that these types of feature are critical for developers to be able to adopt UWP and .NET Core/ .NET Standard, especially for Enterprise applications.

Would there need to be a separate issue opened for UWP support?

Please consider implementing this feature soon!

[vickyharp]

I echo what @David-Engel says about not being able to share a concrete timeframe, but I can tell you that the feature is under active development with high priority for the SqlClient team. This capability is important to our users and it is important for cross platform parity in Microsoft's own data tools.

For directional purposes: it is currently planned for the v3.0 timeframe and we will announce more soon as we become more confident in the timing.

[houseofcat]

My body is ready!