EF Core 5 support

[timmi-on-rails]

.NET 5 is coming and the release is planned for November 2020. At the same time EF Core 5 will be released. Today I can already try the preview builds of .NET 5 and EF Core 5 via NuGet. But I am unable to use the Pomelo.EntityFrameworkCore.MySql adapter, since there are breaking changes in EF Core 5. I couldn't find a build of the adapter targeting EF Core 5 Preview.

Do you have plans for that and what is the progress? I saw there is a 5.0-wip branch.

[lauxjpn]

An EF Core 5 preview compatible Pomelo release is planned for the second half of June.

[timmi-on-rails]

Any news? On what branch are you developing the .net 5.0 upgrade? Maybe I am able to contribute, to speed things up. 5.0-wip branch hasn't been touched since February.

[mguinness]

The current effort is focused on adding spatial and JSON support to 3.x and then afterwards upgrading will be looked at. Based on previous version upgrades it's a significant amount of work which will take several months based on the limited developer resources.

[ansarizafar]

An EF Core 5 preview compatible Pomelo release is planned for the second half of June.

@lauxjpn June is almost over. Can we expect An EF Core 5 preview compatible release this month.

[timmi-on-rails]

Based on version 3.1.1 I got an upgrade to EF Core 5 preview "working". I have inserted some records and ran a select statement successfully.

I still have to run the tests and I need help at 2-3 places. Just wanted to inform you, that it is possible to get a "compiling" version in couple hours. Probably faster, when knowing EF Core + database provider api.

EF Core 5 does not require net5.0, only netstandard2.1. Ofcourse this can still change, because it is preview state, still.

[lauxjpn]

We will publish an official preview after the JSON support release, of which I have a prototype running at the moment, that still needs a bit of polishing. So expect an EF Core 5.0 preview compatible release until about 2020-07-15.

@ansarizafar, @timmi-on-rails If you have time, please share with us the reasons, why you already need a EF Core 5.0 compatible release. Maybe what you need can be done with the current release as well.

[timmi-on-rails]

@lauxjpn I am currently working on a project, which is still in an early phase and the first release is planned for end of 2020, so I wanted to update the project dependencies, e.g. efcore to the preview of 5.0, because this will be the up-to-date version, when the project is released. The reason is to test and find incompatibilities as early as possible. I haven't checked if any new efcore 5 features are useful for the project. Just planning ahead to stay updated.

[Simonl9l]

I believe there is a general hope, across the system developer community, that the teams developing common libraries that are essential components, that in context of the overall developer ecosystem, are this time more broadly lined up with the DotNet Core 5.0 release schedule than they were with the 3.0 release schedule.

Also there are numerous issues on the various .Net Core and more importantly in the case on the EF Core GitHub repo that have a "Fixed in 5.0" conclusion.

Although 3.1 is supposed to the current LTS release, many of the solutions are rhetorically involving breaking changes or are too complex to be made backward compatible with 3.1, so are not even made candidates for Long Term Support.

As such the develop community are eager to get to use and experiment with the new features, and more specifically, if they have a MySQL dependency, need a Pomelo EF Core 5.0 compatible driver (beta) to even make that use/experiment possible.

We all await with bated breath, and any progress in this regard is greatly appreciated!

[lauxjpn]

This finally just got unblocked by #1102 (through #1157 and #1158). Expect something working by the end of the week.

[mguinness]

The current effort is focused on adding spatial and JSON support to 3.x and then afterwards upgrading will be looked at.

Work on both of these features is now complete, thanks to Laurents for all his hard work!

[IvanZheng]

@lauxjpn When and where can I get a preview release to try on EF Core 5?

[jparadnikas]

Hey! Is there a preview release for this? I need to try it, query splitting is essential for me

[lauxjpn]

I have already upgraded Pomelo in the way that it compiles with .NET 5. I am currently upgrading the tests, so that they compile as well. Once that is done (probably tomorrow), we will release a first alpha (on our nightly build feed). Once the tests all run, we will release an official preview (probably around the end of the week). We will then introduce a couple of breaking changes and features. So code depending on alpha or preview releases will break (mostly at compile-time and not very significantly).

[onurh]

@lauxjpn Is there any change on scheduled time for alpha version?

[lauxjpn]

@onurh There is an alpha prerelease in our nightly build feed. I can also update it with the latest version in a couple of hours. In this latest version, there are still about 100 of the 15,000 tests not working yet, but this is unlikely to be significant. I am actively working on them, to get a version out with all tests running as soon as possible.

[zaneclaes]

Amazing. I now have Pomelo working w/ dotnet5, EFCore5, Identity Server 4, and MySqlConnector 1.0.1.

I had to recreate my databases (couldn't get the IS4 migration to work properly, but no big deal). I have tested all the core identity flows (creating an account, verifying, etc.). But the really nice part is that many-to-many tables without explicit mappings are now working, which saved me a TON of boilerplate and headache 🎉

[ezealv]

Amazing. I now have Pomelo working w/ dotnet5, EFCore5, Identity Server 4, and MySqlConnector 1.0.1.

I had to recreate my databases (couldn't get the IS4 migration to work properly, but no big deal). I have tested all the core identity flows (creating an account, verifying, etc.). But the really nice part is that many-to-many tables without explicit mappings are now working, which saved me a TON of boilerplate and headache 🎉

How do you make Pomelo work with EF Core 5? Because Pomelo 5 hasn't come out yet.

[lauxjpn]

How do you make Pomelo work with EF Core 5? Because Pomelo 5 hasn't come out yet.

@ezealv Use the latest prerelease from our nightly build feed. The latest build 5.0.0-alpha.1.20513.5 was just released 2 hours ago.

[Simonl9l]

Will this work with EF Core 5 under DotNet Core 3.1? Since EF Core 5 is supposed to work in this scenario.

[fmauNeko]

Will this work with EF Core 5 under DotNet Core 3.1? Since EF Core 5 is supposed to work in this scenario.

Yes, it targets .NET Standard 2.1, as EF Core 5 does. (https://github.com/PomeloFoundation/Pomelo.EntityFrameworkCore.MySql/blob/5.0-wip/src/EFCore.MySql/EFCore.MySql.csproj#L5 https://github.com/PomeloFoundation/Pomelo.EntityFrameworkCore.MySql/blob/5.0-wip/Directory.Build.props#L32)

[fuzl-llc]

Hello. Thanks for all the helpful info on status and for all the hard work!

I added the package source and tried to update from 3.2.3 to the above mentioned alpha and get this:

Unable to find package MySqlConnector. No packages exist with this id in source(s): pomelo-efcore-public

I also tried to use package manager to install it like this:

PM> Install-Package Pomelo.EntityFrameworkCore.MySql -version 5.0.0-alpha.1.20513.5

...and I get the same error.

I also get this warning:

NU1603: Pomelo.EntityFrameworkCore.MySql 5.0.0-alpha.1.20513.5 depends on Microsoft.EntityFrameworkCore.Relational (>= 5.0.0-rc.1.20451.13) but Microsoft.EntityFrameworkCore.Relational 5.0.0-rc.1.20451.13 was not found. An approximate best match of Microsoft.EntityFrameworkCore.Relational 5.0.0-rc.2.20475.6 was resolved.

I just updated to .NET 5 RC 2... so maybe that is causing the warning, but any ideas on the first one? Any current issues with availability? Do I need to wait for a future release to test it with .NET 5 RC 2?

[mguinness]

@fuzl-llc Did you follow the instructions from Nightly Builds as it seems you're missing the entry for nuget.org source where MySqlConnector is located.

[fuzl-llc]

That was it, thanks! I had followed the link to 5.0.0-alpha.1.20513.5 given above:

The latest build 5.0.0-alpha.1.20513.5 was just released 2 hours ago.

It has "connect to feed" instructions of its own which do not include the nuget.org source. I'm not sure if Pomelo controls that or not, but if you click that link, then the "Connect to feed" button, then "NuGet.exe," you get the instructions I used... which apparently are incomplete.

All is working now. Thanks again!

[fmauNeko]

The issue with the "Connect to feed" is that it includes a <clear /> before the actual feed. I myself use a nuget.config like that without issue, and without specifying the original nuget feed again.

<?xml version="1.0" encoding="utf-8"?>
<configuration>
 <packageSources>
    <add key="Pomelo" value="https://pkgs.dev.azure.com/pomelo-efcore/Pomelo.EntityFrameworkCore.MySql/_packaging/pomelo-efcore-public/nuget/v3/index.json" />
 </packageSources>
</configuration>

[fuzl-llc]

Interesting. At one point, I noticed and removed that "clear" (because it cleared out other things I wanted) and it still didn't work for me. Maybe I had something else misconfigured. But anyway, the Nightly Builds instructions worked perfectly.

[mguinness]

Just an update, Laurents has been slogging away and 5.0.0-alpha.1 is now released - thanks for all your hard work, it's appreciated.

[lauxjpn]

As @mguinness already said, 5.0.0-alpha.1 is out there.

This is basically a release, that is pretty much backward compatible with 3.2.4, but also forward compatible with EF Core 5.0. The only not yet fully supported 5.0 feature in there are collations, but it still contains our own collation support we introduced for 3.x. In the next step, we will implement all the planned breaking changes.

At this point in time, all old and new tests by Pomelo and the EF Core team (17,779 tests) are already green on this alpha release, so we expect it to be quite stable.

The general goal here is to have a stable release out by 2020-11-10, that contains all planned breaking changes, so everybody can feel confident to use Pomelo with EF Core 5.0 right from the start.

[baronics]

Just so I can plan out my next few weeks, how close are you with the 5.0.0 stable release? As soon as it's posted, I plan to implement it in a MySQL 8.0.20 project that is using 3.2.4. I would rather wait for the breaking changes (I'm guessing with the latest MySqlConnector) before spending time with the alpha.1 I'm holding off moving to .NET 5 until this is ready.

[lauxjpn]

We will release 5.0.0-alpha.2 tomorrow, which will contain most of the breaking changes that will impact about everyone. All breaking changes should result in compile-time errors, and they should be obvious to fix when using a modern IDE (if they are not, please let us know; the PRs here usually contain a small sample code section, if something was changed that might impact many people).

After the second alpha release tomorrow, there should only be smaller breaking changes to come (but a couple of nice new features).

However, we do only apply changes with every single test working, so we consider even the alpha versions as stable from a runtime point of view. But a new release can still result in compile time errors (e.g. when we adjust a namespace, add a mandatory parameter, or remove an obsolete feature that has been superseded by another one) that you would need to fix in your code.

So everyone should feel encouraged to start using 5.0.0-alpha.2 once it is out.

[lauxjpn]

I would rather wait for the breaking changes (I'm guessing with the latest MySqlConnector) before spending time with the alpha.1

@baronics The latest MySqlConnector release is already part of 5.0.0-alpha.1 and we don't had any issues so far. So there are currently no changes planned regarding MySqlConnector in the upcoming releases.

For changes in 5.0.0-alpha.2, take a look at the milestones (open and closed).

[jmmccota]

It's possible to use version 3.2.4 with net 5?

[lauxjpn]

It's possible to use version 3.2.4 with net 5?

@jmmccota It is possible to use Pomelo 3.2.4 with .NET 5 and EF Core 3.1.9, because Pomelo is .NET Standard 2.0 compliant (see Compatibility) and so is EF Core 3.1.x.

But Pomelo 3.2.4 is not compatible with EF Core 5.0.0 (see Dependencies on nuget.org), so you would need to use our alpha releases or latest nightly builds for that.

[rexebin]

It is amazing that 5.0.0-alpha.1 just worked, replacing 3.2.4 without any issues. Feels like a stable release. Thank you for the hard work!

[lauxjpn]

@rexebin Thanks! That is how it is supposed to be.

As @mguinness already said, 5.0.0-alpha.1 is out there.

This is basically a release, that is pretty much backward compatible with 3.2.4, but also forward compatible with EF Core 5.0. The only not yet fully supported 5.0 feature in there are collations, but it still contains our own collation support we introduced for 3.x. In the next step, we will implement all the planned breaking changes.

So 5.0.0-alpha.1 has intentionally been a release with as little breaking changes but as much backward and forward compatibility as possible. Basically, if you want most of the advantages of EF Core 5, but don't want to make code changes, this is the release to use.

[lauxjpn]

Release 5.0.0-alpha.2 is now available on nuget.org.

[rexebin]

Release 5.0.0-alpha.2 is now available on nuget.org.

Thank you! Upgraded, need to fix one line: UseMySql(connectionString) to UseMySql(connectionString, ServerVersion.AutoDetect(connectionString)), it's up and running.

[lauxjpn]

Yes, there are the following options to choose from for the now mandatory serverVersion parameter for the UseMySql() call:

• Use new MySqlServerVersion() to explicitly specify the MySQL server version.
• Use new MariaDbServerVersion() to explicitly specify the MariaDB server version.
• Use MySqlServerVersion.LatestSupportedServerVersion to explicitly use the latest MySQL server version supported by Pomelo.
• Use MariaDbServerVersion.LatestSupportedServerVersion to explicitly use the latest MariaDB server version supported by Pomelo.
• Use ServerVersion.FromString() and let the call decide the server version from the supplied string parameter (e.g. 8.0.21-mysql or 10.5.5-mariadb).
• Use ServerVersion.AutoDetect() and let Pomelo just get the server version directly from the database server (at the cost of an additional database roundtrip).

As in the past, Pomelo will not use features, that are not available in the specified version, but might leverage newer features, if they are supported.

[baronics]

Is there a reason you just didn't default to ServerVersion.AutoDetect() and let users overload something else? That way it would not make it a breaking change.

[lauxjpn]

Is there a reason you just didn't default to ServerVersion.AutoDetect() and let users overload something else? That way it would not make it a breaking change.

@baronics Yes, these are basically the main points behind this decision:

• Using ServerVersion.AutoDetect() will open a connection to the database when the UseMySql() call is being executed. However, usually with EF Core, a connection is only opened when you actually query the database for the first time (or explicitly call context.Database.OpenConnection()). The connection opened here will also be a dedicated connection that is unrelated to EF Core, if you call the method with a connection string or a connection object that has not been opened yet. The reason behind that is, that the connection string (by itself or within the unopened connection) might include a database name, that does not exist yet (e.g. because you intend to use context.Database.EnsureCreated(); we should rethink, whether this is relevant to unopened connections, but it is relevant to connection strings, which are the go to option for most people). So the method will remove the database name from the connection string and then open a connection to get the server version. This should work by default, but because of the many different database server implementations, and additional restrictions on shared database servers by cloud providers, its also possible that this might just fail if you don't specify your database on some cloud provider instances. And then, depending on where the database is physically located, there might be latency involved for this call, that you want to avoid (e.g. if your database resides in a data center half around the world).
• However, the main reason is that some queries need features like the still quite new LATERAL statement (e.g. translated for complex Include() and Where() calls) or the ROW_NUMBER() PARTITION BY clause (could be translated for SelectMany() calls). So when you move your app from your state-of-the-art local database instance to your last year remote production database instance, you might suddenly run into unexpected issues. Since we support MySQL and MariaDB in different versions, and there are also Aurora and other MySQL compatible database server implementations out there, but the supported EF Core functionality might only be a subset for those different implementations, we would like users to make a conscious choice about the database version they want to support.

---

However, if you guys think that there are strong arguments to make auto detection the default, we can also talk about changing the current implementation.

[codeaid]

@lauxjpn I'm personally totally with you based on the points you made. I prefer manually configuring things, especially those that can be of many similar yet different types (e.g. as it is with different implementations of MySQL). The application might not even need to connect to the database in certain scenarios for all you know so having to do so only to bootstrap it and read the version seems excessive. If the user does want to do that I don't see a big problem with having to add ServerVersion.AutoDetect() to a few lines.

An upgrade to .NET 5.0 is a conscious decision so the fact that code needs to be adjusted in a couple places is probably a very minor inconvenience when migrating to a new major version of the framework.

[lauxjpn]

@baronics @codeaid The main issue that I currently have with the UseMySql() implementation is its lack of helpful xml docs for the serverVersion parameter, which currently looks just like this:

/// <param name="serverVersion"> The version of the database server. </param>

This should be improved, so that its easier to discover how to actually set the serverVersion parameter, with ServerVersion being abstract.

We could also make MySqlServerVersion and MariaDbServerVersion internal, and hide them (a bit similar to how we did it in 3.x) behind a static ServerVersion method with a version and a type parameter. This would mean that all common options would be reachable from the ServerVersion class itself.

[rexebin]

@lauxjpn Thanks for the information. I totally agree the information given here should go to the documents. I am ok with AutoDetect because I am calling Migrate() every time the app starts, so I do access database every time the app starts. But after reading your posts, I think I should provide version manually instead of using AutoDetect, one extra trip to database is still one extra.

I am using Percona which is MySql 8.0 as far as I understand. Is providing major version without the minor versions sufficient? I am going to try MySqlServerVersion.LatestSupportedServerVersion (it works)

[lauxjpn]

Is providing major version without the minor versions sufficient?

@rexebin No, the minimum you currently have to provide is the major and minor version. Take a look at the ServerVersionTest.TestValidVersion test case. It shows working version strings. We will update the opposite test ServerVersionTest.TestInvalidVersion, to show that a string of simply 8 or 8-mysql throws.

I'm currently haven't Percona installed, but would be interested in the result of running a select @@version; statement against it, if you got time.

[rexebin]

@lauxjpn I ran the query against the Percona docker, it returns 8.0.20-11

[khteh]

Any ETA for this?

[adamhathcock]

Checkout the milestones.

The issues outstanding don't seem like huge issues for most people. I do wonder how the maintainers feel of this is ready enough for general use though.

[lauxjpn]

@khteh The alpha versions are generally stable. We just call them alpha because we are still introducing breaking changes before the GA release. However, those are mainly breaking changes that will lead to compile-time errors, that should be pretty easy to fix. So feel free to use the alpha releases or the nightly builds. As for an official GA release date, somewhere before the end of the second December week seems realistic to me.

I am currently revamping the collation and char set support. After that, the first beta shouldn't be far.

[MarkCiliaVincenti]

I'm using alpha2 on a project since its release and I haven't run into any problems whatsoever.

[patil-vaibhav]

Will there be any problem if character set is different than utf8mb4 . I'm using 10.1.44- MariaDB Server for production with utf8_general_ci .

[lauxjpn]

Will there be any problem if character set is different than utf8mb4 . I'm using 10.1.44- MariaDB Server for production with utf8_general_ci .

@patil-vaibhav 5.0.0 should generally handle charsets as before. However, the UseMySql(..., option => option.CharSet()) method is being made obsolete and will be replaced by a HasCharSet() method on the model (like modelBuilder.HasCharSet()), its entities (tables) and their properties (columns; it is already available at the column level since 3.x in Pomelo).

The default will be utf8mb4, so if you use another character set, you should explicitly set it with one of the HasCharSet() options once they are available.