XLocalizer.DB database concurrency problem

morgrowe commented 2 years ago

Hi Ziya

Hope you're well.

I've been having intermittent issues for the past couple of months with EF complaining about two instances of a DbContext being used at the same time. I recently looked at the logs to try and figure out what's going on and it seems in all cases XLocalizer.DB is involved. The exception I'm referring to is:

System.InvalidOperationException: A second operation was started on this context instance before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext.

I may be misinterpreting the stack trace, but the exception seems to occur within:

XLocalizer.DB.EF.EFDbResourceProvider.TryGetValue<TResource>(string, string)

every time I see it in my logs.

I've attached a couple of stack traces in case that helps. It's worth noting that this doesn't happen all the time. It's hard to reproduce, but it seems to happen when I excessively refresh a page that uses the localize-content tag helper somewhere on it.

If I can provide anymore information or provide additional stack traces, please let me know.

Thanks Morgan

In application 1:

XLocalizer.TagHelpers 1.1.0
XLocalizer 1.0.3
XLocalizer.DB 1.0.3

st-1.txt st-2.txt st-3.txt st-4.txt st-5.txt st-6.txt

In application 2:

XLocalizer.TagHelpers 1.0.0
XLocalizer 1.0.0
XLocalizer.DB 1.0.0

other-app-st-1.txt

morgrowe commented 2 years ago

I have no evidence, but I have a sneaky feeling it might be something to do with:

https://github.com/LazZiya/XLocalizer.DB/blob/master/XLocalizer.DB/EF/EFDbResourceProvider.cs#L41

var culture = CultureInfo.CurrentCulture.Name;

I normally use Thread.CurrentThread.CurrentCulture.Name instead as apparently it's a thread-safe way of accessing the current culture.

LazZiya commented 2 years ago

Hi @morgrowe

I'm fine thanks, hope you are fine too :)

Before I deep dive into the details, do you have below settings as described in XLocalizer.DB docs:

ExpressMemeoryCache enabled in production

DbContext must be registered as Transient

services.AddDbContext<ApplicationDbContext>(options =>
    options.UseSqlServer(Configuration.GetConnectionString("DefaultConnection")), 
    ServiceLifetime.Transient, 
    ServiceLifetime.Transient);

tldr;

As you already know, the DbContext is not thread safe, and it is registered as a scoped service.

On the other hand, all localization services IStringLocalizer, IHtmlLocalizer are singleton services and they expect all their dependencies to be Singleton or Transient, because a Scoped service can not be consumed inside a Singleton service. And here was the challenge to implement XLocalizer.DB life cycle mode!

Since we cannot change the life cycle of localization services, the only solution was to register the DbContext as Transient because it can be consumed inside Singleton localization services.

There are some drawback of this implementation, but it should not affect XLocalizer.DB operations if the ExpressMemoryCache is enabled and consider using multiple DbContexts in your app, where the application DbContext is registered as Scoped by default and XLocalizer.DB DbContext can be registered as Transient so the other services in your application that are using DbContext will not be affected by the Transient lifetime of the localization DB.

Best practices

Here you can find more about XLocalizer best practices.

Best, Ziya

morgrowe commented 2 years ago

Hi Ziya

I'm very good thanks. Glad to hear you're well, too.

Here's the registration of my DbContext. Looks like it's setup to be Transient as your docs advise:

var connectionString = "MyConnectionString";
var migrationsAssembly = typeof(ManagerDbContext).Assembly.FullName;

services.AddDbContext<ManagerDbContext>(options =>
{
    options.UseSqlServer(connectionString, b => b.MigrationsAssembly(migrationsAssembly));
}
, ServiceLifetime.Transient
ServiceLifetime.Transient);

And here are my AddXDbLocalizer options:

.AddXDbLocalizer<ManagerDbContext, DummyTranslator, AppLocalizationResource>((opts) =>
{
    opts.AutoTranslate = false;
    opts.LocalizeDefaultCulture = true;
});

I think I was using ExpressMemeoryCache, but I think I removed the option before LocalizeDefaultCulture was added. I may need to add that back for some perf gains. :)

Thanks for the link to the pros and cons of using a Transient DbContext. I noticed that I wasn't able to return tracked entities from one class to another, so that explains why!

Your suggestion of having two DbContexts is a welcome one. Despite having multiple DbContexts in this application, it didn't even occur to me to have one just for XDbLocalization. Should I try implementing that first to see if that resolves the issue I posted about? Or would you like to do some investigation first with the information I've provided?

Thanks Morgan

LazZiya commented 2 years ago

So ExpressMemoryCache will help a lot in this case. Additionally, I would always recommend a dedicated DB for localization services, this will keep your other application services not affected by the Transient implementation of localization DB service, and you will have a clean DB architecture.

For the issue, it will take some time, I can't promise to bring a solution in a short time, but the above recommendations would help to avoid the problem :)

morgrowe commented 2 years ago

Ok, I will enable ExpressMemoryCache and decouple the XDbLocalizer tables from my application's DbContext in order to load just the XDbLocalizer's DbContext using Transient scoping. That'll let me add my application's DbContext back as Scoped.

Many thanks for agreeing to launch an investigation. In the meantime, I'll share any exceptions relating to this issue with you should they pop up again over the next few days. :)

Thank you very much Morgam

LazZiya commented 2 years ago

Thank you too for your cooperation :)

Best, Ziya

morgrowe commented 2 years ago

Hi Ziya

I didn't have much time to do development this week, but I did get two exceptions relating to this issue yesterday afternoon. They happened a couple of milliseconds after each other like the other ones. These occurred after setting UseExpressMemoryCache = true and creating a separate DbContext for the resources (LocalizationDbContext).

caforb-ex-wk2.txt

Cheers Morgan

morgrowe commented 1 year ago

Hi Ziya

Hope you're well. I can't believe almost a year has passed since I brought this up.

We're still getting the exception (even in Release mode running on IIS). Did you have a chance to look into this?

Thanks Morgan

LazZiya commented 1 year ago

Hi Morgan

I am fine thanks, hope you also fine.

Yea time is passing fast! also projects are growing rapidly... Unfortunately I couldn't really afford time to deep dive into this issue :(

Any PR are welcome :)

Best regards, Ziya

morgrowe commented 1 year ago

Hi Ziya

Ok, no worries. I had a look into this a couple of months ago, but I couldn't find a solution.

I will have another look. If I fix it, I'll do a pull request.

Thanks Morgan

Sayed-Mahmoud commented 1 year ago

I have the same problem when using Arabic! .Net Core 5 Razor Pages with EF Core

LazZiya / XLocalizer

XLocalizer.DB database concurrency problem #28

tldr;

Best practices