dotnet / efcore

EF Core is a modern object-database mapper for .NET. It supports LINQ queries, change tracking, updates, and schema migrations.
https://docs.microsoft.com/ef/
MIT License
13.79k stars 3.19k forks source link

EntityFrameworkCore large Insert works in debug mode only #6333

Closed bricelam closed 2 years ago

bricelam commented 8 years ago

@Elrashid originally asked this on Stack Overflow, but I think we should investigate:

I used the insert code -'shown down' - tho insert sample data to MS SQL DB . one of the tables -'shown down' - (last table to insert 'Records') has large insert it takes 5 minutes in debug mode but when i try to start the project without debugging the table is empty in the DB and IIS-express is not throwing errors

The data has over 36,000 rows and over 40 columns

I assumed maybe because a "Connection Timeout"

so I tried to search for a solution and found two which none of them worked:

1.connection string

Persist Security Info=False;Connect Timeout=1000;

2.Database.SetCommandTimeout

public MainContext()
    {
        //base.Database.SetCommandTimeout(1000);
    }

Her the insert code

Modified from https://github.com/aspnet/MusicStore/blob/dev/samples/MusicStore/Models/SampleData.cs


private static async Task AddOrUpdateAsync<TEntity>(
    IServiceProvider serviceProvider,
    Func<TEntity, object> propertyToMatch, 
    IEnumerable<TEntity> entities)
    where TEntity : class
{
    Exception exp = null;
    using (var scope = serviceProvider
        .GetRequiredService<IServiceScopeFactory>()
        .CreateScope())
    using (var db = scope
        .ServiceProvider
        .GetService<MainContext>())
    using (var transaction = db.Database.BeginTransaction())
    {
        try
        {
            db.ChangeTracker.AutoDetectChangesEnabled = false;
                db.Set<TEntity>().AddRange(entities);
                db.SaveChanges();
                transaction.Commit();
        }
        catch (Exception ex)
        {
            exp = ex;
            throw ex;
        }
    }

    if (exp != null)
    {
        throw exp;
    }
}

ajcvickers commented 8 years ago

What version of EF Core? We had exponential perf inserting large number of entities prior to RTM. Also, is it UWP--when I see "works in debug mode" now that is my immediate thought. :-)

Elrashid commented 8 years ago

the project is asp.net here the project.json file { "userSecretsId": "*****", "webroot": "wwwroot", "dependencies": { "Microsoft.NETCore.App": { "version": "1.0.0-rc2-3002702" }, "AutoMapper": "4.1.1", "Microsoft.AspNetCore.SpaServices": "1.0.0-beta-000002", "Microsoft.AspNetCore.Authentication.JwtBearer": "1.0.0-rc2-final", "AspNet.Security.OpenIdConnect.Server": "1.0.0-beta5-final", "Microsoft.AspNetCore.Http.Extensions": "1.0.0-rc2-final", "Microsoft.ApplicationInsights.AspNetCore": "1.0.0-rc2-final", "Microsoft.AspNetCore.Authentication.Cookies": "1.0.0-rc2-final", "Microsoft.AspNetCore.Diagnostics": "1.0.0-rc2-final", "Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore": "1.0.0-rc2-final", "Microsoft.AspNetCore.Identity.EntityFrameworkCore": "1.0.0-rc2-final", "Microsoft.AspNetCore.Mvc": "1.0.0-rc2-final", "Microsoft.AspNetCore.Razor.Tools": { "version": "1.0.0-preview1-final", "type": "build" }, "Microsoft.AspNetCore.Server.IISIntegration": "1.0.0-rc2-final", "Microsoft.AspNetCore.Server.Kestrel": "1.0.0-rc2-final", "Microsoft.AspNetCore.StaticFiles": "1.0.0-rc2-final", "Microsoft.EntityFrameworkCore": "1.0.0-rc2-final", "Microsoft.EntityFrameworkCore.SqlServer": "1.0.0-rc2-final", "Microsoft.EntityFrameworkCore.Tools": { "version": "1.0.0-preview1-final", "type": "build" }, "Microsoft.Extensions.Configuration.EnvironmentVariables": "1.0.0-rc2-final", "Microsoft.Extensions.Configuration.Json": "1.0.0-rc2-final", "Microsoft.Extensions.Configuration.UserSecrets": "1.0.0-rc2-final", "Microsoft.Extensions.Logging": "1.0.0-rc2-final", "Microsoft.Extensions.Logging.Console": "1.0.0-rc2-final", "Microsoft.Extensions.Logging.Debug": "1.0.0-rc2-final", "Microsoft.VisualStudio.Web.BrowserLink.Loader": "14.0.0-rc2-final", "Microsoft.VisualStudio.Web.CodeGeneration.Tools": { "version": "1.0.0-preview1-final", "type": "build" }, "Microsoft.VisualStudio.Web.CodeGenerators.Mvc": { "version": "1.0.0-preview1-final", "type": "build" }, "Core.BD": "1.0.0-", "Core.Report": "1.0.0-*" },

"tools": { "Microsoft.AspNetCore.Razor.Tools": { "version": "1.0.0-preview1-final", "imports": "portable-net45+win8+dnxcore50" }, "Microsoft.AspNetCore.Server.IISIntegration.Tools": { "version": "1.0.0-preview1-final", "imports": "portable-net45+win8+dnxcore50" }, "Microsoft.EntityFrameworkCore.Tools": { "version": "1.0.0-preview1-final", "imports": [ "portable-net45+win8+dnxcore50", "portable-net45+win8" ] }, "Microsoft.Extensions.SecretManager.Tools": { "version": "1.0.0-preview1-final", "imports": "portable-net45+win8+dnxcore50" }, "Microsoft.VisualStudio.Web.CodeGeneration.Tools": { "version": "1.0.0-preview1-final", "imports": [ "portable-net45+win8+dnxcore50", "portable-net45+win8" ] } },

"frameworks": { "netcoreapp1.0": { "imports": [ "dotnet5.6", "dnxcore50", "portable-net45+win8" ] } },

"buildOptions": { "emitEntryPoint": true, "preserveCompilationContext": true },

"runtimeOptions": { "gcServer": true },

"publishOptions": { "include": [ "wwwroot", "Views", "appsettings.json", "web.config" ] },

"scripts": { "prepublish": [ "npm install", "bower install"], "postpublish": [ "dotnet publish-iis --publish-folder %publish:OutputPath% --framework %publish:FullTargetFramework%" ] }, "runtimes": { "win10-x64": { } } }


here the project.json file for "Core.BD": "1.0.0-*"

{ "version": "1.0.0-*",

"dependencies": { "Microsoft.EntityFrameworkCore": "1.0.0-rc2-final", "Microsoft.EntityFrameworkCore.Relational": "1.0.0-rc2-final", "NETStandard.Library": "1.5.0-rc2-24027", "Newtonsoft.Json": "8.0.3", "System.ComponentModel.Annotations": "4.1.0-rc2-24027" },

"frameworks": { "netstandard1.5": { "imports": [ "dnxcore50", "portable-net452+win81" ] } } }

bricelam commented 8 years ago

Could you try again using the nightly builds?

divega commented 8 years ago

@ajcvickers please find the duplicate bug.

ajcvickers commented 8 years ago

Dupe of #4831