PomeloFoundation / Pomelo.EntityFrameworkCore.MySql

Entity Framework Core provider for MySQL and MariaDB built on top of MySqlConnector
MIT License
2.69k stars 381 forks source link

"How to automatically reclaim memory usage when SaaS application resources are limited #676

Closed qibunext closed 5 years ago

qibunext commented 6 years ago
{
  "runtimeOptions": {
    "tfm": "netcoreapp2.1",
    "framework": {
      "name": "Microsoft.AspNetCore.App",
      "version": "2.1.2"
    },
    "configProperties": {
      "System.GC.Concurrent": false,
      "System.GC.Server": false
    }
  }
}

SaaS应用资源有限的情况下如何自动回收内存使用量 数据访问使用的是 Entity framework + Pomelo.EntityFrameworkCore.MySql 服务器配置 2核4G内存 当应用程序运行在.net core 2.0上时内存使用量会缓存增加,但使用量最大化时会自动重启应用程序,会停止一会,(目前只能先用.net core 2.0) 应用程序运行在.net core 2.1上时内存使用量也会缓存增加,使用量最大化时不会自动重启,一直停留在最大的使用量。导致静态文件访问不了,mysql数据库访问也会超时

Translation:

"How to automatically reclaim memory usage when SaaS application resources are limited Data access using Entity framework + Pomelo.EntityFrameworkCore.MySql server configuration 2 core 4G memory When the application runs on .net core 2.0, the memory usage will increase, but when the usage is maximized, the application will be restarted automatically and will stop for a while. (At present, only .net core 2.0 can be used first) When the application runs on .net core 2.1, the memory usage will also increase. When the usage is maximized, it will not restart automatically, and it will stay at the maximum usage. Resulting in static file access, mysql database access will also time out"

mguinness commented 6 years ago

You haven't specified which version of Pomelo.EntityFrameworkCore.MySql you're using (as well as OS /MySQL version as requested in the new issue template). There was a memory leak discussed in https://github.com/PomeloFoundation/Pomelo.EntityFrameworkCore.MySql/issues/668 which was resolved. Please try the latest build and see if that resolves your problem.

mguinness commented 6 years ago

@qibunext Did you try the latest build? Just providing dumps isn't particularly useful. It appears that you're using Linux, can you reproduce the issue under Windows?

qibunext commented 6 years ago

@mguinness When the server is not busy, the memory is not recycled.Can you help me find the reason? I am using the latest version

qibunext commented 6 years ago
<ItemGroup>
    <PackageReference Include="DotNetCore.NPOI" Version="1.2.0" />
    <PackageReference Include="Microsoft.AspNetCore.App" Version="2.1.3" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.Sqlite" Version="2.1.2" />

    <PackageReference Include="MimeKit" Version="2.0.6" />
    <PackageReference Include="MySql.Data.EntityFrameworkCore" Version="8.0.12" />
    <PackageReference Include="Microsoft.Win32.Registry" Version="4.5.0" />
    <PackageReference Include="OSS.PaySdk.Ali" Version="1.2.0.1" />
    <PackageReference Include="OSS.PaySdk.Wx" Version="1.2.0.3" />
    <PackageReference Include="Pomelo.EntityFrameworkCore.MySql" Version="2.1.2" />
    <PackageReference Include="QRCoder" Version="1.3.3" />
    <PackageReference Include="Senparc.CO2NET" Version="0.2.10" />
    <PackageReference Include="Senparc.Weixin.MP" Version="16.1.1" />
    <PackageReference Include="Senparc.Weixin.MP.MVC" Version="7.1.1" />
    <PackageReference Include="Senparc.Weixin.TenPay" Version="0.2.0" />
    <PackageReference Include="Senparc.Weixin.Work" Version="3.1.1" />
    <PackageReference Include="System.Drawing.Common" Version="4.5.0" />
    <PackageReference Include="TimedJob" Version="2.0.0" />
  </ItemGroup>
mguinness commented 6 years ago

Just an observation, why do you have a reference to MySql.Data.EntityFrameworkCore?

qibunext commented 6 years ago

MySql.Data.EntityFrameworkCore Not used, just used to test

qibunext commented 6 years ago

Entity Framework Core 2.1.3-rtm-32065 initialized 'DataContext' using provider 'Pomelo.EntityFrameworkCore.MySql' with options: None

mguinness commented 6 years ago

Also try moving runtimeOptions to your csproj file and try the ServerGarbageCollection option. You might also try looking for help on Stack Overflow as there are probably more knowledgeable people there regarding memory usage and GC.

qibunext commented 6 years ago
    "configProperties": {
      "System.GC.Concurrent": false,
      "System.GC.Server": false
    }

There are settings, the problem is still

mguinness commented 6 years ago

Take a look at https://github.com/dotnet/coreclr/issues/18577 & https://github.com/dotnet/corefx/issues/30776 for similar issues. You'll probably have more success discussing your issue with those folks.

qibunext commented 6 years ago

@mguinness https://github.com/aspnet/EntityFrameworkCore/issues/8223 Is it possible to be this problem?

有没有可能是这个问题?

mguinness commented 6 years ago

I cannot say, but have you tried each approach to see if there's any difference?

qibunext commented 6 years ago

@mguinness I just tried it, still no obvious effect, the memory is still not recycled. 我刚刚试了,依然没有明显效果,内存还是不回收

mguinness commented 6 years ago

Is this a web or console app? What db operations are you doing? Have you tried another provider?

qibunext commented 6 years ago

@mguinness There are 120 or so memories in the site that are not enough to recycle without problems, but when there are many sites, 330 sites will have memory leaks.

qibunext commented 6 years ago

When the server does not have any traffic, the memory is not recycled.