SqlServer数据库不可用的情况下触发内存溢出问题

[LeaFrock]

问题描述及重现代码：

昨天可能是受某云厂商服务影响，导致某个服务（部署在云服务器A）连接数据库（部署在云服务器B）时候出现连接中断。

查看错误日志，出现Status unavailable没问题，但出现OutOfMemoryException就很奇怪。查了一下相关问题不少，但都没有像我这样遇到内存溢出异常的，所以提个issue看看作者有没有头绪。

错误日志如下：

 ---> System.Exception: 【主库】Status unavailable, waiting for recovery. Exception of type 'System.OutOfMemoryException' was thrown.
 ---> System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at System.Threading.Thread.StartInternal(ThreadHandle t, Int32 stackSize, Int32 priority, Char* pThreadName)
   at System.Threading.Thread.StartCore()
   at System.Threading.Thread.Start(Boolean captureContext, Boolean internalThread)
   at System.Threading.Thread.Start()
   at Microsoft.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal& connection)
   at Microsoft.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection)
   at Microsoft.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)
   at Microsoft.Data.ProviderBase.DbConnectionClosed.TryOpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)
   at Microsoft.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry, SqlConnectionOverrides overrides)
   at Microsoft.Data.SqlClient.SqlConnection.OpenAsync(CancellationToken cancellationToken)
--- End of stack trace from previous location ---
   at FreeSql.SqlServer.SqlServerConnectionPoolPolicy.OnGetAsync(Object`1 obj)
   --- End of inner exception stack trace ---
   at FreeSql.Internal.ObjectPool.ObjectPool`1.GetFree(Boolean checkAvailable)
   at FreeSql.Internal.ObjectPool.ObjectPool`1.GetAsync()
   at FreeSql.Internal.CommonProvider.AdoProvider.ExecuteReaderMultipleAsync(Int32 multipleResult, DbConnection connection, DbTransaction transaction, Func`3 fetchHandler, Action`2 schemaHandler, CommandType cmdType, String cmdText, Int32 cmdTimeout, DbParameter[] cmdParms, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at FreeSql.Internal.CommonProvider.AdoProvider.LoggerException(IObjectPool`1 pool, PrepareCommandResult pc, Exception ex, DateTime dt, StringBuilder logtxt, Boolean isThrowException)
   at FreeSql.Internal.CommonProvider.AdoProvider.ExecuteReaderMultipleAsync(Int32 multipleResult, DbConnection connection, DbTransaction transaction, Func`3 fetchHandler, Action`2 schemaHandler, CommandType cmdType, String cmdText, Int32 cmdTimeout, DbParameter[] cmdParms, CancellationToken cancellationToken)
   at FreeSql.Internal.CommonProvider.Select0Provider`2.ToListMrPrivateAsync[TReturn](String sql, ReadAnonymousTypeAfInfo af, ReadAnonymousTypeOtherInfo[] otherData, CancellationToken cancellationToken)
   at FreeSql.Internal.CommonProvider.Select0Provider`2.InternalToListAsync[TReturn](Expression select, CancellationToken cancellationToken)
   at FreeSql.Internal.CommonProvider.Select1Provider`1.ToListAsync[TReturn](Expression`1 select, CancellationToken cancellationToken)
   at FreeSql.Internal.CommonProvider.Select1Provider`1.ToOneAsync[TReturn](Expression`1 select, CancellationToken cancellationToken)

再往下就是业务代码的堆栈了，略过。该类型错误日志集中在1s内，大约100条（请求）。

还有个别错误日志如下：

 ---> System.InvalidOperationException: BeginExecuteReader requires an open and available Connection. The connection's current state is closed.
   at Microsoft.Data.SqlClient.SqlCommand.<>c.<ExecuteDbDataReaderAsync>b__188_0(Task`1 result)
   at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke()
   at System.Threading.Tasks.Task.<>c.<.cctor>b__273_0(Object obj)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--- End of stack trace from previous location ---
   at FreeSql.Internal.CommonProvider.AdoProvider.ExecuteReaderMultipleAsync(Int32 multipleResult, DbConnection connection, DbTransaction transaction, Func`3 fetchHandler, Action`2 schemaHandler, CommandType cmdType, String cmdText, Int32 cmdTimeout, DbParameter[] cmdParms, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at FreeSql.Internal.CommonProvider.AdoProvider.LoggerException(IObjectPool`1 pool, PrepareCommandResult pc, Exception ex, DateTime dt, StringBuilder logtxt, Boolean isThrowException)
   at FreeSql.Internal.CommonProvider.AdoProvider.ExecuteReaderMultipleAsync(Int32 multipleResult, DbConnection connection, DbTransaction transaction, Func`3 fetchHandler, Action`2 schemaHandler, CommandType cmdType, String cmdText, Int32 cmdTimeout, DbParameter[] cmdParms, CancellationToken cancellationToken)
   at FreeSql.Internal.CommonProvider.Select0Provider`2.ToListMrPrivateAsync[TReturn](String sql, ReadAnonymousTypeAfInfo af, ReadAnonymousTypeOtherInfo[] otherData, CancellationToken cancellationToken)
   at FreeSql.Internal.CommonProvider.Select0Provider`2.InternalToListAsync[TReturn](Expression select, CancellationToken cancellationToken)
   at FreeSql.Internal.CommonProvider.Select1Provider`1.ToListAsync[TReturn](Expression`1 select, CancellationToken cancellationToken)
   at FreeSql.Internal.CommonProvider.Select1Provider`1.ToOneAsync[TReturn](Expression`1 select, CancellationToken cancellationToken)

数据库版本

SqlServer 2016

安装的Nuget包

FreeSql.Provider.SqlServer 3.2.802

.net framework/. net core？ 及具体版本

.NET 7.0.13

[2881099]

因为io出现的问题，异步长时间不能完成 大慨是异步任务堆积太多造成的内存溢出

[LeaFrock]

异步长时间不能完成

是要在连接字符串那里添加连接超时配置？还是说FreeSql可以在底层增加异步超时处理的策略？

[2881099]

整个异步链条可能都会有问题，不止是 FreeSql。

异步链条卡死，线程池被耗尽。

[LeaFrock]

异步链条卡死，线程池被耗尽

这个不太可能，或者说很奇怪。因为堆栈上层的接口业务逻辑都使用的异步API，不存在sync in async的问题，理论上不该出现线程饥饿的情况。

假设真出现了线程池被耗尽的情况，那向Redis、RPC等其他服务的异步请求，都有几率报类似错误才对。但看下来只有FreeSql这里的堆栈异常。

[2881099]

sync in async 理论上不存在，但会有大量的 HttpRequest 入口进来占用线程池

前面的任务处理不完，后面一直有新的请求打入。

[LeaFrock]

您对我这边能做什么有建议吗？