search

docintelapp / DocIntel

Open Source Platform for storing, organizing, and searching documents related to cyber threats
https://docintel.org
Other
154 stars 25 forks source link

Failure in OnConfiguring #40

Closed brennane closed 1 year ago

brennane commented 1 year ago

This seems to be part of a general set of issues related to the dbcontext and async (unsafe) or threading (unsafe) : https://learn.microsoft.com/en-us/ef/core/dbcontext-configuration/#avoiding-dbcontext-threading-issues (based on an error message)

this may be a repeat of what I reported over slack. Adding here a note on an issue with OnConfiguring blowing up. I don't know if this is because I am running this on a larger multi-node system or note, but I hope these error message help. Cheers.

2022-12-16 21:50:27.2542 [INFO] [RunMethodsSequentially.LockAndRunCode.GetLockAndThenRunServices] The startup service class [BaseDataDbService] was successfully executed.
2022-12-16 21:50:28.0991 [INFO] [RunMethodsSequentially.LockAndRunCode.GetLockAndThenRunServices] The startup service class [InstallSynapseCustomObjects] was successfully executed.
2022-12-16 21:50:28.1219 [INFO] [Microsoft.Hosting.Lifetime] Application started. Press Ctrl+C to shut down.
2022-12-16 21:50:28.1219 [INFO] [Microsoft.Hosting.Lifetime] Hosting environment: Production
2022-12-16 21:50:28.1219 [INFO] [Microsoft.Hosting.Lifetime] Content root path: /app
2022-12-16 22:05:24.9277 [INFO] [DocIntel.Services.TagIndexer.TagIndexerTimedConsumer] Timed Hosted Service is working. Count: 1
2022-12-16 22:05:24.9289 [INFO] [DocIntel.Services.TagIndexer.TagFacetIndexerTimedConsumer] Timed Hosted Service is working. Count: 1
2022-12-16 22:05:24.9377 [ERROR] [DocIntel.Core.Services.DynamicContextConsumer] An attempt was made to use the context instance while it is being configured. A DbContext instance cannot be used inside 'OnConfiguring' since it is still being configured at this point. This can happen if a second operation is started on this context instance before a previous operation completed. Any instance members are not guaranteed to be thread safe.
Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.
   at DocIntel.Core.Repositories.EFCore.TagFacetEFRepository.GetAllAsync(AmbientContext ambientContext, Func`2 query)+MoveNext() in /src/DocIntel.Core/Repositories/EFCore/TagFacetEFRepository.cs:line 195
   at DocIntel.Core.Repositories.EFCore.TagFacetEFRepository.GetAllAsync(AmbientContext ambientContext, Func`2 query)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
   at System.Linq.AsyncEnumerable.<ToListAsync>g__Core|424_0[TSource](IAsyncEnumerable`1 source, CancellationToken cancellationToken) in /_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/ToList.cs:line 36
   at System.Linq.AsyncEnumerable.<ToListAsync>g__Core|424_0[TSource](IAsyncEnumerable`1 source, CancellationToken cancellationToken) in /_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/ToList.cs:line 36
   at DocIntel.Services.TagIndexer.TagFacetIndexerTimedConsumer.DoWork(Object state) in /src/DocIntel.Services.TagIndexer/TagFacetIndexerTimedConsumer.cs:line 72
   at System.Threading.Tasks.Task.<>c.<ThrowAsync>b__128_1(Object state)
   at System.Threading.QueueUserWorkItemCallback.Execute()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
breevans@Arrow Docintel-K8S % Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.

also

2022-12-16 22:35:37.2989 [INFO] [DocIntel.Services.TagIndexer.TagIndexerTimedConsumer] Timed Hosted Service is working. Count: 1
2022-12-16 22:35:37.2994 [INFO] [DocIntel.Services.TagIndexer.TagFacetIndexerTimedConsumer] Timed Hosted Service is working. Count: 1
2022-12-16 22:35:37.3160 [ERROR] [Microsoft.EntityFrameworkCore.Query] An exception occurred while iterating over the results of a query for context type 'DocIntel.Core.Models.DocIntelContext'.
System.InvalidOperationException: A second operation was started on this context instance before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913.
   at Microsoft.EntityFrameworkCore.Infrastructure.Internal.ConcurrencyDetector.EnterCriticalSection()
   at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.AsyncEnumerator.MoveNextAsync()
2022-12-16 22:35:37.3167 [ERROR] [DocIntel.Core.Services.DynamicContextConsumer] A second operation was started on this context instance before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913.
Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.
   at DocIntel.Core.Repositories.EFCore.TagEFRepository.GetAllAsync(AmbientContext ambientContext, Func`2 query)+MoveNext() in /src/DocIntel.Core/Repositories/EFCore/TagEFRepository.cs:line 286
   at DocIntel.Core.Repositories.EFCore.TagEFRepository.GetAllAsync(AmbientContext ambientContext, Func`2 query)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
   at System.Linq.AsyncEnumerable.<ToListAsync>g__Core|424_0[TSource](IAsyncEnumerable`1 source, CancellationToken cancellationToken) in /_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/ToList.cs:line 36
   at System.Linq.AsyncEnumerable.<ToListAsync>g__Core|424_0[TSource](IAsyncEnumerable`1 source, CancellationToken cancellationToken) in /_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/ToList.cs:line 36
   at DocIntel.Services.TagIndexer.TagIndexerTimedConsumer.DoWork(Object state) in /src/DocIntel.Services.TagIndexer/TagIndexerTimedConsumer.cs:line 72
   at System.Threading.Tasks.Task.<>c.<ThrowAsync>b__128_1(Object state)
   at System.Threading.QueueUserWorkItemCallback.Execute()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()

reproduction: Loaded https://blog.talosintelligence.com/get-a-loda-this/ into document UI, add "new" tags "Talos", "Loda" - from that page. (I only have "France", "United Kingdom", "Ireland" added to a geography facet pre-made.)

ancailliau commented 1 year ago

This should be solved with the latest docker images. Could you test?