[BUG] The process cannot access the file even on "shared" connection

[alex-jitbit]

Version .NET 5, LiteDB 5.0.11

Describe the bug The process cannot access the file 'xxx' because it is being used by another process. during Upserts and Deletes in a very heavily used web app.

Too bad, I though LiteDB is thread safe.

Should I maybe create one static instance of db and keep using it throughout the app? Will it be thread-safe then?

Full exception

System.IO.IOException: The process cannot access the file 'xxxx' because it is being used by another process.
   at System.IO.FileStream.ValidateFileHandle(SafeFileHandle fileHandle)
   at System.IO.FileStream.CreateFileOpenHandle(FileMode mode, FileShare share, FileOptions options)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options)
   at LiteDB.Engine.FileStreamFactory.GetStream(Boolean canWrite, Boolean sequencial)
   at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
   at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
   at System.Lazy`1.CreateValue()
   at LiteDB.Engine.DiskService..ctor(EngineSettings settings, Int32[] memorySegmentSizes)
   at LiteDB.Engine.LiteEngine..ctor(EngineSettings settings)
   at LiteDB.SharedEngine.OpenDatabase()
   at LiteDB.SharedEngine.Delete(String collection, IEnumerable`1 ids)
   at LiteDB.LiteCollection`1.Delete(BsonValue id)

[alex-jitbit]

Maybe update this article https://github.com/mbdavid/LiteDB/wiki/Concurrency for v5 best practices?

[alex-jitbit]

I tried changing to singleton static instance pattern, with "direct" connection - I get the same "file in use" exception during new LiteDatabase constructor.

I tried using singleton but with shared connection - I get an exception System.ApplicationException: Object synchronization method was called from an unsynchronized block of code.

Mutex.ReleaseMutex()
SharedEngine.CloseDatabase()
SharedEngine.<Query>b__10_0()
SharedDataReader.Dispose(Boolean disposing)
SharedDataReader.Dispose()
LiteQueryable`1.ToDocuments()+MoveNext()
SelectEnumerableIterator`2.MoveNext()
Enumerable.TryGetFirst[TSource](IEnumerable`1 source, Boolean& found)
Enumerable.FirstOrDefault[TSource](IEnumerable`1 source)
LiteCollection`1.FindById(BsonValue id)

Seems like LiteDB is not a good fit for heavy load ASP.NET Core apps with overlapped recycles and all...

[Feofilakt]

Hello! If you initialize LiteDatabase only once (probably at app start), how can you get exception?

[alex-jitbit]

@Feofilakt that's how web apps work. When ASP.NET app is recycled/deployed there are two copies of the app running at the same time (very short time). Global mutexes should solve this problem probably, but ATM LiteDB is not suited for web apps very well.

[9600692024]

Hi Team,

LiteDb: 5.0.11 .NET Framework 4.8

I am also facing the same issue, and I can able to fix this issue, by applying the code changes given in https://github.com/mbdavid/LiteDB/issues/1893

When we can expect the next release or patch update for this fix.

[tjmoore]

I've had this with Connection=Shared and sometimes starts off with an unauthorised access exception on the log file.

This is with a singleton instance of LiteDatabase. This is a service app which has three service processes. While they do each have their own singleton instance, they're all in shared mode and only one of the processes is actively writing to the database. The others barely touch it.

This isn't easily reproducible, it only happens occasionally. It's not an external folder issue in my case as far as I can tell (happens to developers also occasionally, files all on same PC - UPDATE - developer issue was something else. This is only affecting certain customer servers and difficult to reproduce in house).

System.UnauthorizedAccessException: Access to the path 'C:\XXXX\mydb-log.db' is denied.
   at Microsoft.Win32.SafeHandles.SafeFileHandle.CreateFile(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize)
   at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize)
   at LiteDB.Engine.FileStreamFactory.GetStream(Boolean canWrite, Boolean sequencial)
   at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
   at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
   at System.Lazy`1.CreateValue()
   at LiteDB.Engine.DiskService.Dispose()
   at LiteDB.Engine.LiteEngine.Dispose(Boolean disposing)
   at LiteDB.SharedEngine.CloseDatabase()
   at LiteDB.SharedDataReader.Dispose()
   at LiteDB.LiteQueryable`1.ToDocuments()+System.IDisposable.Dispose()
   at System.Linq.Enumerable.SelectEnumerableIterator`2.Dispose()
   at System.Linq.Enumerable.TryGetFirst[TSource](IEnumerable`1 source, Boolean& found)
   at System.Linq.Enumerable.FirstOrDefault[TSource](IEnumerable`1 source)

System.IO.IOException: The process cannot access the file 'C:\XXX\mydb.db' because it is being used by another process.
   at Microsoft.Win32.SafeHandles.SafeFileHandle.CreateFile(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize)
   at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize)
   at LiteDB.Engine.FileStreamFactory.GetStream(Boolean canWrite, Boolean sequencial)
   at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
   at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
   at System.Lazy`1.CreateValue()
   at LiteDB.Engine.DiskService..ctor(EngineSettings settings, Int32[] memorySegmentSizes)
   at LiteDB.Engine.LiteEngine..ctor(EngineSettings settings)
   at LiteDB.SharedEngine.OpenDatabase()
   at LiteDB.SharedEngine.Query(String collection, Query query)
   at LiteDB.LiteQueryable`1.ToDocuments()+MoveNext()
   at System.Linq.Enumerable.SelectEnumerableIterator`2.MoveNext()
   at System.Linq.Enumerable.TryGetFirst[TSource](IEnumerable`1 source, Boolean& found)
   at System.Linq.Enumerable.FirstOrDefault[TSource](IEnumerable`1 source)

I'm getting to think I need to do my own mutex around all database operations, but I believe shared mode is using a mutex anyway. It's slow though.

An alternative is to push all DB operations into a single service and use Direct mode, but it's a big rewrite. Or another option is to jump to Mongo DB (but another big rewrite and have some issues where some customers won't allow a separate DB or can't cope with installing one).

Could this be related to the singleton, holding a file lock somewhere? Operations are short and complete fine, but once it's in this state, it's stuck. Something not being released or disposed inside LiteDB at the end of a shared mode operation?

When using Shared mode there may be little benefit to a singleton vs create and dispose so maybe it's better to switch to the latter.

[tjmoore]

Still getting this with 5.0.15. Hits certain customers only. Can't see that they're on a network share. We store data under %ProgramData% though there's potential that is on a network share, but our diagnostic logs aren't showing network drives.

Restarting the services fixes it for a while then happens again and locked out until restarted.

[tjmoore]

Still happens. Moved to create/dispose LiteDatabase instance per operation and it has resolved the problem of blocking the application for the lifetime of the app, but still the errors are occurring as we log them each time and we have to retry but it could be locked out for a while which causes other issues.

We cannot reproduce this ourselves but are trying to investigate the customer environments, without much success at present. One at least did show MsSense.exe (Microsoft Defender for Endpoint) accessing the database files, but others say they have excluded the data folders and insist there is no AV activity.

We can look at what has file handles using Process Monitor but have to wait for the problem to occur to catch it. Don't suppose when there's a file-in-use exception it reveals which process is using it, as could at least log that. Could put in the exception handler some code to dump the handles I guess.

Other than that we're at a loss to understand what is going on.

[tjmoore]

Update - our mitigations don't solve the issue and app is still blocked and access denied on the database journal file reported, even though that makes no sense the way the app is now designed with a separate thread for database flushing.

Investigating further but one thing have noticed is ProcMon shows DELETE PENDING on the journal file just when access denied occurs.

Will gather info to file a proper bug but I doubt I can easily provide a reproduction especially as this only occurs on some servers. It may or may not be related to the multiple processes we have. I can try to write some simple code mimicking what our app is doing but might need multiple processes and has to be left for days to fall over.

@9600692024 I am also facing the same issue, and I can able to fix this issue, by applying the code changes given in #1893

That seems to be related to shared folders (network share?).

The issue raised here is shared connection, where you may have multiple processes accessing same file, but the file is on a local disc. Also I'm seeing the main problem being the journal file that's frequently created and deleted, not the database file itself.

The fix there might be a workaround for a network share but appears to undo the attempt to guarantee the flush to disc on local folder ?

[mw911]

Hi not sure if this is related. I moved my application using the latest LiteDb version from an AWS server to Azure. Everything seemed to be running fine but sooner or later I get this

System.IO.IOException: The process cannot access the file 'e:\abc\def\DBTasks.db' because it is being used by another process. at System.IO.Error.WinIOError(Int32 errorCode, String maybeFullPath) at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost) at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options) at LiteDB.Engine.FileStreamFactory.GetStream(Boolean canWrite, Boolean sequencial) at LiteDB.Engine.StreamPool.<>c__DisplayClass3_0.<.ctor>b0() at System.Lazy1.CreateValue() at System.Lazy1.LazyInitValue() at System.Lazy1.get_Value() at LiteDB.Engine.DiskService..ctor(EngineSettings settings, EngineState state, Int32[] memorySegmentSizes) at LiteDB.Engine.LiteEngine.Open() at LiteDB.Engine.LiteEngine..ctor(EngineSettings settings) at LiteDB.SharedEngine.OpenDatabase() at LiteDB.SharedEngine.Upsert(String collection, IEnumerable1 docs, BsonAutoId autoId) at LiteDB.LiteCollection1.Upsert(IEnumerable1 entities) at LiteDB.LiteCollection`1.Upsert(T entity)

It happens here Using db As New LiteDB.LiteDatabase($"Filename={DBLite};connection=shared") db.GetCollection(Of Task)("Task").Upsert(Me) End Using

Is anyone having an idea why this could be the case?

Best regards Martin