[BUG] Detected loop in FindAll({0})

[kamilwarowny]

Version .Net 8.0 LiteDB.dll 5.0.21.0

Describe the bug About 16000 records in file. File size 33MB

Code to Reproduce

using (var db = new LiteDatabase(dbFilePAth))
{                   
    var col = db.GetCollection<UserHash>(CollectionName);

    var userHashes = col.Query().ToList();

    db.Checkpoint();

    db.Rebuild();

    return userHashes;
}

line db.Rebuild(); throws error below:

at LiteDB.Engine.IndexService.FindAll(CollectionIndex index, Int32 order)+MoveNext()
at LiteDB.LinqExtensions.<>cDisplayClass20`2.<g|0>d.MoveNext()
at LiteDB.Engine.LiteEngine.<>c__DisplayClass5_0.b__0(TransactionService transaction)
at LiteDB.Engine.LiteEngine.AutoTransaction[T](Func`2 fn)
at LiteDB.Engine.LiteEngine.EnsureIndex(String collection, String name, BsonExpression expression, Boolean unique)
at LiteDB.Engine.LiteEngine.RebuildContent(IFileReader reader)
at LiteDB.Engine.RebuildService.Rebuild(RebuildOptions options)
at LiteDB.Engine.LiteEngine.Rebuild(RebuildOptions options)
at LiteDB.LiteDatabase.Rebuild(RebuildOptions options)

[holychaos]

I had this problem too. I tried to move the 'db' file out of the original path, run the program once to generate a new 'db' file, and finally overwrite the moved 'db' file back, and everything worked.

[marss72]

Just BTW, not regarding your issue, but it might be possible to improve the provided code by maybe better performance and clarity.

If it's really necessary to use List, ignore this one. But you can improve memory efficiency and lazily load all the entries. I am not an expert on LiteDB so this might cause issues since you are rebuilding it right after.

Also, you can modify col.Query().ToList(); to use FindAll() instead of the Query(). Try it yourself, you might get a small performance benefit or not, but I haven't measured it.

[k0ka]

I have the same problem. It happens if any table on rebuild has more than 2550 records. Here is why: 1) RedbuildService creates a new database https://github.com/mbdavid/LiteDB/blob/c4db4ab3b1e594439921d594f403775c82f9e8c7/LiteDB/Engine/Services/RebuildService.cs#L55 2) It starts rebuilding it using the main IFileReader https://github.com/mbdavid/LiteDB/blob/c4db4ab3b1e594439921d594f403775c82f9e8c7/LiteDB/Engine/Engine/Rebuild.cs#L52 3) When it creates IndexService it passes _disk.MAX_ITEM_COUNT from the newly created database. Which means it is always minimum value of 2550. https://github.com/mbdavid/LiteDB/blob/c4db4ab3b1e594439921d594f403775c82f9e8c7/LiteDB/Engine/Engine/Rebuild.cs#L63

I'm not sure how to properly fix it. The easiest way would be to just pass the correct MAX_ITEM_COUNT from the RebuildService where it still knows this value.