When using multiple threads to call GetFile<T>, the lock was being held per SqPack, which effectively is a .dat file. Changed that to lock per file inside a .dat file, and added SqPackStreamPool? GameData.StreamPool which will reuse already opened file handles to avoid performance hit from creating handle per every file accessed.
The attached test code iterates 800,054 paths from 040000. The test was done on 16 P-threads on 13900k.
When using multiple threads to call
GetFile<T>, the lock was being held perSqPack, which effectively is a.datfile. Changed that to lock per file inside a.datfile, and addedSqPackStreamPool? GameData.StreamPoolwhich will reuse already opened file handles to avoid performance hit from creating handle per every file accessed.The attached test code iterates 800,054 paths from
040000. The test was done on 16 P-threads on 13900k.SqPackStreamPool_TestCode.txt SqPackStreamPool_TestResult.txt
Notes
Microsoft.Extensions.ObjectPoolhas been added as a dependency.GameDatais now implementsIDisposable, so that it disposesInstanceOfGameData.StreamPoolalong with it.SqPack.FileCacheandSqPack.CacheLockare now obsolete, and they do nothing.