search

mbdavid / LiteDB

LiteDB - A .NET NoSQL Document Store in a single data file
http://www.litedb.org
MIT License
8.49k stars 1.24k forks source link

LiteDB Studio takes forever to connect 2gb data file #1312

Open vickyRathee opened 5 years ago

vickyRathee commented 5 years ago

I have 2 gb of Data file and LiteDb Studio takes forever to connect. I guess it's trying to read all the collection/documents in memory for something while connection because the memory usage just keeps increasing in task manager.

I am closing it now after 3 minutes.

Screen 1

image

Screen 2

image

mbdavid commented 4 years ago

Hi @vickyrathee, yes, there is no memory managment in LiteDB Studio if you query big documents (limit only in 1000 document, but if each document are too big LiteDB Studio will crash).

I will implement some memory limit result according document size. Will add this issue in my project

vickyRathee commented 4 years ago

@mbdavid thanks.

To give more details - I was not querying, just connecting to database was taking too long. Because it's upgrading the old DB file from v4 to v5 i guess. So, you can review the skip/take to fetch documents in 10k batches in upgrading the db file as well.

mbdavid commented 4 years ago

If you are just open a old database, this long time was to convert into new version. Use around 1Gb is default (has some variables can be drop this size but will take long time to process).

Maybe a better user message or upgrade progress bar can help in this case.