search

JabRef / jabref

Graphical Java application for managing BibTeX and biblatex (.bib) databases
https://devdocs.jabref.org
MIT License
3.55k stars 2.48k forks source link

huge memory usage on big library (> 1000 items) #10209

Open ghost opened 1 year ago

ghost commented 1 year ago

JabRef version

5.9 (latest release)

Operating system

Windows

Details on version and operating system

Windows 10 22H2

Checked with the latest development build

Steps to reproduce the behaviour

When opening a big library (45974 items), the memory usage of JabRef goes up to 1.6 GB RAM . Is this normal or are there memory problems?

When opening 2 big libraries at the same time (45974 items) and (134352 items), the memory usage is at a whopping 2.4 GB RAM.

JabRef running without any library uses 960 MB RAM on my Windows PC.

Also when those 2 big libraries are opened at the same time, JabRef becomes slow when scrolling the entry list of the library. When JabRef is running some minutes on the libraries, the slow scrolling dissapears after some time

Is JabRef not designed for those kind of big libraries or is this a problem in the code I found?

Attached the 2 databases

games.zip

Appendix

...

Log File ``` Paste an excerpt of your log file here ```
ghost commented 1 year ago

In 5.10, the memory usage got up with another 200 MB.

After loading both databases, I have 2.6 GB RAM usage instead of 2.4 GB RAM on version 5.9. So for those 2 specific database, the development version uses more memory then version 5.9.

I also didn't notice any speed improvement in loading/scrolling the items of the library.

ThiloteE commented 1 year ago

This is not the first time this has come up. If you go through issues with the label "performance", you will find a few things to look out for.

In most cases, using lots of RAM is not something that impacts performance, as RAM is faster than your hard-drive. It is also possible to cache the results of certain computing processes into the RAM instead of computing them over and over, again and again and this can increase performance.

Yes, JabRef uses a lot more RAM than other programms in comparison. This is partly by design and partly, because JabRef has been around for a long time and the code base is pretty big, so there is room for optimization. Also, JabRef is written in Java and Java by design uses lots of RAM https://en.wikipedia.org/wiki/Java_performance#Memory_use.

In general, as long as the RAM usage stays roughly constant, there is no problem. If your RAM usage constantly increases, without ever stopping and without you doing anything, then this would be a memory leak and would be a bug and needed to be fixed.

I am more concerned about JabRef becoming slow, when scrolling the entry list of the library.

ThiloteE commented 1 year ago

If I may ask, what is your current hardware? What is the name of the processor (CPU), motherboard, RAM and what kind of harddrive do you use (SSD vs HDD; pcie, m.2 or sata)? If you do not know, you can simply search for "Systeminformation" under Windows > Start. I suspect your CPU might hit its limit.

If you have a huge library, for now, there are only a few things you can do:

  1. Try to find performance bottlenecks that are reproducible (e.g. enabling, disabling a preference) and reporting it here
  2. Disable or do not use all known features that are known to have an impact on performance. Some are listed here: https://docs.jabref.org/faq#q-i-have-a-huge-library.-what-can-i-do-to-mitigate-performance-issues
  3. Get a better and faster computer.
ThiloteE commented 1 year ago

By the way, since I know you use regex for your groups, it could also be that those slow down JabRef, as RegEx is known to be very slow.

ghost commented 1 year ago

By the way, since I know you use regex for your groups, it could also be that those slow down JabRef, as RegEx is known to be very slow.

In this libraries no group is made when I loaded it. The RAM goes up when loading library but then it stays the same.

Core i7 4.2 ghz, 32 Gb RAM, pcie samsung M2 2 TB, geforce RTX 3070

It's a gaming pc and It's never slow anywhere. I think it must be jabref

Siedlerchr commented 1 year ago

A high ram usage is per se nothing bad. JabRef loads all entries in memory. As long as the memory stays more or elss constant it's fine.

koppor commented 1 year ago

This especially refs https://github.com/JabRef/jabref/issues/8906 and https://github.com/JabRef/jabref/issues/7567

calixtus commented 1 year ago

The main table uses a standard javafx table, which prepares every cell in memory. Virtualizing the cells could help. See https://github.com/palexdev/VirtualizedFX