IrosTheBeggar / mStream

The easiest music streaming server available
http://mstream.io
GNU General Public License v3.0
2.22k stars 188 forks source link

Improve DB Memory Usage When Scanning Files #317

Closed IrosTheBeggar closed 3 years ago

IrosTheBeggar commented 3 years ago

The Problem:

When doing a file scan and large libraries, RAM usage can spike causing mStream to crash on devices with low memory.

The Reason:

mStream uses an in-memory DB called LokiJS. The bigger the DB gets, the RAM it needs. This usually isn't a a problem (metadata doesn't take up much space). But hwne a scan happens the DB is copied to a separate thread that handles the scan logic. When the scan finishes, the DB is reloaded in the main thread, causing memory usage to spike even higher.

The Solution:

The main thread should have a special API for the scan thread to communicate with. The scan thread can commit CRUD operations on the DB by using this new API. This way the scan thread doesn't have to duplicate the DB and the main thread doesn't have to reload the DB.