search

LibrePhotos / librephotos

A self-hosted open source photo management service. This is the repository of the backend.
MIT License
7.01k stars 309 forks source link

Reduce memory consumption #93

Closed ghost closed 3 years ago

ghost commented 3 years ago

memory consumption of librephotos is way too high now, even if the both client and server are doing nothing. Is it possible to load the torch models only when the they are being used by facial recognition or object identification?

derneuere commented 3 years ago

Related to: #40

Possible torch models that could be loaded differently: https://github.com/LibrePhotos/librephotos/blob/ab20e7ea24768bc2fca14036804b0e0c4acd8e50/api/places365/places365.py#L19

ghost commented 3 years ago

@derneuere Can it also be unloaded when the scanning is done? Then the memory can be released for other tasks on the same server.

Rihcus commented 3 years ago

Related #108 (sorry for poor formatting)

The memory usage seems to be correlated with the size of the photo library (tested both on mac and windows with docker desktop). A 400gb library tends to lead to issues where admin pannel fails to see large directory or even user and if a machine has 4gb load times and frequent logouts are present. On the flip side a 14 photo library has minimal memory and cpu usage with none of the above issues regarding admin pannel or logouts present.