search

Freika / dawarich

Self-hosted alternative to Google Location History (Google Maps Timeline)
https://dawarich.app
GNU Affero General Public License v3.0
2.13k stars 48 forks source link

Huge RAM usage #197

Closed Tazmanian79 closed 2 months ago

Tazmanian79 commented 2 months ago

Describe the bug Since the last updates (0.12.x) I have a huge RAM usage by dawarich_sidekiq

Version 0.12.1

Additional context I know it has to do with the new features VISITS and I see in the logs it's because of the reverse geocoding. But only Sidekiq has a RAM usage of 3Gb and it's taking my NAS down. I used to have BACKGROUND_PROCESSING_CONCURRENCY set to 100 before (for the import of Google data) and I changed it now back to the standard 10 but the RAM usage is still + 3Gb.

How can I limit the RAM usage?

I tried to set it in the container to 1Gb limit in Docker but than after +- 1h the container fails and shuts down.

Korenchkin commented 2 months ago

i believe it is new features that trigger 'reindex' or what it is doing,and this is eating ram,running it in ready state and adding few points per hour keeps it low,may i suggest running it on a system with enough ram (for me even 8GB is not enough,also in the similar boat of not enough ram),and once it completes,transferring database to your nas? probably each(bigger) update...and in the future more ram for nas...

Freika commented 2 months ago

The visits suggestion process takes a lot of time and memory, but resources consumption can be limited in docker-compose.yml https://stackoverflow.com/questions/42345235/how-to-specify-memory-cpu-limit-in-docker-compose-version-3

Korenchkin commented 2 months ago

okay,i was doing clean install,importing my 1.1GB google export since ~2016,it quickly went to ~6GB ram(maybe more,i went to sleep,now i have proxmox history,which may not show peak values,not sure...lxc container),but after processing import now it does reverse geocoding and sits around 1.1GB ram (with lots of disk io 14M-not sure what units,just diskwrite)

Freika commented 2 months ago

Release 0.12.3 introduces resource limiting into default docker-compose.file.

Tazmanian79 commented 1 month ago

I'm not able to use these

When I try to create my project I get the error that NanoCPU's are not supported and the container is not created. When I remove the above from my .yaml file there is no problem.

Korenchkin commented 1 month ago

i believe this is too old docker,like really old,some nas probably?

Tazmanian79 commented 1 month ago

Yes possible, Synology is really running behind with their docker version ....