search

matiasdelellis / facerecognition

Nextcloud app that implement a basic facial recognition system.
GNU Affero General Public License v3.0
510 stars 46 forks source link

Add a (hidden) option to avoid hard memory checks #653

Open llucax opened 1 year ago

llucax commented 1 year ago

Coming from https://github.com/matiasdelellis/facerecognition/issues/543#issuecomment-1474511972

After hacking the sources to avoid the hard memory limits (there are several places where a 1G minimum memory is checked, or even some limit using 2/3 of the total memory) I was able to succesfuly run this app (the background_job) with 725M set as memory (setup -M 725M) on a system with 1GB total RAM (and 4GB of swap, but only about 750M is being used).

The processing does trigger swap activity, but it's not that bad and it seems to be working. Measuring just quickly, it seems to be taking 40s per photo aprox (the web interface says: "Analyzing images - 5350 images detected - 5316 images in queue - Ends approximately in 2 days", so about 32s per photo).

I'm running this on a very cheap shared VPS instance, only one vCPU, 1GB RAM total, with good old spinning HDs (but in RAID60), so swapping is not particularly fast.

/proc/cpuinfo

``` processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 62 model name : Intel(R) Xeon(R) CPU E5-2430L v2 @ 2.40GHz stepping : 4 microcode : 0x1 cpu MHz : 2394.230 cache size : 16384 KB physical id : 0 siblings : 1 core id : 0 cpu cores : 1 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm pti ssbd ibrs ibpb stibp fsgsbase tsc_adjust smep erms xsaveopt arat md_clear bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit mmio_unknown bogomips : 4788.46 clflush size : 64 cache_alignment : 64 address sizes : 46 bits physical, 48 bits virtual power management: ```

Having all this in mind, I think it would be good if there were not so many blockers to run this on less than 1GB, IMHO this should be something that users should be able to try out and decide for themselves if they want to give it a try and if it works for them. Maybe adding some hidden option like avoid_mem_checks that will just let the user configure whatever memory they want and ignore minimum memory checks?

Tirpitz93 commented 1 year ago

I think this is a sensible suggestion, doesnt affect me right now but I think this may be useful to many people