search

dmunch / couch-chakra

A CouchDB Query Server Runtime build with ChakraCore
Apache License 2.0
17 stars 2 forks source link

Huge virtual memory usage #7

Open excieve opened 7 years ago

excieve commented 7 years ago

I've been tracing resource usage with my benchmarks and noticed that couch-chakra processes are using about 35G of virtual memory (with only ≈55M RSS):

#      Time   UID       PID    %usr %system  %guest    %CPU   CPU  minflt/s  majflt/s     VSZ     RSS   %MEM   kB_rd/s   kB_wr/s kB_ccwr/s iodelay threads   fd-nr  Command
 1489004421  1000     13690  100,00   65,02    0,00   65,30     6 193037,44      0,00 2980360  799660   4,89      0,00   1442,36      0,00       0      29      77  /opt/couchdb/bin/../erts-6.2/bin/beam.smp -K true -A 16 -Bd -- -root /opt/couchdb/bin/.. -progname couchdb -- -home /opt/couchd
 1489004421  1000     14130    0,49    0,00    0,00    0,06     6      0,00      0,00  228460   25420   0,16      0,00      0,00      0,00       0       1       4  python3 utils/addview.py data/reference/income_map_function.es6 -u admin -p admin -D map_tests_chakra -v read_income_test -l ch
 1489004421  1000     14134   24,63    0,99    0,00    3,23     1    120,20      0,00 35836712   56364   0,34      0,00      0,00      0,00       0       7       9  /opt/couch-chakra/couch-chakra /opt/couch-chakra/main.js 
 1489004421  1000     14143   13,79    0,00    0,00    1,74     2     15,76      0,00 35834916   54580   0,33      0,00      0,00      0,00       0       7       9  /opt/couch-chakra/couch-chakra /opt/couch-chakra/main.js 
 1489004421  1000     14144   22,66    0,49    0,00    2,92     6     67,00      0,00 35835356   56140   0,34      0,00      0,00      0,00       0       7       9  /opt/couch-chakra/couch-chakra /opt/couch-chakra/main.js 
 1489004421  1000     14145   22,17    0,49    0,00    2,86     4     83,25      0,00 35834736   55340   0,34      0,00      0,00      0,00       0       7       9  /opt/couch-chakra/couch-chakra /opt/couch-chakra/main.js 
 1489004421  1000     14146   20,20    0,49    0,00    2,61     1     86,70      0,00 35835912   55884   0,34      0,00      0,00      0,00       0       7       9  /opt/couch-chakra/couch-chakra /opt/couch-chakra/main.js 
 1489004421  1000     14147   26,11    0,49    0,00    3,35     2    127,59      0,00 35835208   55868   0,34      0,00      0,00      0,00       0       7       9  /opt/couch-chakra/couch-chakra /opt/couch-chakra/main.js 
 1489004421  1000     14148   20,69    0,49    0,00    2,67     7    183,74      0,00 35769912   55496   0,34      0,00      0,00      0,00       0       7       9  /opt/couch-chakra/couch-chakra /opt/couch-chakra/main.js 
 1489004421  1000     14149   23,15    0,49    0,00    2,98     5     22,17      0,00 35761756   49764   0,30      0,00      0,00      0,00       0       7       9  /opt/couch-chakra/couch-chakra /opt/couch-chakra/main.js

I'm pretty sure it shouldn't be like this. Other query servers (and even CouchDB itself) don't use anywhere near this much. This is maintained even when no data is being processed.

dmunch commented 7 years ago

What a hungry little beast, that's unexpected! I'm not entirely sure if that comes from the way I use ChakraCore or if it's the runtime itself which just behaves this way.

One thing you could try is to use the ´-S´ switch as a parameter to chakra-core on the command line, which will call JsSetRuntimeMemoryLimit.

excieve commented 7 years ago

-S 200000000 didn't help with this, still ≈35GB of virtual mem. I think it only affects real memory.