Ideal memory requirement for NameRes on Translator - Githubissues

TranslatorSRI / NameResolution

A service for finding CURIEs from lexical strings.

3 stars 2 forks source link

Ideal memory requirement for NameRes on Translator #152

Open gaurav opened 4 days ago

gaurav commented 4 days ago

Some numbers on this issue:

We have NameRes Solr set up with both a memory request and memory limit of 58Gi on both RENCI (Dev) and ITRB (CI, Test, Prod)
Solr itself reports approx 2.30GB/58.00GB of JVM Memory used and 13.83GB/58.00GB of physical memory in normal use, but peaking at 26.61GB when running the babel-validation test suite.
Kubernetes shows memory usage of 7.11Gi (WSS), 7.00 GiB (RSS) and 15.88 GiB (Cache), even when running the babel-validation test suite.

We are getting CPU-limited, so I'm going to try to increase our CPU limit first, and then see if we need to increase our memory limit.

gaurav commented 4 days ago

This time we peaked at 2.57 CPUs, which means we had very slight (2%) of CPU throttling. Memory usage peaked at 6.83 GiB (WSS). However, we still had four requests time out. I should try to increase CPUs to e.g. 5 and see if that fixes it; otherwise, there might be some kind of Solr config we need to tweak.