search

dice-group / AGDISTIS

AGDISTIS - Agnostic Named Entity Disambiguation
http://aksw.org/Projects/AGDISTIS.html
GNU Affero General Public License v3.0
141 stars 37 forks source link

Memory Issues #70

Closed webersab closed 5 years ago

webersab commented 5 years ago

I am running AGDISTIS as a server on a machine with 72 CPUs and 315G memory. I call it by running "mvn tomcat:run". My program then sends the AGDISTIS server a sentence with annotated named entities and receives the response. When I was running this with smaller amounts of data (like a few 100 sentences) this worked fine. Now that I want to run the AGDISTIS server over a longer period, the server slows down and then crashes with the error message that it was unable to assign memory. I tried increasing and decreasing the allocated memory in the pom.xml file, but the problem stayed the same. I use the DBPedia index. My code sends about 525 sentences in 10 seconds, 3150 sentences per minute. Every sentence is about 200 characters long and contains at least two named entities. Attached is an exemplary error log from one of the runs. hs_err_pid59960.log.zip

DiegoMoussallem commented 5 years ago

Hey, @webersab we realized that indices open every time while requests come in and are never closed. Please, pull the fixed code and try again.