ressources saturation when loading networks based on Gephi

[samthiriot]

Hello,

I was using the controlling API to load inside Netlogo networks and run simulations on them. I tried the nw:load-gml primitive and observed a huge general slowdown in the JVM after several runs.

After investigating, new event threads are created everytime a network is readen. They are responsible of this slowdown. It is not perceptible in a standard use, but becomes problematic once you load networks hundreds of times throught the controlling API, as JVMs are known to be imperfect when it comes to run hundreds of threads.

To reproduce the problem, you can either:

• start Netlogo in graphical mode from Eclipse or other, create a model which loads a network using for instance nw:load-gml , and run it several times; you should observe the number of threads which is constantly increasing.
• or create a model which loads a network, control it with the controlling API; call several times the command "nw:load-gml" and monitor Thread.activeCount() inbetween. You'll see the number of active thread increasing.
• in both cases, replace the loading in gml with a loading in graphml (which relies on JUNG for its implementation): no leak of resource.

After having a look to the sourcecode, I see most network formats are provided by Gephi. Unfortunatly, and despite of its numerous qualities, Gephi is a bit greedy when it comes to threads. Everytime you create or convert a network, some threads are created to transmit events. We had a discussion some time ago on the Gephi forums which disappeared since. At that time they had not found any workaround enabling to stop the threads after using the network.

[qiemem]

Thanks for bringing this up. I've been trying to reproduce these problems by running nw:load-gml (and other load functions) hundreds of times. I can see that new threads are sometimes created during this process, but it looks like they're always correctly shutdown and GCed afterwards. After running approximately 1000 loads (on small networks) my live thread count is sitting at 20 (monitoring with VisualVM, so a few are RMI threads).

This leads me to believe that Gephi and NW on their own do not leak threads.

Would you post the relevant parts of your code where you're using the controlling API?

[SethTisue]

sounds like maybe you are creating instances of HeadlessWorkspace and then not calling dispose on them when you're done with them?

[samthiriot]

Hello !

Thanks for investigating. I tried to create a minimalistic code for reproducing the error, and end with the same conclusions as you do when comparing graphml and gml - even without disposition the workspace, for instance to reuse it and avoid the overhead of compilation everytime.

So I assume it as to be on my side... sorry for the wrong report then... :-/

[qiemem]

No worries! Thanks for reporting anyway!