How can I see the producer hierarchy ?

[dbeniamine]

Hi,

When I try to visualize my (Moca) traces, Ocelotl aggregate all the producers while there is a hierarchy designed to avoid this aggregation.

I guess there is some issue in the way the importer computes the hierarchy, but I don't see anything weird when I import the traces in debug mode, It would be very helpful if I could see the producer hierarchy... I tried to visualize the trace as Pie chart and open the event producer filter settings but it seems to freeze Framesoc.

Any Ideas ?

Thanks, David

[ycorre]

Unfortunately, there is no dedicated tool to view the producer hierarchy.

However I don't know if you are aware that the aggregation level can be set in the option: the maximum number of leaves in the hierarchy tree can be specified, if the current number of leaves is above this number then Ocelotl look for the lowest level in the hierarchy tree that can satisfy this condition and aggregated all the producers in the levels below. So that may be why Ocelotl aggregates all producers.

Another thing is that you can right-click on an event producer representation in the spatio-temporal visualisations. That will open a new window that displays the event producer hierarchy tree with the clicked producer as root of the tree (but if the producers were aggregated by Ocelotl, it will show you the aggregated version of the hierarchy tree).

[dbeniamine]

Thanks for the right-click trick it allowed me to see the full hierarchy.

I think that there is an issue with the aggregation level parameter that you are talking about, I've set it to 50, when I look at my hierarchy at the first level there are 28 producers, so this level should be fine, yet Ocelotl aggregate all producers and I only have one big block with visual aggregation ...

Moreover once i've right-clicked on the root producer to see the hierarchy, the main framesoc window freeze and I still have several process using all my CPUS ...

[ycorre]

Which trace are you using ?

I tried with the latest trace you provided (55116 producers for the physical, 56408 for the virtual) with the aggregation level set to 50 and the aggregation was working correctly.

When you right-click on the hierarchy, do you see only one producer (meaning that all the producers has been aggregated) or do you see the hierarchy correctly aggregated (i.e. one or several levels of the hierarchy with a total number of leaves that does not exceed 50) ?

Or maybe are you talking about the visual aggregation (blocks with a diagonally-striped texture) ? In this case that can also be disabled in the options : Aggregation tab -> Visual Aggregation (at the bottom) -> uncheck Enable Visual Aggregation.

[dbeniamine]

So the hierarchy is correct when I right click, and I can indeed see the trace if I disable visual aggregation. Still while I can see the trace, I can't interact with it: any selection or click take a very long time to complete and after one or two "clicks" Framesoc. The freeze look very similar to the one reported here: https://github.com/soctrace-inria/ocelotl/issues/188.

Thanks again for your help :)

[ycorre]

I have tried with same settings as previously (trace for the new moca, aggregation level at 50) and I did not encounter the problem while performing selections on the view (although they are very slow (1 or 2 second delay between the selection and its actual display on the screen)).

However each time I tried to modify the parameter p value, there was indeed a freeze of the whole interface (1 or 2 minutes on my computer) before the monitor actually shows up and the computation continues. Do you have the same behavior?

I will investigate further but to me it seems like one of the operations that are happening when changing the parameter p value is not threaded and takes a lot of time to compute, thus freezing the whole application.

[dbeniamine]

I have the same problem, but the selection is slower and once the interface freezes it stays freezed if I let it run long enough it goes completly grey. Furthermore if I kill framesoc when freezed, some threads doing heavy computations are still alive ....

[ycorre]

OK, I think I know why there are freezes: they should only occurs when a subset of the event producers is selected (or displayed) and a new run is launched (either throught the run button or by changing the value of the parameter p). Is it the case for you? Or are there freezes in other cases?

So the freeze occur because of the method setConfiguration() which is called before the computation thread is started. In this method a call to updateCurrentProducers() is made, which must go through the whole collection of event producers, which is usually not a big deal, but becomes problematic if there are a lot of producers. I'll look for a fix to see if I can at least get it to execute in a thread.

For the threads being still alive after killing eclipse, I think this is because the threads are attached to the Java VM which can keep running even after eclipse is closed. I don't know if this because we don't clean up properly in framesoc, if this is just an unavoidable side effect when killing it.

[dbeniamine]

Yup the freezes occurs when either I click "run" or I change the p value.

The selection works indeed (except that I can't re run the visualization once selected).

[ycorre]

OK, I have pushed a fix that should (hopefully) stop the freezes.

[dbeniamine]

I'm trying the patched version, now I see a popup window showing the computation progress when I change the p value, but it seems to freeze at the beginning: it's stuck on "Computing Aggregated View" since almost half an hour and the progress bar is not moving at all.

[dbeniamine]

I let framesoc run, after a while a window "Problem Occurred" appeared. Sadly I can't read the details as the window is completly grey.

I have the following error on eclipse console:

Adding Shared_Read variable
Exception in thread "Thread-15" Exception in thread "[Timer] - Main Queue Handler" Exception in thread "Thread-16" Adding Shared_Write variable
java.lang.NullPointerException
    at fr.inria.soctrace.tools.ocelotl.core.microdesc.Microscopic3DDescription.matrixWrite(Microscopic3DDescription.java:226)
    at fr.inria.soctrace.tools.ocelotl.microdesc.operators.VariableDistribution$OcelotlThread.matrixUpdate(VariableDistribution.java:81)
    at fr.inria.soctrace.tools.ocelotl.microdesc.operators.VariableDistribution$OcelotlThread.run(VariableDistribution.java:106)
java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOfRange(Arrays.java:2694)
    at java.lang.String.<init>(String.java:203)
    at java.lang.StringBuffer.toString(StringBuffer.java:561)
    at org.eclipse.equinox.internal.util.impl.tpt.threadpool.ThreadPoolFactoryImpl.execute0(ThreadPoolFactoryImpl.java:94)
    at org.eclipse.equinox.internal.util.impl.tpt.timer.TimerImpl.run(TimerImpl.java:110)
    at java.lang.Thread.run(Thread.java:745)
java.lang.OutOfMemoryError: Java heap space
    at java.util.HashMap.inflateTable(HashMap.java:316)
    at java.util.HashMap.put(HashMap.java:488)
    at fr.inria.soctrace.tools.ocelotl.core.microdesc.Microscopic3DDescription.matrixPushType(Microscopic3DDescription.java:231)
    at fr.inria.soctrace.tools.ocelotl.microdesc.operators.VariableDistribution$OcelotlThread.matrixUpdate(VariableDistribution.java:78)
    at fr.inria.soctrace.tools.ocelotl.microdesc.operators.VariableDistribution$OcelotlThread.run(VariableDistribution.java:106)
Exception in thread "Thread-17" java.lang.OutOfMemoryError: Java heap space
    at java.util.HashMap.createEntry(HashMap.java:897)
    at java.util.HashMap.addEntry(HashMap.java:884)
    at java.util.HashMap.put(HashMap.java:505)
    at fr.inria.soctrace.tools.ocelotl.core.microdesc.Microscopic3DDescription.matrixPushType(Microscopic3DDescription.java:231)
    at fr.inria.soctrace.tools.ocelotl.microdesc.operators.VariableDistribution$OcelotlThread.matrixUpdate(VariableDistribution.java:78)
    at fr.inria.soctrace.tools.ocelotl.microdesc.operators.VariableDistribution$OcelotlThread.run(VariableDistribution.java:106)
Exception in thread "Thread-18" java.lang.NullPointerException
    at fr.inria.soctrace.tools.ocelotl.core.microdesc.Microscopic3DDescription.matrixWrite(Microscopic3DDescription.java:226)
    at fr.inria.soctrace.tools.ocelotl.microdesc.operators.VariableDistribution$OcelotlThread.matrixUpdate(VariableDistribution.java:81)
    at fr.inria.soctrace.tools.ocelotl.microdesc.operators.VariableDistribution$OcelotlThread.run(VariableDistribution.java:106)
!SESSION 2016-02-15 15:12:44.165 -----------------------------------------------

[ycorre]

java.lang.OutOfMemoryError: Java heap space -> this means that the JVM has run out of memory.

How much memory are you allocating to the JVM? If you are using Framesoc from the developer version, you can change this parameter by going in Run -> Run Configuration -> Select "Eclipse Application" -> Go the Arguments tab and in the VM arguments field, change the value after -Xmx by a higher value.

Running low on memory would also explain why it becomes slow after a while.

[dbeniamine]

I was already using 7Gio of memory: -Xmx7168m I'll try with 16Gio (I have 8Gio of RAM and as much of SWAP).

[ycorre]

OK, that's already a lot (I am not sure including swap is going to change anything though).

What trace are you using (how many event producers?) and which aggregation level parameter are you using? When I tested it with the trace that has aroud 55000 producers and an aggregation level of 50, I was around 1.5 Gb of memory, but maybe there are memory leaks, I'll look into it.

[dbeniamine]

There is the trace that I'm working on. I imported it with the last version of the Moca importer with the following parameters: Max hierarchy depth: 16 TrimLoneProducer: Yes Keep Only accesses in data structures: yes

I try to visualize the virtual trace which contains 362093 events on 56430 producers.

[ycorre]

I have found the problem: the aggregation was not performed at all when zooming in on a subset of producers, which lead to a huge memory consumption in the later stages of the computation.

I have pushed a patched version that should fix things in most cases, but I might publish a more extensive solution later.

[dbeniamine]

Thanks ! With this new commit I manage to do some interactions on the trace, it's still a bit slow but it works :)

[ycorre]

I have made another modification that should speed up the aggregation process.

[dbeniamine]

Thank you very much, it seems indeed faster!

I close the issue, thanks again for your help :)