search

testng-team / testng

TestNG testing framework
https://testng.org
Apache License 2.0
1.99k stars 1.02k forks source link

group-by-instances="true" causes memory leak [org.testng.internal.DynamicGraph$Edge] #2099

Open kaszarek opened 5 years ago

kaszarek commented 5 years ago

TestNG Version

6.14.3 6.13.1

Expected behavior

Memory should stay stable and not causing test execution to halt

Actual behavior

Test execution with group-by-instance='true" causes tests to get stuck due to the allocated memory.

When this flag is removed, everything works fine.

Overview Memory overview

Is the issue reproductible on runner?

Test case sample

this is a sample of testng xml

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="Full regression suite" verbose="1" time-out="3600000">
  <test name="All tests " group-by-instances="true" preserve-order="false">
    <groups>
        <run>
            <exclude name="group1" />
            <exclude name="group2" />
            <exclude name="group3" />
        </run>
    </groups>
    <packages>
        <package name="basepackage.*" />
    </packages>
  </test>
</suite>

Maybe related to https://github.com/cbeust/testng/issues/1907

The issue does not occur on version 6.11

krmahadevan commented 5 years ago

@kaszarek - Can you please retry again using the latest released version of TestNG 7.0.0-beta5 and post back your results ? If this is still a problem, please help create a sample project which we can use to reproduce the problem.

kaszarek commented 5 years ago

@krmahadevan thanks for a prompt response. I am trying tests with version 6.11. I will also try with the suggested one. I thought about creating a sample project, however, right now I don't have time for it. I wanted to alert with the bug so I don't forget about this (as I already did for a couple of weeks)

kaszarek commented 5 years ago

Version 6.11 did not manifest the issue with memory.

krmahadevan commented 5 years ago

@kaszarek - Irrespective of whether the issue exists in the older versions or not, we can fix it only in the upcoming versions. That is why I was hoping that you help run this with the latest beta build and post back your findings. I understand and hear you on the time constraints. But the sample would do us a lot of good in terms of fixing this as soon as possible. Also if possible can you please add a high resolution screenshots (the ones that you added aren't that clear). Also please add any logs that you can extract out of the tool that you used which can give us insights.

logs = textual content of the graphical view depicted by your screenshot

kaszarek commented 5 years ago

Checking version 7.0.0.-x is not trivial as it seems not to be backward compatible (as major version changed). I know already that version 6.13.1 had this memory problem with DynamicGraph type. @krmahadevan I will create a sample project but I need to free up some time for this. I fully understand that this would help you reproduce the issue. However, the only thing that I can provide now is where the breaking change occurred - this should give you guys an indicator of what was the change that caused this problem.

juherr commented 5 years ago

@kaszarek We know what is the change. But things changed in 7.x and maybe it is already fixed.

You right that 7.x may not be backward compatible but it should work in the most cases.

kaszarek commented 5 years ago

@juherr @krmahadevan I will try the case with version 7 when I have time. I created a sample project for checking the leak but seems that it is too simple.