andre15silva
commented
3 months ago Hi @Coppelian,
Thanks for your question!
Currently, you can use the verbose mode of the run command to obtain the output of the GitHub Actions runs.
This is done by enabling the option -v or --verbose in your gitbug-java run command.
However, you will get the stdout and stderr of the GitHub Action runs together with some other logging information that comes from GitBug-Java itself.
To fix this I have opened a PR (https://github.com/gitbugactions/gitbug-java/pull/31) that includes stdout and stderr of the GitHub Action runs in the json file generated by the gitbug-java run command.
If you checkout this branch, you will be able to find the results in the file at {checkout_dir}/.gitbug-java/test-results.json.
Lastly, please do note that while Defects4J provides only the stack-trace, our output includes information from the entire GitHub Action run, including the stack-trace from the test run but also the output from other steps in the executed workflows. Currently, we do not have a way of extracting only the test execution stack-trace from this, and it might be a hard task since each project/bug has very different workflow setups.
Coppelian
Hi,
Thanks for providing this great benchmark dataset.
I have a question regarding the failing report. Is it possible to generate a failing_test file containing all test error logs? Defects4j offered a detailed log in this format:
--- org.jfree.chart.renderer.category.junit.AbstractCategoryItemRendererTests::test2947660 junit.framework.AssertionFailedError: expected:<1> but was:<0> at junit.framework.Assert.fail(Assert.java:57) at junit.framework.Assert.failNotEquals(Assert.java:329) at junit.framework.Assert.assertEquals(Assert.java:78) at junit.framework.Assert.assertEquals(Assert.java:234) at junit.framework.Assert.assertEquals(Assert.java:241) at junit.framework.TestCase.assertEquals(TestCase.java:409) at org.jfree.chart.renderer.category.junit.AbstractCategoryItemRendererTests.test2947660(AbstractCategoryItemRendererTests.java:409) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at junit.framework.TestCase.runTest(TestCase.java:176) at junit.framework.TestCase.runBare(TestCase.java:141) at junit.framework.TestResult$1.protect(TestResult.java:122) at junit.framework.TestResult.runProtected(TestResult.java:142) at junit.framework.TestResult.run(TestResult.java:125) at junit.framework.TestCase.run(TestCase.java:129) at junit.framework.TestSuite.runTest(TestSuite.java:255) at junit.framework.TestSuite.run(TestSuite.java:250) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:520) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTask.executeInVM(JUnitTask.java:1484) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTask.execute(JUnitTask.java:872) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTask.executeOrQueue(JUnitTask.java:1972) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTask.execute1(JUnitTask.java:824) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTask.execute(JUnitTask.java:2277) at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291) at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106) at org.apache.tools.ant.Task.perform(Task.java:348) at org.apache.tools.ant.Target.execute(Target.java:392) at org.apache.tools.ant.Target.performTasks(Target.java:413) at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399) at org.apache.tools.ant.Project.executeTarget(Project.java:1368) at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41) at org.apache.tools.ant.Project.executeTargets(Project.java:1251) at org.apache.tools.ant.Main.runBuild(Main.java:811) at org.apache.tools.ant.Main.startAnt(Main.java:217) at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280) at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109)
Is it possible to retrieve this information after running gitbug_java run command?
Thank you for your help!