search

eclipse-jdt / eclipse.jdt.core

Eclipse Public License 2.0
164 stars 130 forks source link

[test] Random failing CompletionTests.testType1, testType10 #2082

Open mpalat opened 8 months ago

mpalat commented 8 months ago

From https://download.eclipse.org/eclipse/downloads/drops4/I20240229-0520/testresults/html/org.eclipse.jdt.core.tests.model_ep431I-unit-win32-java17_win32.win32.x86_64_17.html

the following tests are failing:

testExtraLibraries07 ----------- Expected ------------ /P[CPE_SOURCE][K_SOURCE][isExported:false]\n C:\Users\genie.releng\workspace\AutomatedTests\ep431I-unit-win32-java17\workarea\I20240229-0520\eclipse-testing\test-eclipse\eclipse\jdt_model_folder\jclMin.jar[CPE_LIBRARY][K_BINARY][isExported:false]\n C:\Users\genie.releng\workspace\AutomatedTests\ep431I-unit-win32-java17\workarea\I20240229-0520\eclipse-testing\test-eclipse\eclipse\jdt_model_folder\lib2.jar[CPE_LIBRARY][K_BINARY][isExported:true]\n C:\Users\genie.releng\workspace\AutomatedTests\ep431I-unit-win32-java17\workarea\I20240229-0520\eclipse-testing\test-eclipse\eclipse\jdt_model_folder\lib1.jar[CPE_LIBRARY][K_BINARY][isExported:true] ------------ but was ------------ /P[CPE_SOURCE][K_SOURCE][isExported:false]\n C:\Users\genie.releng\workspace\AutomatedTests\ep431I-unit-win32-java17\workarea\I20240229-0520\eclipse-testing\test-eclipse\eclipse\jdt_model_folder\jclMin.jar[CPE_LIBRARY][K_BINARY][isExported:false]\n C:/Users/genie.releng/workspace/AutomatedTests/ep431I-unit-win32-java17/workarea/I20240229-0520/eclipse-testing/test-eclipse/eclipse/jdt_model_folder/lib2.jar[CPE_LIBRARY][K_BINARY][isExported:true]\n C:\Users\genie.releng\workspace\AutomatedTests\ep431I-unit-win32-java17\workarea\I20240229-0520\eclipse-testing\test-eclipse\eclipse\jdt_model_folder\lib1.jar[CPE_LIBRARY][K_BINARY][isExported:true] ---------------------- ---------- expected:<...Exported:false]\n C:[\Users\genie.releng\workspace\AutomatedTests\ep431I-unit-win32-java17\workarea\I20240229-0520\eclipse-testing\test-eclipse\eclipse\jdt_model_folder]lib2.jar[CPE_LIBRARY...> but was:<...Exported:false]\n C:[/Users/genie.releng/workspace/AutomatedTests/ep431I-unit-win32-java17/workarea/I20240229-0520/eclipse-testing/test-eclipse/eclipse/jdt_model_folder/]lib2.jar[CPE_LIBRARY...>

junit.framework.ComparisonFailure: ----------- Expected ------------ /P[CPE_SOURCE][K_SOURCE][isExported:false]\n C:\Users\genie.releng\workspace\AutomatedTests\ep431I-unit-win32-java17\workarea\I20240229-0520\eclipse-testing\test-eclipse\eclipse\jdt_model_folder\jclMin.jar[CPE_LIBRARY][K_BINARY][isExported:false]\n C:\Users\genie.releng\workspace\AutomatedTests\ep431I-unit-win32-java17\workarea\I20240229-0520\eclipse-testing\test-eclipse\eclipse\jdt_model_folder\lib2.jar[CPE_LIBRARY][K_BINARY][isExported:true]\n C:\Users\genie.releng\workspace\AutomatedTests\ep431I-unit-win32-java17\workarea\I20240229-0520\eclipse-testing\test-eclipse\eclipse\jdt_model_folder\lib1.jar[CPE_LIBRARY][K_BINARY][isExported:true] ------------ but was ------------ /P[CPE_SOURCE][K_SOURCE][isExported:false]\n C:\Users\genie.releng\workspace\AutomatedTests\ep431I-unit-win32-java17\workarea\I20240229-0520\eclipse-testing\test-eclipse\eclipse\jdt_model_folder\jclMin.jar[CPE_LIBRARY][K_BINARY][isExported:false]\n C:/Users/genie.releng/workspace/AutomatedTests/ep431I-unit-win32-java17/workarea/I20240229-0520/eclipse-testing/test-eclipse/eclipse/jdt_model_folder/lib2.jar[CPE_LIBRARY][K_BINARY][isExported:true]\n C:\Users\genie.releng\workspace\AutomatedTests\ep431I-unit-win32-java17\workarea\I20240229-0520\eclipse-testing\test-eclipse\eclipse\jdt_model_folder\lib1.jar[CPE_LIBRARY][K_BINARY][isExported:true]

expected:<...Exported:false]\n C:[\Users\genie.releng\workspace\AutomatedTests\ep431I-unit-win32-java17\workarea\I20240229-0520\eclipse-testing\test-eclipse\eclipse\jdt_model_folder]lib2.jar[CPE_LIBRARY...> but was:<...Exported:false]\n C:[/Users/genie.releng/workspace/AutomatedTests/ep431I-unit-win32-java17/workarea/I20240229-0520/eclipse-testing/test-eclipse/eclipse/jdt_model_folder/]lib2.jar[CPE_LIBRARY...> at org.eclipse.jdt.core.tests.junit.extension.TestCase.assertStringEquals(TestCase.java:267) at org.eclipse.jdt.core.tests.junit.extension.TestCase.assertEquals(TestCase.java:242) at org.eclipse.jdt.core.tests.junit.extension.TestCase.assertEquals(Test

testType1

e ----------- Expected ------------ booleanClass[TYPE_REF]{booleanClass, test, Ltest.booleanClass;, null, null, 72} ------------ but was ------------ ---------------------- ---------- expected:<[booleanClass[TYPE_REF]{booleanClass, test, Ltest.booleanClass;, null, null, 72}]> but was:<[]>junit.framework.ComparisonFailure:----------- Expected ------------booleanClass[TYPE_REF]{booleanClass, test, Ltest.booleanClass;, null, null, 72}------------ but was ---------------------------------- ----------expected:<[booleanClass[TYPE_REF]{booleanClass, test, Ltest.booleanClass;, null, null, 72}]> but was:<[]>at org.eclipse.jdt.core.tests.junit.extension.TestCase.assertStringEquals(TestCase.java:267)at org.eclipse.jdt.core.tests.junit.extension.TestCase.assertEquals(TestCase.java:242)at org.eclipse.jdt.core.tests.junit.extension.TestCase.assertEquals(TestCase.java:239)at org.eclipse.jdt.core.tests.model.AbstractJavaModelCompletionTests.assertResults(AbstractJavaModelCompletionTests.java:215)at org.eclipse.jdt.core.tests.model.CompletionTests.testType1(CompletionTests.java:19111

testType10 ----------- Expected ------------ voi[POTENTIAL_METHOD_DECLARATION]{voi, Ltest.Test$InnerTest;, ()V, voi, null, 39}\n void[KEYWORD]{void, null, null, void, null, 49}\n voidClass[TYPE_REF]{voidClass, test, Ltest.voidClass;, null, null, 52} ------------ but was ------------ voi[POTENTIAL_METHOD_DECLARATION]{voi, Ltest.Test$InnerTest;, ()V, voi, null, 39}\n void[KEYWORD]{void, null, null, void, null, 49} ---------------------- ---------- expected:<...ull, void, null, 49}[\n voidClass[TYPE_REF]{voidClass, test, Ltest.voidClass;, null, null, 52}]> but was:<...ull, void, null, 49}[]>

junit.framework.ComparisonFailure: ----------- Expected ------------ voi[POTENTIAL_METHOD_DECLARATION]{voi, Ltest.Test$InnerTest;, ()V, voi, null, 39}\n void[KEYWORD]{void, null, null, void, null, 49}\n voidClass[TYPE_REF]{voidClass, test, Ltest.voidClass;, null, null, 52} ------------ but was ------------ voi[POTENTIAL_METHOD_DECLARATION]{voi, Ltest.Test$InnerTest;, ()V, voi, null, 39}\n void[KEYWORD]{void, null, null, void, null, 49}

expected:<...ull, void, null, 49}[\n voidClass[TYPE_REF]{voidClass, test, Ltest.voidClass;, null, null, 52}]> but was:<...ull, void, null, 49}[]> at org.eclipse.jdt.core.tests.junit.extension.TestCase.assertStringEquals(TestCase.java:267) at org.eclipse.jdt.core.tests.junit.extension.TestCase.assertEquals(TestCase.java:242) at org.eclipse.jdt.core.tests.junit.extension.TestCase.assertEquals(TestCase.java:239) at org.eclipse.jdt.core.tests.model.AbstractJavaModelCompletionTests.assertResults(AbstractJavaModelCompletionTests.java:215) at org.eclipse.jdt.core.tests.model.CompletionTests.testTyp

Investigate/Recheck for continued failures.

jukzi commented 8 months ago

caused by b2f56fbee3311425c351990cd8ca66d0ca73a940

jukzi commented 8 months ago

somehow the path separators differ: image

jukzi commented 8 months ago

@mpalat i could reproduce and fix testExtraLibraries07 constantly failing on windows. However i can not locally reproduce failed CompletionTests testType1, testType10. Those seem to be unrelated random failures.

see https://ci.eclipse.org/releng/job/AutomatedTests/job/ep431I-unit-win32-java17/lastSuccessfulBuild/testReport/ image

mpalat commented 8 months ago

@mpalat i could reproduce and fix testExtraLibraries07 constantly failing on windows. However i can not locally reproduce failed CompletionTests testType1, testType10. Those seem to be unrelated random failures.

Thanks @jukzi for trying out and fixing the two issues. you can check for the other two if they are random and close this issue for after a day or two.

jukzi commented 5 months ago

CompletionTests.testType1 and CompletionTests.testType10 still randomly fail for example:

https://download.eclipse.org/eclipse/downloads/drops4/I20240514-1800/testresults/html/org.eclipse.jdt.core.tests.model_ep432I-unit-win32-java17_win32.win32.x86_64_17.html

----------- Expected ------------
booleanClass[TYPE_REF]{booleanClass, test, Ltest.booleanClass;, null, null, 72}
------------ but was ------------

---------------------- ----------
expected:<[booleanClass[TYPE_REF]{booleanClass, test, Ltest.booleanClass;, null, null, 72}]> but was:<[]>
at org.eclipse.jdt.core.tests.junit.extension.TestCase.assertStringEquals(TestCase.java:267)
at org.eclipse.jdt.core.tests.junit.extension.TestCase.assertEquals(TestCase.java:242)
at org.eclipse.jdt.core.tests.junit.extension.TestCase.assertEquals(TestCase.java:239)
at org.eclipse.jdt.core.tests.model.AbstractJavaModelCompletionTests.assertResults(AbstractJavaModelCompletionTests.java:215)
at org.eclipse.jdt.core.tests.model.CompletionTests.testType1(CompletionTests.java:19111)
jukzi commented 4 months ago

still randomly failing https://download.eclipse.org/eclipse/downloads/drops4/I20240624-1800/testresults/html/org.eclipse.jdt.core.tests.model_ep433I-unit-win32-java17_win32.win32.x86_64_17.html