JavaClasspathTest.testNoJavaInClasspath fails on latest

[cdietrich]

org.eclipse.xtend.ide.tests.buildpath.JavaClasspathTest.testNoJavaInClasspath fails on latest reason unclear

cannot reproduce from dev env yet.

[cdietrich]

2024-07-12T17:28:51.1734694Z Running org.eclipse.xtend.ide.tests.buildpath.JavaClasspathTest
2024-07-12T17:28:56.7282243Z Tests run: 7, Failures: 6, Errors: 0, Skipped: 0, Time elapsed: 5.568 s <<< FAILURE! -- in org.eclipse.xtend.ide.tests.buildpath.JavaClasspathTest
2024-07-12T17:28:56.7284493Z org.eclipse.xtend.ide.tests.buildpath.JavaClasspathTest.testNoJavaInClasspath -- Time elapsed: 0.382 s <<< FAILURE!
2024-07-12T17:28:56.7285903Z java.lang.AssertionError: 
2024-07-12T17:28:56.7286382Z LogEntry [
2024-07-12T17:28:56.7287194Z   message = "Error initializing type java:/Objects/org.eclipse.xtext.xbase.lib.ArrayExtensions"
2024-07-12T17:28:56.7288468Z   source = "org.eclipse.xtext.common.types.access.jdt.JdtTypeMirror"
2024-07-12T17:28:56.7289278Z   timeStamp = 1720805331605
2024-07-12T17:28:56.7289711Z   level = ERROR
2024-07-12T17:28:56.7290112Z ] expected:<true> but was:<false>
2024-07-12T17:28:56.7290655Z    at org.junit.Assert.fail(Assert.java:89)
2024-07-12T17:28:56.7291331Z    at org.junit.Assert.failNotEquals(Assert.java:835)
2024-07-12T17:28:56.7292085Z    at org.junit.Assert.assertEquals(Assert.java:120)
2024-07-12T17:28:56.7293232Z    at org.eclipse.xtend.ide.tests.buildpath.JavaClasspathTest$1.run(JavaClasspathTest.java:79)
2024-07-12T17:28:56.7294742Z    at org.eclipse.xtext.testing.logging.LoggingTester.captureLogging(LoggingTester.java:73)
2024-07-12T17:28:56.7296255Z    at org.eclipse.xtext.testing.logging.LoggingTester.captureLogging(LoggingTester.java:49)
2024-07-12T17:28:56.7297969Z    at org.eclipse.xtend.ide.tests.buildpath.JavaClasspathTest.testNoJavaInClasspath(JavaClasspathTest.java:48)
2024-07-12T17:28:56.7299598Z    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2024-07-12T17:28:56.7301088Z    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
2024-07-12T17:28:56.7302822Z    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

hmmmmm

@szarnekow does this ring any bells. still cannot reproduce locally

[cdietrich]

maybe in maven the maven/tycho compiled version of the array extensions class is used in dev eclipse i still have the m1 variant i guess

[cdietrich]

from https://ci.eclipse.org/xtext/job/xtext/job/main/1300/artifact/org.eclipse.xtend.ide.tests/target/work/data/.metadata/.log

!ENTRY org.apache.log4j 4 0 2024-07-12 15:42:00.170
!MESSAGE org.eclipse.xtext.common.types.access.jdt.JdtTypeMirror  - Error initializing type java:/Objects/org.eclipse.xtext.xbase.lib.DoubleExtensions

!STACK 0
java.lang.IllegalStateException: Could not create binding for 'org.eclipse.xtext.xbase.lib.DoubleExtensions' in context of project 'testProject'.
    at org.eclipse.xtext.common.types.access.jdt.JdtBasedTypeFactory.createType(JdtBasedTypeFactory.java:228)
    at org.eclipse.xtext.common.types.access.jdt.JdtBasedTypeFactory.createType(JdtBasedTypeFactory.java:305)
    at org.eclipse.xtext.common.types.access.jdt.JdtBasedTypeFactory.createType(JdtBasedTypeFactory.java:1)
    at org.eclipse.xtext.common.types.access.jdt.JdtTypeMirror.initialize(JdtTypeMirror.java:71)
    at org.eclipse.xtext.common.types.access.TypeResource.doLoad(TypeResource.java:135)
    at org.eclipse.emf.ecore.resource.impl.ResourceImpl.load(ResourceImpl.java:1563)
    at org.eclipse.xtext.common.types.access.TypeResource.load(TypeResource.java:121)
    at org.eclipse.xtext.common.types.access.jdt.JdtTypeProvider.createResourceAndFindType(JdtTypeProvider.java:290)
    at org.eclipse.xtext.common.types.access.jdt.JdtTypeProvider.findObjectTypeInJavaProject(JdtTypeProvider.java:273)
    at org.eclipse.xtext.common.types.access.jdt.JdtTypeProvider.doFindObjectType(JdtTypeProvider.java:216)
    at org.eclipse.xtext.common.types.access.jdt.JdtTypeProvider.findObjectType(JdtTypeProvider.java:196)
    at org.eclipse.xtext.common.types.access.jdt.JdtTypeProvider.doFindTypeByName(JdtTypeProvider.java:154)
    at org.eclipse.xtext.common.types.access.jdt.JdtTypeProvider.findTypeByName(JdtTypeProvider.java:140)
    at org.eclipse.xtext.common.types.util.TypeReferences.findDeclaredType(TypeReferences.java:256)
    at org.eclipse.xtext.common.types.util.TypeReferences.findDeclaredType(TypeReferences.java:233)
    at org.eclipse.xtext.xbase.scoping.batch.ImplicitlyImportedFeatures.getTypes(ImplicitlyImportedFeatures.java:82)
    at org.eclipse.xtext.xbase.scoping.batch.ImplicitlyImportedFeatures.getExtensionClasses(ImplicitlyImportedFeatures.java:76)
    at org.eclipse.xtext.xbase.scoping.batch.XbaseBatchScopeProvider.newSession(XbaseBatchScopeProvider.java:121)
    at org.eclipse.xtext.xbase.typesystem.internal.DefaultReentrantTypeResolver.resolve(DefaultReentrantTypeResolver.java:164)
    at org.eclipse.xtext.xbase.typesystem.internal.DefaultReentrantTypeResolver.reentrantResolve(DefaultReentrantTypeResolver.java:140)
    at org.eclipse.xtext.xbase.typesystem.internal.CompoundReentrantTypeResolver.reentrantResolve(CompoundReentrantTypeResolver.java:80)
    at org.eclipse.xtext.xbase.typesystem.internal.CachingBatchTypeResolver$LazyResolvedTypes.resolveTypes(CachingBatchTypeResolver.java:81)
    at org.eclipse.xtext.xbase.typesystem.internal.CachingBatchTypeResolver$2.process(CachingBatchTypeResolver.java:58)
    at org.eclipse.xtext.xbase.typesystem.internal.CachingBatchTypeResolver$2.process(CachingBatchTypeResolver.java:1)
    at org.eclipse.xtext.util.concurrent.IUnitOfWork$Void.exec(IUnitOfWork.java:38)
    at org.eclipse.xtext.util.OnChangeEvictingCache.execWithoutCacheClear(OnChangeEvictingCache.java:145)
    at org.eclipse.xtext.xbase.typesystem.internal.CachingBatchTypeResolver.doResolveTypes(CachingBatchTypeResolver.java:54)
    at org.eclipse.xtext.xbase.typesystem.internal.AbstractBatchTypeResolver.resolveTypes(AbstractBatchTypeResolver.java:70)
    at org.eclipse.xtext.xbase.resource.BatchLinkingService.resolveBatched(BatchLinkingService.java:72)
    at org.eclipse.xtext.xbase.resource.BatchLinkableResource.resolveLazyCrossReferences(BatchLinkableResource.java:166)
    at org.eclipse.xtext.xbase.resource.BatchLinkableResource.linkBatched(BatchLinkableResource.java:196)
    at org.eclipse.xtext.ui.editor.syntaxcoloring.HighlightingReconciler.beforeRefresh(HighlightingReconciler.java:379)
    at org.eclipse.xtext.ui.editor.syntaxcoloring.HighlightingReconciler$2$1.exec(HighlightingReconciler.java:356)
    at org.eclipse.xtext.ui.editor.syntaxcoloring.HighlightingReconciler$2$1.exec(HighlightingReconciler.java:1)
    at org.eclipse.xtext.util.concurrent.CancelableUnitOfWork.exec(CancelableUnitOfWork.java:27)
    at org.eclipse.xtext.util.concurrent.WrappingCancelableUnitOfWork.exec(WrappingCancelableUnitOfWork.java:58)
    at org.eclipse.xtext.util.concurrent.CancelableUnitOfWork.exec(CancelableUnitOfWork.java:27)
    at org.eclipse.xtext.resource.OutdatedStateManager.exec(OutdatedStateManager.java:70)
    at org.eclipse.xtext.ui.editor.model.XtextDocument$XtextDocumentLocker.internalReadOnly(XtextDocument.java:525)
    at org.eclipse.xtext.ui.editor.model.XtextDocument$XtextDocumentLocker.readOnly(XtextDocument.java:497)
    at org.eclipse.xtext.ui.editor.model.XtextDocument.readOnly(XtextDocument.java:136)
    at org.eclipse.xtext.util.concurrent.IReadAccess.tryReadOnly(IReadAccess.java:50)
    at org.eclipse.xtext.util.concurrent.IReadAccess.tryReadOnly(IReadAccess.java:71)
    at org.eclipse.xtext.ui.editor.syntaxcoloring.HighlightingReconciler$2.run(HighlightingReconciler.java:352)
    at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)

!ENTRY org.apache.log4j 4 0 2024-07-12 15:42:00.172
!MESSAGE org.eclipse.xtext.common.types.access.impl.AbstractClassMirror  - resource is empty: java:/Objects/org.eclipse.xtext.xbase.lib.DoubleExtensions

!STACK 0
java.lang.IllegalStateException
    at org.eclipse.xtext.common.types.access.impl.AbstractClassMirror.getEObject(AbstractClassMirror.java:95)
    at org.eclipse.xtext.common.types.access.TypeResource.getEObject(TypeResource.java:95)
    at org.eclipse.xtext.common.types.access.jdt.JdtTypeProvider.findTypeBySignature(JdtTypeProvider.java:627)
    at org.eclipse.xtext.common.types.access.jdt.JdtTypeProvider.findLoadedOrDerivedObjectType(JdtTypeProvider.java:256)
    at org.eclipse.xtext.common.types.access.jdt.JdtTypeProvider.doFindObjectType(JdtTypeProvider.java:208)
    at org.eclipse.xtext.common.types.access.jdt.JdtTypeProvider.findObjectType(JdtTypeProvider.java:196)
    at org.eclipse.xtext.common.types.access.jdt.JdtTypeProvider.doFindTypeByName(JdtTypeProvider.java:154)
    at org.eclipse.xtext.common.types.access.jdt.JdtTypeProvider.findTypeByName(JdtTypeProvider.java:140)
    at org.eclipse.xtext.common.types.util.TypeReferences.findDeclaredType(TypeReferences.java:256)
    at org.eclipse.xtext.common.types.util.TypeReferences.findDeclaredType(TypeReferences.java:233)
    at org.eclipse.xtext.xbase.scoping.batch.ImplicitlyImportedFeatures.getTypes(ImplicitlyImportedFeatures.java:82)
    at org.eclipse.xtext.xbase.scoping.batch.ImplicitlyImportedFeatures.getExtensionClasses(ImplicitlyImportedFeatures.java:76)
    at org.eclipse.xtext.xbase.scoping.batch.XbaseBatchScopeProvider.newSession(XbaseBatchScopeProvider.java:121)

[cdietrich]

=> different builds, different types affected

[LorenzoBettini]

And jdt.core is used here, isn't it?

[cdietrich]

i assume tycho will do

[cdietrich]

can be reprodcued from command line

[cdietrich]

java.lang.NullPointerException: Cannot invoke "org.eclipse.jdt.core.dom.ITypeBinding.isParameterizedType()" because "binding" is null
    at org.eclipse.xtext.common.types.access.jdt.TypeURIHelper.getQualifiedName(TypeURIHelper.java:318)
    at org.eclipse.xtext.common.types.access.jdt.TypeURIHelper.getQualifiedName(TypeURIHelper.java:307)
    at org.eclipse.xtext.common.types.access.jdt.JdtBasedTypeFactory.getQualifiedName(JdtBasedTypeFactory.java:449)
    at org.eclipse.xtext.common.types.access.jdt.JdtBasedTypeFactory.enhanceExecutable(JdtBasedTypeFactory.java:1157)
    at org.eclipse.xtext.common.types.access.jdt.JdtBasedTypeFactory.createOperation(JdtBasedTypeFactory.java:1442)
    at org.eclipse.xtext.common.types.access.jdt.JdtBasedTypeFactory.createMethods(JdtBasedTypeFactory.java:921)
    at org.eclipse.xtext.common.types.access.jdt.JdtBasedTypeFactory.createType(JdtBasedTypeFactory.java:396)
    at org.eclipse.xtext.common.types.access.jdt.JdtBasedTypeFactory.createType(JdtBasedTypeFactory.java:341)
    at org.eclipse.xtext.common.types.access.jdt.JdtBasedTypeFactory.createType(JdtBasedTypeFactory.java:236)
    at org.eclipse.xtext.common.types.access.jdt.JdtBasedTypeFactory.createType(JdtBasedTypeFactory.java:305)
    at org.eclipse.xtext.common.types.access.jdt.JdtBasedTypeFactory.createType(JdtBasedTypeFactory.java:1)

[cdietrich]

bad executable e.g.

org.eclipse.xtext.xbase.lib.ArrayExtensions.clone

this is generic

public static T[] clone(T[] array) { return array.clone(); }

calling

if (binding.isArray()) { return getQualifiedName(binding.getComponentType(), new StringBuilder()).append("[]").toString(); }

does not work. get component type returns null

[cdietrich]

@srikanth-sankaran does this ring a bell?

[cdietrich]

[cdietrich]

[cdietrich]

the missing binding is [MISSING:java.lang.Object]

maybe this was not reported before but now is. am not sure that the test is intending to test

[cdietrich]

before the resolver was not even called ....

=> maybe also something in pde/resource/build changed

[cdietrich]

maybe also a reace in the test. am not sure if the delete will wait for the concurrently running build

[cdietrich]

hmmm

the missing type is there too. but we dont return null

[cdietrich]

[cdietrich]

hmmm

https://github.com/eclipse-jdt/eclipse.jdt.core/commit/f6fe3c8fee3d83b2478db25eb34781b62cab668b

cc @stephan-herrmann

[stephan-herrmann]

hmmm

Hi @cdietrich how can I help you? :)

[stephan-herrmann]

the missing type is there too. but we dont return null

ah, maybe I can help: one of the effects of https://github.com/eclipse-jdt/eclipse.jdt.core/commit/f6fe3c8fee3d83b2478db25eb34781b62cab668b is, that ecj now more reliably sets ReferenceBinding.tagBits |= TagBits.HasMissingType when we encounter a reference to a missing type.

While generally this is private business of the compiler, DefaultBindingResolver indeed inspects this flag to figure out, how to translate compiler bindings into DOM bindings. What happens when a missing type is encountered is then controlled by the flag DefaultBindingResolver.isRecoveringBindings.

So: if your test indeed runs without java.lang.Object in scope, and if you request DOM bindings without recovery, then type variables bounded by java.lang.Object will not be translated into DOM bindings, because those bindings would be incomplete.

Any questions?

[stephan-herrmann]

testNoJavaInClasspath

What is the expectation of a test using Java but without Java?

[cdietrich]

Xtend will put a marker to file saying no java The test tests this but also expect no errors to be logged.

we use the binding.getComponent type at many places that will crash now

• when will a binding.getXXXx return null and when not? (e.g. binding.getQualifiedName() on a primitive)
• we will likely have to rewrite a ton of code
• and i dont know how to properly rewrite it. (hope sebastian can give some insights)
• looking at the current code there is null handling nowhere when dealing with the bindings in our TypeUriHelper

[stephan-herrmann]

It seems that binding recovery is not enabled in this situation. In that case null must be expected even before the change in JDT. I did not change any general null contract, only fine tuned which elements of a type binding will taint the entire binding.

So if resilience against missing types is desired enable binding recovery.

[cdietrich]

So we need to figure out what the desired behavior is I am not sure if we use recovery we will miss situations that should be an error

[cdietrich]

in a quick try we also cannot deal with a recovered binding

java.lang.IllegalArgumentException: Unexpected type: org.eclipse.jdt.core.dom.RecoveredTypeBinding@7a2daa29
    at org.eclipse.xtext.common.types.access.jdt.JdtBasedTypeFactory.createAnnotationValue(JdtBasedTypeFactory.java:569)
    at org.eclipse.xtext.common.types.access.jdt.JdtBasedTypeFactory.createAnnotationValue(JdtBasedTypeFactory.java:517)
    at org.eclipse.xtext.common.types.access.jdt.JdtBasedTypeFactory.createAnnotationReference(JdtBasedTypeFactory.java:490)
    at org.eclipse.xtext.common.types.access.jdt.JdtBasedTypeFactory.createAnnotationValues(JdtBasedTypeFactory.java:463)
    at org.eclipse.xtext.common.types.access.jdt.JdtBasedTypeFactory.createOperation(JdtBasedTypeFactory.java:1452)
    at org.eclipse.xtext.common.types.access.jdt.JdtBasedTypeFactory.createMethods(JdtBasedTypeFactory.java:922)
    at org.eclipse.xtext.common.types.access.jdt.JdtBasedTypeFactory.createType(JdtBasedTypeFactory.java:397)
    at org.eclipse.xtext.common.types.access.jdt.JdtBasedTypeFactory.createType(JdtBasedTypeFactory.java:342)
    at org.eclipse.xtext.common.types.access.jdt.JdtBasedTypeFactory.createType(JdtBasedTypeFactory.java:236)
    at org.eclipse.xtext.common.types.access.jdt.JdtBasedTypeFactory.createType(JdtBasedTypeFactory.java:306)
    at org.eclipse.xtext.common.types.access.jdt.JdtBasedTypeFactory.createType(JdtBasedTypeFactory.java:1)
    at org.eclipse.xtext.common.types.access.jdt.JdtTypeMirror.initialize(JdtTypeMirror.java:71)
    at org.eclipse.xtext.common.types.access.TypeResource.doLoad(TypeResource.java:135)
    at org.eclipse.emf.ecore.resource.impl.ResourceImpl.load(ResourceImpl.java:1563)

[cdietrich]

org.eclipse.xtext.common.types.access.jdt.TypeURIHelperTest.testBug300216() indicates we at least want to deal with non existing types and have some retaining of the original name but TypeURIHelperTest also seem to do 1000 things

[stephan-herrmann]

I know that resilience in the face of an incomplete classpath is very painful. That's what the change in JDT is all about: resilience without incorrect behavior.

[...] indicates we at least want to deal with non existing types and have some retaining of the original name

In this case binding recovery sounds like the option of choice, but it may be hard to switch from an implementation that (incompletely?) expects null for error cases towards expecting RecoveredTypeBinding instead.

[stephan-herrmann]

If you are able to isolate the problem to one or two specific situations (like type variable bound java.lang.Object missing), then we could negotiate some tiny exceptions in JDT for situations where the missing type has no semantic relevance.

[cdietrich]

need to wait for feedback from sebastian. hard to judge/do an impact analysis

[szarnekow]

I’ll look into it next week.

[cdietrich]

@szarnekow should temp ignore this test on 4.33?

[LorenzoBettini]

I was about to suggest that

[szarnekow]

Yes please