Storage2UriMapperJavaImpl::doInitializeCache causes needless rebuilds after restarting Eclipse (2021-12)

[haubi]

So I do have annoying runs of the auto build here for ~10min when restarting Eclipse despite everything was built up to date just before the restart - on Linux with our application having >500 projects and >50k source files:

1. Have each (auto) builder up to date, as in: repeated disable+enable auto build does not build anything anymore.
2. Leave auto build enabled.
3. Restart Eclipse 2021-12 (not discovered with 2021-06).
4. See the auto build running for annoying long duration.

The culprit is that "Storage2UriMapperJavaImpl::doInitializeCache" thread may run before the "Initializing Java Tooling" thread. Because of being early, project information may not be restored from saved Eclipse state yet. Hence, projects are set up from scratch to some degree, ending up with calls to Project.touch(). The result is a modified ResourceInfo.charsetAndContentId on the working tree projects, causing builders to be out of date for no good reason.

When using a breakpoint in Storage2UriMapperJavaImpl.doInitializeCache to wait for the "Initializing Java Tooling" thread to finish, Project.touch() is not called any more, and the builders remain up to date as restored from saved Eclipse state.

Stack trace of ResourceInfo.incrementContentId() being called in "Storage2UriMapperJavaImpl::doInitializeCache" thread:

"Storage2UriMapperJavaImpl::doInitializeCache" #48 prio=6 os_prio=0 cpu=2534.78ms elapsed=48.41s tid=0x00007fc58f5dd000 nid=0x3bbb5 at breakpoint [0x00007fc4d8281000] java.lang.Thread.State: RUNNABLE at org.eclipse.core.internal.resources.ResourceInfo.incrementContentId(ResourceInfo.java:287) at org.eclipse.core.internal.resources.Resource.touch(Resource.java:1759) at org.eclipse.core.internal.resources.Project.touch(Project.java:1323) at org.eclipse.jdt.internal.core.SetContainerOperation.executeOperation(SetContainerOperation.java:122) at org.eclipse.jdt.internal.core.JavaModelOperation.run(JavaModelOperation.java:740) at org.eclipse.core.internal.resources.Workspace.run(Workspace.java:2313) at org.eclipse.core.internal.resources.Workspace.run(Workspace.java:2338) at org.eclipse.jdt.internal.core.JavaModelOperation.runOperation(JavaModelOperation.java:811) at org.eclipse.jdt.internal.core.JavaModelManager.getClasspathContainer(JavaModelManager.java:2095) at org.eclipse.jdt.core.JavaCore.getClasspathContainer(JavaCore.java:3783) at org.eclipse.jdt.internal.core.JavaProject.resolveClasspath(JavaProject.java:3240) at org.eclipse.jdt.internal.core.JavaProject.resolveClasspath(JavaProject.java:3404) at org.eclipse.jdt.internal.core.JavaProject.getResolvedClasspath(JavaProject.java:2489) at org.eclipse.jdt.internal.core.JavaProject.buildStructure(JavaProject.java:535) at org.eclipse.jdt.internal.core.Openable.generateInfos(Openable.java:266) at org.eclipse.jdt.internal.core.JavaElement.openWhenClosed(JavaElement.java:597) at org.eclipse.jdt.internal.core.JavaElement.getElementInfo(JavaElement.java:328) at org.eclipse.jdt.internal.core.JavaElement.getElementInfo(JavaElement.java:314) at org.eclipse.jdt.internal.core.JavaElement.getChildren(JavaElement.java:269) at org.eclipse.jdt.internal.core.JavaProject.getPackageFragmentRoots(JavaProject.java:2353) at org.eclipse.xtext.ui.resource.Storage2UriMapperJavaImpl.updateCache(Storage2UriMapperJavaImpl.java:478) at org.eclipse.xtext.ui.resource.Storage2UriMapperJavaImpl.access$7(Storage2UriMapperJavaImpl.java:474) at org.eclipse.xtext.ui.resource.Storage2UriMapperJavaImpl$4.run(Storage2UriMapperJavaImpl.java:674) at org.eclipse.core.internal.resources.Workspace.run(Workspace.java:2313) at org.eclipse.core.internal.resources.Workspace.run(Workspace.java:2338) at org.eclipse.xtext.ui.resource.Storage2UriMapperJavaImpl.doInitializeCache(Storage2UriMapperJavaImpl.java:686) at org.eclipse.xtext.ui.resource.Storage2UriMapperJavaImpl.access$3(Storage2UriMapperJavaImpl.java:665) at org.eclipse.xtext.ui.resource.Storage2UriMapperJavaImpl$3.run(Storage2UriMapperJavaImpl.java:654) at java.lang.Thread.run(java.base@11.0.14/Thread.java:829)

[cdietrich]

do you have any idea how to resolve this?

can you also test with platform 4.24 M1 and Xtext 2.27.0.M1 https://github.com/eclipse/xtext-eclipse/issues/1820

[cdietrich]

cc @iloveeclipse. was there anything changed about "project information may not be restored from saved Eclipse state yet"

[cdietrich]

@haubi is this related to https://bugs.eclipse.org/bugs/show_bug.cgi?id=568324 ?

[iloveeclipse]

2021-06 has already https://bugs.eclipse.org/bugs/show_bug.cgi?id=568324

[haubi]

With 2021-06 I have neither discovered nor debugged this situation, but I can imagine just the timing between these threads to be different there to not trigger this problem. As far as I understand from it's description, https://bugs.eclipse.org/bugs/show_bug.cgi?id=568324 may indeed be related.

cc @jukzi for (helping me to) eventually test with more recent platform and Xtext

[iloveeclipse]

Hard to say what could be changes in last few years especially if startup timing is the problem. If one could at least find the exact release... There were lot of changes that might affect startup timing / order in entire SDK.

But if Storage2UriMapperJavaImpl requires JDT to be initialized, may be better to search for the solution. If this code in Xtext starts before JDT is fully initialized, the question is: what is "fully initialized JDT state" that is needed by Storage2UriMapperJavaImpl. #1836 however looks like a hack.

[ghost]

i guess you only just faced https://bugs.eclipse.org/bugs/show_bug.cgi?id=578968

[haubi]

i guess you only just faced https://bugs.eclipse.org/bugs/show_bug.cgi?id=578968

Indeed! And the original report is talking about Xtext as well: https://bugs.eclipse.org/bugs/show_bug.cgi?id=578564

Thanks a lot! (closing)