search

Camelcade / Perl5-IDEA

Perl5 plugins for IntelliJ IDEA
https://plugins.jetbrains.com/plugin/7796-perl/
Other
406 stars 74 forks source link

Do not use stubs while building stubs #2348

Closed hurricup closed 2 months ago

hurricup commented 3 years ago 
Build: IU-203.7717.56
OS: Linux amd64 5.8.5-050805-generic
Java Vendor: JetBrains s.r.o.
Java Version: 11.0.10
Java Arch: 64 bit
Java Runtime Version: 11.0.10+8-b1145.96
Perl Plugin Version: 203.5784.10-EAP-SNAPSHOT
Description: <none>
Message: none
Throwable:
java.lang.Throwable: Indexing process should not rely on non-indexed file data.
Indexing file = /home/cam/dev/gitlab.com/camstuartdev/act/app/script/backtest/strategy-crius, file id = 27274, queried file id = 12777
    at com.intellij.openapi.diagnostic.Logger.error(Logger.java:165)
    at com.intellij.indexing.shared.platform.impl.MergedInvertedIndex.getIndexedFileData(MergedInvertedIndex.java:206)
    at com.intellij.util.indexing.FileBasedIndexEx.lambda$getFileData$3(FileBasedIndexEx.java:168)
    at com.intellij.util.indexing.FileBasedIndexEx.lambda$processExceptions$5(FileBasedIndexEx.java:234)
    at com.intellij.util.ConcurrencyUtil.withLock(ConcurrencyUtil.java:229)
    at com.intellij.util.indexing.FileBasedIndexEx.lambda$processExceptions$6(FileBasedIndexEx.java:234)
    at com.intellij.util.indexing.IndexAccessValidator.validate(IndexAccessValidator.java:33)
    at com.intellij.util.indexing.FileBasedIndexEx.processExceptions(FileBasedIndexEx.java:234)
    at com.intellij.util.indexing.FileBasedIndexEx.getFileData(FileBasedIndexEx.java:161)
    at com.intellij.util.indexing.FileBasedIndexImpl.getFileData(FileBasedIndexImpl.java:1051)
    at com.intellij.psi.stubs.StubTreeLoaderImpl.readFromVFile(StubTreeLoaderImpl.java:99)
    at com.intellij.psi.stubs.StubTreeLoaderImpl.readOrBuild(StubTreeLoaderImpl.java:42)
    at com.intellij.psi.impl.source.PsiFileImpl.getStubTree(PsiFileImpl.java:630)
    at com.intellij.psi.impl.source.PsiFileImpl.getGreenStubTree(PsiFileImpl.java:951)
    at com.intellij.psi.impl.source.PsiFileImpl.getStubbedSpine(PsiFileImpl.java:256)
    at com.intellij.psi.stubs.StubProcessingHelperBase.lambda$getAllSpines$0(StubProcessingHelperBase.java:84)
    at com.intellij.util.containers.ContainerUtil.map(ContainerUtil.java:1983)
    at com.intellij.psi.stubs.StubProcessingHelperBase.getAllSpines(StubProcessingHelperBase.java:84)
    at com.intellij.psi.stubs.StubProcessingHelperBase.processStubsInFile(StubProcessingHelperBase.java:64)
    at com.intellij.psi.stubs.StubIndexImpl.lambda$processElements$2(StubIndexImpl.java:291)
    at com.intellij.psi.stubs.StubIndexImpl.processElements(StubIndexImpl.java:327)
    at com.intellij.psi.stubs.StubIndex.processElements(StubIndex.java:49)
    at com.perl5.lang.perl.psi.stubs.PerlStubIndexBase.processElements(PerlStubIndexBase.java:61)
    at com.perl5.lang.perl.util.PerlPackageUtil.processNamespaces(PerlPackageUtil.java:362)
    at com.perl5.lang.perl.util.PerlPackageUtil.getNamespaceDefinitions(PerlPackageUtil.java:328)
    at com.perl5.lang.perl.psi.mro.PerlMro.getPackageParents(PerlMro.java:185)
    at com.perl5.lang.perl.psi.mro.PerlMro.getLinearISA(PerlMro.java:169)
    at com.perl5.lang.perl.idea.codeInsight.typeInference.value.PerlCallObjectValue.processCallTargets(PerlCallObjectValue.java:120)
    at com.perl5.lang.perl.idea.codeInsight.typeInference.value.PerlCallValue.processCallTargets(PerlCallValue.java:163)
    at com.perl5.lang.perl.extensions.mojo.MojoLightDelegatingSubDefinition.processTargets(MojoLightDelegatingSubDefinition.java:124)
    at com.perl5.lang.perl.extensions.mojo.MojoLightDelegatingSubDefinition.getSubArgumentsList(MojoLightDelegatingSubDefinition.java:55)
    at com.perl5.lang.perl.psi.light.PerlLightSubDefinitionElement.equals(PerlLightSubDefinitionElement.java:141)
    at com.intellij.util.IdempotenceChecker.checkPsiEquivalence(IdempotenceChecker.java:267)
    at com.intellij.util.IdempotenceChecker.checkValueEquivalence(IdempotenceChecker.java:182)
    at com.intellij.util.IdempotenceChecker.checkArrayEquivalence(IdempotenceChecker.java:305)
    at com.intellij.util.IdempotenceChecker.checkCollectionElements(IdempotenceChecker.java:285)
    at com.intellij.util.IdempotenceChecker.checkValueEquivalence(IdempotenceChecker.java:167)
    at com.intellij.util.IdempotenceChecker.checkCachedValueData(IdempotenceChecker.java:232)
    at com.intellij.util.IdempotenceChecker.checkValueEquivalence(IdempotenceChecker.java:164)
    at com.intellij.util.IdempotenceChecker.checkEquivalence(IdempotenceChecker.java:73)
    at com.intellij.util.CachedValueBase.getValueWithLock(CachedValueBase.java:245)
    at com.intellij.psi.impl.PsiCachedValueImpl.getValue(PsiCachedValueImpl.java:43)
    at com.intellij.util.CachedValuesManagerImpl.getCachedValue(CachedValuesManagerImpl.java:78)
    at com.intellij.psi.util.CachedValuesManager.getCachedValue(CachedValuesManager.java:150)
    at com.intellij.psi.util.CachedValuesManager.getCachedValue(CachedValuesManager.java:120)
    at com.perl5.lang.perl.psi.impl.PerlPolyNamedElement.getLightElements(PerlPolyNamedElement.java:56)
    at com.perl5.lang.perl.psi.stubs.PerlPolyNamedElementType.createStub(PerlPolyNamedElementType.java:66)
    at com.perl5.lang.perl.psi.stubs.PerlPolyNamedElementType.createStub(PerlPolyNamedElementType.java:37)
    at com.intellij.psi.stubs.DefaultStubBuilder$StubBuildingWalkingVisitor.createStub(DefaultStubBuilder.java:85)
    at com.intellij.psi.stubs.DefaultStubBuilder$StubBuildingWalkingVisitor.visitNode(DefaultStubBuilder.java:60)
    at com.intellij.psi.stubs.DefaultStubBuilder$StubBuildingWalkingVisitor.buildStubTree(DefaultStubBuilder.java:55)
    at com.intellij.psi.stubs.DefaultStubBuilder.buildStubTreeFor(DefaultStubBuilder.java:33)
    at com.intellij.psi.stubs.DefaultStubBuilder.buildStubTree(DefaultStubBuilder.java:22)
    at com.intellij.psi.stubs.StubTreeBuilder.buildStubTree(StubTreeBuilder.java:122)
    at com.intellij.psi.stubs.StubUpdatingIndex$1.computeValue(StubUpdatingIndex.java:172)
    at com.intellij.psi.stubs.StubUpdatingIndex$1.computeValue(StubUpdatingIndex.java:150)
    at com.intellij.psi.stubs.StubUpdatingIndex$1.computeValue(StubUpdatingIndex.java:117)
    at com.intellij.util.indexing.SingleEntryIndexer.map(SingleEntryIndexer.java:30)
    at com.intellij.util.indexing.SingleEntryIndexer.map(SingleEntryIndexer.java:19)
    at com.intellij.util.indexing.impl.MapReduceIndex.mapByIndexer(MapReduceIndex.java:291)
    at com.intellij.util.indexing.impl.MapReduceIndex.mapInput(MapReduceIndex.java:283)
    at com.intellij.util.indexing.impl.storage.VfsAwareMapReduceIndex.mapInput(VfsAwareMapReduceIndex.java:168)
    at com.intellij.util.indexing.impl.storage.VfsAwareMapReduceIndex.mapInput(VfsAwareMapReduceIndex.java:46)
    at com.intellij.util.indexing.impl.MapReduceIndex.mapInputAndPrepareUpdate(MapReduceIndex.java:226)
    at com.intellij.psi.stubs.StubUpdatingIndex$MyIndex.mapInputAndPrepareUpdate(StubUpdatingIndex.java:448)
    at com.intellij.psi.stubs.StubUpdatingIndex$MyIndex.mapInputAndPrepareUpdate(StubUpdatingIndex.java:408)
    at com.intellij.indexing.shared.platform.impl.MergedInvertedIndex.updateBaseIndex(MergedInvertedIndex.java:330)
    at com.intellij.indexing.shared.platform.impl.MergedInvertedIndex.mapInputAndPrepareUpdate(MergedInvertedIndex.java:116)
    at com.intellij.indexing.shared.platform.impl.MergedInvertedIndex.mapInputAndPrepareUpdate(MergedInvertedIndex.java:38)
    at com.intellij.util.indexing.FileBasedIndexImpl.lambda$indexUnsavedDocument$12(FileBasedIndexImpl.java:1027)
    at com.intellij.openapi.fileTypes.impl.FileTypeManagerImpl.freezeFileTypeTemporarilyIn(FileTypeManagerImpl.java:555)
    at com.intellij.util.indexing.FileBasedIndexImpl.indexUnsavedDocument(FileBasedIndexImpl.java:999)
    at com.intellij.util.indexing.RegisteredIndexes$DocumentUpdateTask.doProcess(RegisteredIndexes.java:181)
    at com.intellij.util.indexing.RegisteredIndexes$DocumentUpdateTask.doProcess(RegisteredIndexes.java:172)
    at com.intellij.util.indexing.UpdateTask.process(UpdateTask.java:64)
    at com.intellij.util.indexing.UpdateTask.processAll(UpdateTask.java:33)
    at com.intellij.util.indexing.FileBasedIndexImpl.lambda$indexUnsavedDocuments$11(FileBasedIndexImpl.java:955)
    at com.intellij.util.indexing.StorageBufferingHandler.runUpdate(StorageBufferingHandler.java:33)
    at com.intellij.util.indexing.FileBasedIndexImpl.indexUnsavedDocuments(FileBasedIndexImpl.java:955)
    at com.intellij.util.indexing.FileBasedIndexImpl.ensureUpToDate(FileBasedIndexImpl.java:730)
    at com.intellij.psi.stubs.StubIndexImpl.getContainingIds(StubIndexImpl.java:465)
    at com.intellij.psi.stubs.StubIndexImpl.processElements(StubIndexImpl.java:298)
    at com.intellij.psi.stubs.StubIndex.getElements(StubIndex.java:105)
    at com.intellij.psi.stubs.StubIndex.getElements(StubIndex.java:93)
    at org.jetbrains.kotlin.idea.stubindex.KotlinFileFacadeFqNameIndex.get(KotlinFileFacadeFqNameIndex.kt:19)
    at org.jetbrains.kotlin.idea.caches.resolve.IDEKotlinAsJavaSupport$findFilesForFacade$1.invoke(IDEKotlinAsJavaSupport.kt:249)
    at org.jetbrains.kotlin.idea.caches.resolve.IDEKotlinAsJavaSupport$findFilesForFacade$1.invoke(IDEKotlinAsJavaSupport.kt:39)
    at org.jetbrains.kotlin.idea.util.application.ApplicationUtilsKt$sam$com_intellij_openapi_util_Computable$0.compute(ApplicationUtils.kt)
    at com.intellij.openapi.application.impl.ApplicationImpl.runReadAction(ApplicationImpl.java:871)
    at org.jetbrains.kotlin.idea.util.application.ApplicationUtilsKt.runReadAction(ApplicationUtils.kt:17)
    at org.jetbrains.kotlin.idea.caches.resolve.IDEKotlinAsJavaSupport.findFilesForFacade(IDEKotlinAsJavaSupport.kt:248)
    at org.jetbrains.kotlin.idea.caches.resolve.IDEKotlinAsJavaSupport.getFacadeClasses(IDEKotlinAsJavaSupport.kt:181)
    at org.jetbrains.kotlin.asJava.finder.JavaElementFinder.findClasses(JavaElementFinder.kt:58)
    at org.jetbrains.kotlin.asJava.finder.JavaElementFinder.findClass(JavaElementFinder.kt:46)
    at com.intellij.psi.impl.JavaPsiFacadeImpl.doFindClass(JavaPsiFacadeImpl.java:89)
    at com.intellij.psi.impl.JavaPsiFacadeImpl.findClass(JavaPsiFacadeImpl.java:67)
    at com.intellij.jam.JavaLibraryUtils.lambda$getLibraryClassMap$0(JavaLibraryUtils.java:46)
    at com.intellij.util.containers.ConcurrentFactoryMap$2.create(ConcurrentFactoryMap.java:174)
    at com.intellij.util.containers.ConcurrentFactoryMap.get(ConcurrentFactoryMap.java:40)
    at java.base/java.util.concurrent.ConcurrentMap.getOrDefault(ConcurrentMap.java:88)
    at com.intellij.jam.JavaLibraryUtils.hasLibraryClass(JavaLibraryUtils.java:29)
    at com.intellij.quarkus.QuarkusUtils.hasQuarkusLibrary(QuarkusUtils.kt:17)
    at com.intellij.quarkus.config.QsConfigKeyReferenceSearcher.processQuery(QsConfigKeyReferenceSearcher.kt:20)
    at com.intellij.quarkus.config.QsConfigKeyReferenceSearcher.processQuery(QsConfigKeyReferenceSearcher.kt:18)
    at com.intellij.openapi.application.QueryExecutorBase.execute(QueryExecutorBase.java:77)
    at com.intellij.util.ExecutorsQuery.processResults(ExecutorsQuery.java:28)
    at com.intellij.util.AbstractQuery.doProcessResults(AbstractQuery.java:99)
    at com.intellij.util.AbstractQuery.delegateProcessResults(AbstractQuery.java:116)
    at com.intellij.util.MergeQuery.processResults(MergeQuery.java:22)
    at com.intellij.util.AbstractQuery.doProcessResults(AbstractQuery.java:99)
    at com.intellij.util.AbstractQuery.delegateProcessResults(AbstractQuery.java:116)
    at com.intellij.util.UniqueResultsQuery.processResults(UniqueResultsQuery.java:39)
    at com.intellij.util.AbstractQuery.doProcessResults(AbstractQuery.java:99)
    at com.intellij.util.AbstractQuery.forEach(AbstractQuery.java:91)
    at com.intellij.util.AbstractQuery.findAll(AbstractQuery.java:28)
    at com.intellij.util.AbstractQuery.iterator(AbstractQuery.java:39)
    at com.perl5.lang.perl.idea.intellilang.PerlStringLanguageInjector.computeInjectionContext(PerlStringLanguageInjector.java:131)
    at com.perl5.lang.perl.idea.intellilang.PerlStringLanguageInjector.lambda$getPerlInjectionContext$0(PerlStringLanguageInjector.java:89)
    at com.intellij.openapi.util.RecursionManager$1.computePreventingRecursion(RecursionManager.java:111)
    at com.intellij.openapi.util.RecursionGuard.doPreventingRecursion(RecursionGuard.java:42)
    at com.intellij.openapi.util.RecursionManager.doPreventingRecursion(RecursionManager.java:68)
    at com.perl5.lang.perl.idea.intellilang.PerlStringLanguageInjector.getPerlInjectionContext(PerlStringLanguageInjector.java:88)
    at com.perl5.lang.perl.idea.intellilang.PerlStringLanguageInjector.getLanguagesToInject(PerlStringLanguageInjector.java:67)
    at com.intellij.psi.impl.source.tree.injected.InjectedLanguageManagerImpl.processInPlaceInjectorsFor(InjectedLanguageManagerImpl.java:418)
    at com.intellij.psi.impl.source.tree.injected.InjectedLanguageUtilBase.probeElementsUp(InjectedLanguageUtilBase.java:207)
    at com.intellij.psi.impl.source.tree.injected.InjectedLanguageUtilBase.enumerate(InjectedLanguageUtilBase.java:137)
    at com.intellij.psi.impl.source.tree.injected.InjectedLanguageManagerImpl.enumerateEx(InjectedLanguageManagerImpl.java:318)
    at com.intellij.codeInsight.daemon.impl.LineMarkersPass.queryLineMarkersForInjected(LineMarkersPass.java:211)
    at com.intellij.codeInsight.daemon.impl.LineMarkersPass.queryProviders(LineMarkersPass.java:173)
    at com.intellij.codeInsight.daemon.impl.LineMarkersPass.lambda$doCollectInformation$3(LineMarkersPass.java:91)
    at com.intellij.codeInsight.daemon.impl.Divider.divideInsideAndOutsideInOneRoot(Divider.java:81)
    at com.intellij.codeInsight.daemon.impl.LineMarkersPass.doCollectInformation(LineMarkersPass.java:78)
    at com.intellij.codeHighlighting.TextEditorHighlightingPass.collectInformation(TextEditorHighlightingPass.java:56)
    at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.lambda$doRun$1(PassExecutorService.java:400)
    at com.intellij.openapi.application.impl.ApplicationImpl.tryRunReadAction(ApplicationImpl.java:1137)
    at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.lambda$doRun$2(PassExecutorService.java:393)
    at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:658)
    at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:610)
    at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:65)
    at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.doRun(PassExecutorService.java:392)
    at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.lambda$run$0(PassExecutorService.java:368)
    at com.intellij.openapi.application.impl.ReadMostlyRWLock.executeByImpatientReader(ReadMostlyRWLock.java:172)
    at com.intellij.openapi.application.impl.ApplicationImpl.executeByImpatientReader(ApplicationImpl.java:183)
    at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.run(PassExecutorService.java:366)
    at com.intellij.concurrency.JobLauncherImpl$VoidForkJoinTask$1.exec(JobLauncherImpl.java:188)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
    at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
hurricup commented 8 months ago

Another root issue: https://camelcade.myjetbrains.com/youtrack/issue/CAMELCADE-18243/java.lang.Throwable-Indexing-process-should-not-rely-on-non-indexed-file-data.

hurricup commented 2 months ago

So, what is going on here. When building stubs for the light elements, we are using CVM which may perform random idempotency checking. And it involves equality checking for the light elements we collected.

For some reason, mojo lite methods were implemented via runtime declarations (acutally perl does this, so I probably just did the same). Atm for mojo lite we are not just exporting some implicitly declared methods, but creating some light declarations for each file with mojo lite usage. And they are delegating arguments computation to their counterparts. And comparison of them require indexes usage.

Feels like to fix this i need to change mojo lite approach to the implicit declarations + imports. And this should work just fine.