search

cedricziel / idea-php-typo3-plugin

TYPO3 CMS Plugins for IntelliJ IDEA / PhpStorm
MIT License
96 stars 8 forks source link

High load due to repeated indexing #361

Open liayn opened 2 years ago

liayn commented 2 years ago

Describe the bug Opening a plain clone of ext:oidc in phpstorm (currently this commit is master https://github.com/xperseguers/t3ext-oidc/commit/c27ec0be337ced724be783ad61b0986225d541c3) causes massive CPU load due to repeated indexing every few seconds.

The diagnosis shows that the typo3-plugin is causing the indexing with the message Rebuild requested for index com.cedricziel.idea.typo3.index.core_service_map

This did not happen with other projects to me lately, so this is a nice simple project to reproduce.

(Maybe related to #327 ?)

To Reproduce Steps to reproduce the behavior:

  1. Clone https://github.com/xperseguers/t3ext-oidc
  2. Open as project
  3. Simply wait and watch the indexing happening

Expected behavior No indexing, when nothing has changed.

Versions

PhpStorm 2021.2.3
Build #PS-212.5457.49, built on October 13, 2021
Runtime version: 11.0.12+7-b1504.40 amd64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
Windows 10 10.0
GC: ParNew, ConcurrentMarkSweep
Memory: 5700M
Cores: 12
Registry: run.processes.with.pty=TRUE, ide.tooltip.initialDelay=720, ide.images.show.chessboard=true
Non-Bundled Plugins: String Manipulation (8.20.203.000.1), com.urswolfer.intellij.plugin.gerrit (1.2.5-203), YAML/Ansible support (0.11.2), de.sgalinski.fluid.plugin.id (2.5.1), de.sgalinski.typoscript.enterprise.plugin.id (2.5.1), com.cedricziel.idea.typo3 (212.7.2)
boehmmatthias commented 2 years ago

Same issue - different project. 

PhpStorm 2021.3
Build #PS-213.5744.279
Runtime version: 11.0.13+7-b1751.19 amd64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
OS: Ubuntu 20.04.3 LTS

Logs:

2021-12-22 09:03:06,712 [ 235700]   INFO - indexing.UnindexedFilesUpdater - Cancelled indexing of sitesetup 
2021-12-22 09:03:06,712 [ 235700]   INFO - ij.psi.search.LogFileTypeIndex - File type index snapshot dropped 
2021-12-22 09:03:06,712 [ 235700]   INFO - ij.psi.search.LogFileTypeIndex - Loading file type index snapshot 
2021-12-22 09:03:06,713 [ 235701]   INFO - indexing.UnindexedFilesUpdater - Started indexing of sitesetup. Reason: Rebuild requested for index com.cedricziel.idea.typo3.index.core_service_map 
2021-12-22 09:03:06,713 [ 235701]   INFO - indexing.UnindexedFilesUpdater - Performing delayed pushing properties tasks for sitesetup took 0ms; general responsiveness: ok; EDT responsiveness: ok 
2021-12-22 09:03:06,714 [ 235702]   INFO - indexing.UnindexedFilesUpdater - Scanning of sitesetup uses 11 scanning threads 
2021-12-22 09:03:07,422 [ 236410]   INFO - indexing.UnindexedFilesUpdater - Scanning completed for sitesetup. Number of scanned files: 15409; Number of files for indexing: 834 took 709ms; general responsiveness: ok; EDT responsiveness: ok 
2021-12-22 09:03:07,428 [ 236416]   INFO - indexing.UnindexedFilesUpdater - Use 4 indexing threads for indexing of sitesetup 
2021-12-22 09:03:07,433 [ 236421]   INFO - l.indexing.impl.MapReduceIndex - An exception during updateWithMap(). Index com.cedricziel.idea.typo3.index.core_service_map will be rebuilt. 
com.intellij.util.indexing.StorageException: java.lang.IllegalArgumentException: Argument for @NotNull parameter 'value' of com/intellij/util/io/EnumeratorStringDescriptor.save must not be null
    at com.intellij.util.indexing.impl.MapReduceIndex.updateWithMap(MapReduceIndex.java:355)
    at com.intellij.util.indexing.impl.storage.VfsAwareMapReduceIndex.updateWithMap(VfsAwareMapReduceIndex.java:340)
    at com.intellij.util.indexing.impl.MapReduceIndex$IndexUpdateComputable.compute(MapReduceIndex.java:384)
    at com.intellij.util.indexing.impl.MapReduceIndex$IndexUpdateComputable.compute(MapReduceIndex.java:366)
    at com.intellij.openapi.util.Computable.get(Computable.java:18)
    at com.intellij.indexing.composite.CompositeInvertedIndexBase.lambda$mapInputAndPrepareUpdate$3(CompositeInvertedIndexBase.java:73)
    at com.intellij.util.indexing.IndexInfrastructureExtensionUpdateComputation.compute(IndexInfrastructureExtensionUpdateComputation.java:27)
    at com.intellij.util.indexing.IndexInfrastructureExtensionUpdateComputation.compute(IndexInfrastructureExtensionUpdateComputation.java:11)
    at com.intellij.util.indexing.FileBasedIndexImpl.lambda$runUpdateForPersistentData$30(FileBasedIndexImpl.java:1597)
    at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:698)
    at com.intellij.openapi.progress.impl.CoreProgressManager.computeUnderProgress(CoreProgressManager.java:646)
    at com.intellij.openapi.progress.impl.CoreProgressManager.computeInNonCancelableSection(CoreProgressManager.java:237)
    at com.intellij.util.indexing.FileBasedIndexImpl.lambda$runUpdateForPersistentData$31(FileBasedIndexImpl.java:1597)
    at com.intellij.util.indexing.StorageBufferingHandler.runUpdate(StorageBufferingHandler.java:22)
    at com.intellij.util.indexing.FileBasedIndexImpl.runUpdateForPersistentData(FileBasedIndexImpl.java:1596)
    at com.intellij.util.indexing.FileBasedIndexImpl.updateSingleIndex(FileBasedIndexImpl.java:1559)
    at com.intellij.util.indexing.FileBasedIndexImpl.lambda$doIndexFileContent$27(FileBasedIndexImpl.java:1408)
    at com.intellij.openapi.fileTypes.impl.FileTypeManagerImpl.freezeFileTypeTemporarilyIn(FileTypeManagerImpl.java:627)
    at com.intellij.util.indexing.FileBasedIndexImpl.doIndexFileContent(FileBasedIndexImpl.java:1376)
    at com.intellij.util.indexing.FileBasedIndexImpl.indexFileContent(FileBasedIndexImpl.java:1330)
    at com.intellij.util.indexing.contentQueue.IndexUpdateRunner.lambda$indexOneFileOfJob$5(IndexUpdateRunner.java:267)
    at com.intellij.openapi.application.impl.NonBlockingReadActionImpl$Submission.insideReadAction(NonBlockingReadActionImpl.java:521)
    at com.intellij.openapi.application.impl.NonBlockingReadActionImpl$Submission.lambda$attemptComputation$3(NonBlockingReadActionImpl.java:486)
    at com.intellij.openapi.application.impl.ApplicationImpl.tryRunReadAction(ApplicationImpl.java:1084)
    at com.intellij.openapi.progress.util.ProgressIndicatorUtils.lambda$runInReadActionWithWriteActionPriority$0(ProgressIndicatorUtils.java:75)
    at com.intellij.openapi.progress.util.ProgressIndicatorUtils.runActionAndCancelBeforeWrite(ProgressIndicatorUtils.java:158)
    at com.intellij.openapi.progress.util.ProgressIndicatorUtils.lambda$runWithWriteActionPriority$1(ProgressIndicatorUtils.java:115)
    at com.intellij.openapi.progress.ProgressManager.lambda$runProcess$0(ProgressManager.java:57)
    at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcess$2(CoreProgressManager.java:188)
    at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$executeProcessUnderProgress$12(CoreProgressManager.java:624)
    at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:698)
    at com.intellij.openapi.progress.impl.CoreProgressManager.computeUnderProgress(CoreProgressManager.java:646)
    at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:623)
    at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:66)
    at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:175)
    at com.intellij.openapi.progress.ProgressManager.runProcess(ProgressManager.java:57)
    at com.intellij.openapi.progress.util.ProgressIndicatorUtils.runWithWriteActionPriority(ProgressIndicatorUtils.java:112)
    at com.intellij.openapi.progress.util.ProgressIndicatorUtils.runInReadActionWithWriteActionPriority(ProgressIndicatorUtils.java:75)
    at com.intellij.openapi.application.impl.NonBlockingReadActionImpl$Submission.attemptComputation(NonBlockingReadActionImpl.java:486)
    at com.intellij.openapi.application.impl.NonBlockingReadActionImpl$Submission.executeSynchronously(NonBlockingReadActionImpl.java:421)
    at com.intellij.openapi.application.impl.NonBlockingReadActionImpl.executeSynchronously(NonBlockingReadActionImpl.java:186)
    at com.intellij.util.indexing.contentQueue.IndexUpdateRunner.indexOneFileOfJob(IndexUpdateRunner.java:270)
    at com.intellij.util.indexing.contentQueue.IndexUpdateRunner.lambda$indexJobsFairly$4(IndexUpdateRunner.java:210)
    at com.intellij.openapi.progress.impl.ProgressSuspender.executeNonSuspendableSection(ProgressSuspender.java:83)
    at com.intellij.util.indexing.contentQueue.IndexUpdateRunner.indexJobsFairly(IndexUpdateRunner.java:213)
    at com.intellij.util.indexing.contentQueue.IndexUpdateRunner.lambda$doIndexFiles$3(IndexUpdateRunner.java:147)
    at com.intellij.util.concurrency.BoundedTaskExecutor.doRun(BoundedTaskExecutor.java:246)
    at com.intellij.util.concurrency.BoundedTaskExecutor.access$200(BoundedTaskExecutor.java:32)
    at com.intellij.util.concurrency.BoundedTaskExecutor$1.execute(BoundedTaskExecutor.java:225)
    at com.intellij.util.ConcurrencyUtil.runUnderThreadName(ConcurrencyUtil.java:213)
    at com.intellij.util.concurrency.BoundedTaskExecutor$1.run(BoundedTaskExecutor.java:214)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:668)
    at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:665)
    at java.base/java.security.AccessController.doPrivileged(Native Method)
    at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1.run(Executors.java:665)
    at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.IllegalArgumentException: Argument for @NotNull parameter 'value' of com/intellij/util/io/EnumeratorStringDescriptor.save must not be null
    at com.intellij.util.io.EnumeratorStringDescriptor.$$$reportNull$$$0(EnumeratorStringDescriptor.java)
    at com.intellij.util.io.EnumeratorStringDescriptor.save(EnumeratorStringDescriptor.java)
    at com.cedricziel.idea.typo3.index.externalizer.StringSetDataExternalizer.save(StringSetDataExternalizer.java:23)
    at com.cedricziel.idea.typo3.index.externalizer.StringSetDataExternalizer.save(StringSetDataExternalizer.java:13)
    at com.intellij.util.indexing.InputMapExternalizer.save(InputMapExternalizer.java:69)
    at com.intellij.util.indexing.InputMapExternalizer.save(InputMapExternalizer.java:19)
    at com.intellij.util.indexing.impl.forward.AbstractForwardIndexAccessor.serializeToByteSeq(AbstractForwardIndexAccessor.java:81)
    at com.intellij.util.indexing.impl.forward.AbstractForwardIndexAccessor.serializeValueToByteSeq(AbstractForwardIndexAccessor.java:69)
    at com.intellij.util.indexing.impl.forward.AbstractForwardIndexAccessor.serializeIndexedData(AbstractForwardIndexAccessor.java:50)
    at com.intellij.util.indexing.impl.forward.AbstractForwardIndexAccessor.serializeIndexedData(AbstractForwardIndexAccessor.java:44)
    at com.intellij.util.indexing.impl.MapReduceIndex.updateForwardIndex(MapReduceIndex.java:276)
    at com.intellij.util.indexing.impl.storage.TransientFileContentIndex.updateForwardIndex(TransientFileContentIndex.java:86)
    at com.intellij.util.indexing.impl.MapReduceIndex.lambda$mapInputAndPrepareUpdate$4(MapReduceIndex.java:261)
    at com.intellij.util.indexing.impl.UpdateData.updateForwardIndex(UpdateData.java:63)
    at com.intellij.util.indexing.impl.MapReduceIndex.updateWithMap(MapReduceIndex.java:348)
    ... 57 more
2021-12-22 09:03:07,436 [ 236424]   INFO - il.indexing.FileBasedIndexImpl - Rebuild requested for index com.cedricziel.idea.typo3.index.core_service_map 
com.intellij.util.indexing.StorageException: java.lang.IllegalArgumentException: Argument for @NotNull parameter 'value' of com/intellij/util/io/EnumeratorStringDescriptor.save must not be null
    at com.intellij.util.indexing.impl.MapReduceIndex.updateWithMap(MapReduceIndex.java:355)
    at com.intellij.util.indexing.impl.storage.VfsAwareMapReduceIndex.updateWithMap(VfsAwareMapReduceIndex.java:340)
    at com.intellij.util.indexing.impl.MapReduceIndex$IndexUpdateComputable.compute(MapReduceIndex.java:384)
    at com.intellij.util.indexing.impl.MapReduceIndex$IndexUpdateComputable.compute(MapReduceIndex.java:366)
    at com.intellij.openapi.util.Computable.get(Computable.java:18)
    at com.intellij.indexing.composite.CompositeInvertedIndexBase.lambda$mapInputAndPrepareUpdate$3(CompositeInvertedIndexBase.java:73)
    at com.intellij.util.indexing.IndexInfrastructureExtensionUpdateComputation.compute(IndexInfrastructureExtensionUpdateComputation.java:27)
    at com.intellij.util.indexing.IndexInfrastructureExtensionUpdateComputation.compute(IndexInfrastructureExtensionUpdateComputation.java:11)
    at com.intellij.util.indexing.FileBasedIndexImpl.lambda$runUpdateForPersistentData$30(FileBasedIndexImpl.java:1597)
    at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:698)
    at com.intellij.openapi.progress.impl.CoreProgressManager.computeUnderProgress(CoreProgressManager.java:646)
    at com.intellij.openapi.progress.impl.CoreProgressManager.computeInNonCancelableSection(CoreProgressManager.java:237)
    at com.intellij.util.indexing.FileBasedIndexImpl.lambda$runUpdateForPersistentData$31(FileBasedIndexImpl.java:1597)
    at com.intellij.util.indexing.StorageBufferingHandler.runUpdate(StorageBufferingHandler.java:22)
    at com.intellij.util.indexing.FileBasedIndexImpl.runUpdateForPersistentData(FileBasedIndexImpl.java:1596)
    at com.intellij.util.indexing.FileBasedIndexImpl.updateSingleIndex(FileBasedIndexImpl.java:1559)
    at com.intellij.util.indexing.FileBasedIndexImpl.lambda$doIndexFileContent$27(FileBasedIndexImpl.java:1408)
    at com.intellij.openapi.fileTypes.impl.FileTypeManagerImpl.freezeFileTypeTemporarilyIn(FileTypeManagerImpl.java:627)
    at com.intellij.util.indexing.FileBasedIndexImpl.doIndexFileContent(FileBasedIndexImpl.java:1376)
    at com.intellij.util.indexing.FileBasedIndexImpl.indexFileContent(FileBasedIndexImpl.java:1330)
    at com.intellij.util.indexing.contentQueue.IndexUpdateRunner.lambda$indexOneFileOfJob$5(IndexUpdateRunner.java:267)
    at com.intellij.openapi.application.impl.NonBlockingReadActionImpl$Submission.insideReadAction(NonBlockingReadActionImpl.java:521)
    at com.intellij.openapi.application.impl.NonBlockingReadActionImpl$Submission.lambda$attemptComputation$3(NonBlockingReadActionImpl.java:486)
    at com.intellij.openapi.application.impl.ApplicationImpl.tryRunReadAction(ApplicationImpl.java:1084)
    at com.intellij.openapi.progress.util.ProgressIndicatorUtils.lambda$runInReadActionWithWriteActionPriority$0(ProgressIndicatorUtils.java:75)
    at com.intellij.openapi.progress.util.ProgressIndicatorUtils.runActionAndCancelBeforeWrite(ProgressIndicatorUtils.java:158)
    at com.intellij.openapi.progress.util.ProgressIndicatorUtils.lambda$runWithWriteActionPriority$1(ProgressIndicatorUtils.java:115)
    at com.intellij.openapi.progress.ProgressManager.lambda$runProcess$0(ProgressManager.java:57)
    at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcess$2(CoreProgressManager.java:188)
    at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$executeProcessUnderProgress$12(CoreProgressManager.java:624)
    at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:698)
    at com.intellij.openapi.progress.impl.CoreProgressManager.computeUnderProgress(CoreProgressManager.java:646)
    at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:623)
    at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:66)
    at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:175)
    at com.intellij.openapi.progress.ProgressManager.runProcess(ProgressManager.java:57)
    at com.intellij.openapi.progress.util.ProgressIndicatorUtils.runWithWriteActionPriority(ProgressIndicatorUtils.java:112)
    at com.intellij.openapi.progress.util.ProgressIndicatorUtils.runInReadActionWithWriteActionPriority(ProgressIndicatorUtils.java:75)
    at com.intellij.openapi.application.impl.NonBlockingReadActionImpl$Submission.attemptComputation(NonBlockingReadActionImpl.java:486)
    at com.intellij.openapi.application.impl.NonBlockingReadActionImpl$Submission.executeSynchronously(NonBlockingReadActionImpl.java:421)
    at com.intellij.openapi.application.impl.NonBlockingReadActionImpl.executeSynchronously(NonBlockingReadActionImpl.java:186)
    at com.intellij.util.indexing.contentQueue.IndexUpdateRunner.indexOneFileOfJob(IndexUpdateRunner.java:270)
    at com.intellij.util.indexing.contentQueue.IndexUpdateRunner.lambda$indexJobsFairly$4(IndexUpdateRunner.java:210)
    at com.intellij.openapi.progress.impl.ProgressSuspender.executeNonSuspendableSection(ProgressSuspender.java:83)
    at com.intellij.util.indexing.contentQueue.IndexUpdateRunner.indexJobsFairly(IndexUpdateRunner.java:213)
    at com.intellij.util.indexing.contentQueue.IndexUpdateRunner.lambda$doIndexFiles$3(IndexUpdateRunner.java:147)
    at com.intellij.util.concurrency.BoundedTaskExecutor.doRun(BoundedTaskExecutor.java:246)
    at com.intellij.util.concurrency.BoundedTaskExecutor.access$200(BoundedTaskExecutor.java:32)
    at com.intellij.util.concurrency.BoundedTaskExecutor$1.execute(BoundedTaskExecutor.java:225)
    at com.intellij.util.ConcurrencyUtil.runUnderThreadName(ConcurrencyUtil.java:213)
    at com.intellij.util.concurrency.BoundedTaskExecutor$1.run(BoundedTaskExecutor.java:214)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:668)
    at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:665)
    at java.base/java.security.AccessController.doPrivileged(Native Method)
    at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1.run(Executors.java:665)
    at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.IllegalArgumentException: Argument for @NotNull parameter 'value' of com/intellij/util/io/EnumeratorStringDescriptor.save must not be null
    at com.intellij.util.io.EnumeratorStringDescriptor.$$$reportNull$$$0(EnumeratorStringDescriptor.java)
    at com.intellij.util.io.EnumeratorStringDescriptor.save(EnumeratorStringDescriptor.java)
    at com.cedricziel.idea.typo3.index.externalizer.StringSetDataExternalizer.save(StringSetDataExternalizer.java:23)
    at com.cedricziel.idea.typo3.index.externalizer.StringSetDataExternalizer.save(StringSetDataExternalizer.java:13)
    at com.intellij.util.indexing.InputMapExternalizer.save(InputMapExternalizer.java:69)
    at com.intellij.util.indexing.InputMapExternalizer.save(InputMapExternalizer.java:19)
    at com.intellij.util.indexing.impl.forward.AbstractForwardIndexAccessor.serializeToByteSeq(AbstractForwardIndexAccessor.java:81)
    at com.intellij.util.indexing.impl.forward.AbstractForwardIndexAccessor.serializeValueToByteSeq(AbstractForwardIndexAccessor.java:69)
    at com.intellij.util.indexing.impl.forward.AbstractForwardIndexAccessor.serializeIndexedData(AbstractForwardIndexAccessor.java:50)
    at com.intellij.util.indexing.impl.forward.AbstractForwardIndexAccessor.serializeIndexedData(AbstractForwardIndexAccessor.java:44)
    at com.intellij.util.indexing.impl.MapReduceIndex.updateForwardIndex(MapReduceIndex.java:276)
    at com.intellij.util.indexing.impl.storage.TransientFileContentIndex.updateForwardIndex(TransientFileContentIndex.java:86)
    at com.intellij.util.indexing.impl.MapReduceIndex.lambda$mapInputAndPrepareUpdate$4(MapReduceIndex.java:261)
    at com.intellij.util.indexing.impl.UpdateData.updateForwardIndex(UpdateData.java:63)
    at com.intellij.util.indexing.impl.MapReduceIndex.updateWithMap(MapReduceIndex.java:348)
    ... 57 more
liayn commented 2 years ago

Curiously the issue with the extension oidc seems to be resolved with the updated versions

PhpStorm 2021.3
Build #PS-213.5744.279, built on December 2, 2021

Non-Bundled Plugins:
    com.cedricziel.idea.typo3 (213.7.3)
sfroemkenjw commented 1 year ago

I can confirm the problem:

PhpStorm 2023.1.2 Build #PS-231.9011.38, built on May 16, 2023

TYPO3 CMS Support: 231.9.2

EXT:oicd 1.2.0