search

Roave / infection-static-analysis-plugin

:white_check_mark: :dragon_face: Static analysis on top of mutation testing - prevents escaped mutants from being invalid according to static analysis
MIT License
120 stars 17 forks source link

Error after update to `1.28` version #465

Open vjik opened 1 year ago

vjik commented 1 year ago 
In ClassLikeStorageProvider.php line 45:

  Could not get class storage for yiisoft\html\tag\base\booleaninputtag

Example: https://github.com/yiisoft/html/actions/runs/3817491976/jobs/6576122054

With 1.27 no errors.

Ocramius commented 1 year ago

Got a longer trace perhaps?

vjik commented 1 year ago 
In ClassLikeStorageProvider.php line 45:

  [InvalidArgumentException]
  Could not get class storage for yiisoft\html\tag\base\booleaninputtag

Exception trace:
  at /projects/open-source/yiisoft/html/vendor/vimeo/psalm/src/Psalm/Internal/Provider/ClassLikeStorageProvider.php:45
 Psalm\Internal\Provider\ClassLikeStorageProvider->get() at /projects/open-source/yiisoft/html/vendor/vimeo/psalm/src/Psalm/Internal/Codebase/Methods.php:240
 Psalm\Internal\Codebase\Methods->methodExists() at /projects/open-source/yiisoft/html/vendor/vimeo/psalm/src/Psalm/Internal/Analyzer/Statements/Expression/Call/Method/AtomicMethodCallAnalyzer.php:263
 Psalm\Internal\Analyzer\Statements\Expression\Call\Method\AtomicMethodCallAnalyzer::analyze() at /projects/open-source/yiisoft/html/vendor/vimeo/psalm/src/Psalm/Internal/Analyzer/Statements/Expression/Call/MethodCallAnalyzer.php:184
 Psalm\Internal\Analyzer\Statements\Expression\Call\MethodCallAnalyzer::analyze() at /projects/open-source/yiisoft/html/vendor/vimeo/psalm/src/Psalm/Internal/Analyzer/Statements/ExpressionAnalyzer.php:186
 Psalm\Internal\Analyzer\Statements\ExpressionAnalyzer::handleExpression() at /projects/open-source/yiisoft/html/vendor/vimeo/psalm/src/Psalm/Internal/Analyzer/Statements/ExpressionAnalyzer.php:86
 Psalm\Internal\Analyzer\Statements\ExpressionAnalyzer::analyze() at /projects/open-source/yiisoft/html/vendor/vimeo/psalm/src/Psalm/Internal/Analyzer/Statements/Expression/Call/MethodCallAnalyzer.php:63
 Psalm\Internal\Analyzer\Statements\Expression\Call\MethodCallAnalyzer::analyze() at /projects/open-source/yiisoft/html/vendor/vimeo/psalm/src/Psalm/Internal/Analyzer/Statements/ExpressionAnalyzer.php:186
 Psalm\Internal\Analyzer\Statements\ExpressionAnalyzer::handleExpression() at /projects/open-source/yiisoft/html/vendor/vimeo/psalm/src/Psalm/Internal/Analyzer/Statements/ExpressionAnalyzer.php:86
 Psalm\Internal\Analyzer\Statements\ExpressionAnalyzer::analyze() at /projects/open-source/yiisoft/html/vendor/vimeo/psalm/src/Psalm/Internal/Analyzer/Statements/Expression/Call/MethodCallAnalyzer.php:63
 Psalm\Internal\Analyzer\Statements\Expression\Call\MethodCallAnalyzer::analyze() at /projects/open-source/yiisoft/html/vendor/vimeo/psalm/src/Psalm/Internal/Analyzer/Statements/ExpressionAnalyzer.php:186
 Psalm\Internal\Analyzer\Statements\ExpressionAnalyzer::handleExpression() at /projects/open-source/yiisoft/html/vendor/vimeo/psalm/src/Psalm/Internal/Analyzer/Statements/ExpressionAnalyzer.php:86
 Psalm\Internal\Analyzer\Statements\ExpressionAnalyzer::analyze() at /projects/open-source/yiisoft/html/vendor/vimeo/psalm/src/Psalm/Internal/Analyzer/Statements/Expression/AssignmentAnalyzer.php:242
 Psalm\Internal\Analyzer\Statements\Expression\AssignmentAnalyzer::analyze() at /projects/open-source/yiisoft/html/vendor/vimeo/psalm/src/Psalm/Internal/Analyzer/Statements/ExpressionAnalyzer.php:544
 Psalm\Internal\Analyzer\Statements\ExpressionAnalyzer::analyzeAssignment() at /projects/open-source/yiisoft/html/vendor/vimeo/psalm/src/Psalm/Internal/Analyzer/Statements/ExpressionAnalyzer.php:178
 Psalm\Internal\Analyzer\Statements\ExpressionAnalyzer::handleExpression() at /projects/open-source/yiisoft/html/vendor/vimeo/psalm/src/Psalm/Internal/Analyzer/Statements/ExpressionAnalyzer.php:86
 Psalm\Internal\Analyzer\Statements\ExpressionAnalyzer::analyze() at /projects/open-source/yiisoft/html/vendor/vimeo/psalm/src/Psalm/Internal/Analyzer/StatementsAnalyzer.php:565
 Psalm\Internal\Analyzer\StatementsAnalyzer::analyzeStatement() at /projects/open-source/yiisoft/html/vendor/vimeo/psalm/src/Psalm/Internal/Analyzer/StatementsAnalyzer.php:194
 Psalm\Internal\Analyzer\StatementsAnalyzer->analyze() at /projects/open-source/yiisoft/html/vendor/vimeo/psalm/src/Psalm/Internal/Analyzer/FunctionLikeAnalyzer.php:457
 Psalm\Internal\Analyzer\FunctionLikeAnalyzer->analyze() at /projects/open-source/yiisoft/html/vendor/vimeo/psalm/src/Psalm/Internal/Analyzer/ClassAnalyzer.php:1798
 Psalm\Internal\Analyzer\ClassAnalyzer->analyzeClassMethod() at /projects/open-source/yiisoft/html/vendor/vimeo/psalm/src/Psalm/Internal/Analyzer/ClassAnalyzer.php:425
 Psalm\Internal\Analyzer\ClassAnalyzer->analyze() at /projects/open-source/yiisoft/html/vendor/vimeo/psalm/src/Psalm/Internal/Analyzer/FileAnalyzer.php:203
 Psalm\Internal\Analyzer\FileAnalyzer->analyze() at /projects/open-source/yiisoft/html/vendor/vimeo/psalm/src/Psalm/Internal/Codebase/Analyzer.php:1581
 Psalm\Internal\Codebase\Analyzer->analysisWorker() at /projects/open-source/yiisoft/html/vendor/vimeo/psalm/src/Psalm/Internal/Codebase/Analyzer.php:522
 Psalm\Internal\Codebase\Analyzer->doAnalysis() at /projects/open-source/yiisoft/html/vendor/vimeo/psalm/src/Psalm/Internal/Codebase/Analyzer.php:272
 Psalm\Internal\Codebase\Analyzer->analyzeFiles() at /projects/open-source/yiisoft/html/vendor/roave/infection-static-analysis-plugin/src/Roave/InfectionStaticAnalysis/Psalm/RunStaticAnalysisAgainstMutant.php:49
 Roave\InfectionStaticAnalysis\Psalm\RunStaticAnalysisAgainstMutant->isMutantStillValidAccordingToStaticAnalysis() at /projects/open-source/yiisoft/html/vendor/roave/infection-static-analysis-plugin/src/Roave/InfectionStaticAnalysis/RunStaticAnalysisAgainstEscapedMutant.php:44
 Roave\InfectionStaticAnalysis\RunStaticAnalysisAgainstEscapedMutant->createFromProcess() at /projects/open-source/yiisoft/html/vendor/infection/infection/src/Process/Factory/MutantProcessFactory.php:90
 Infection\Process\Factory\MutantProcessFactory::Infection\Process\Factory\{closure}() at /projects/open-source/yiisoft/html/vendor/infection/infection/src/Process/MutantProcess.php:88
 Infection\Process\MutantProcess->terminateProcess() at /projects/open-source/yiisoft/html/vendor/infection/infection/src/Process/Runner/ParallelProcessRunner.php:134
 Infection\Process\Runner\ParallelProcessRunner->freeTerminatedProcesses() at /projects/open-source/yiisoft/html/vendor/infection/infection/src/Process/Runner/ParallelProcessRunner.php:106
 Infection\Process\Runner\ParallelProcessRunner->run() at /projects/open-source/yiisoft/html/vendor/infection/infection/src/Process/Runner/MutationTestingRunner.php:121
 Infection\Process\Runner\MutationTestingRunner->run() at /projects/open-source/yiisoft/html/vendor/infection/infection/src/Engine.php:136
 Infection\Engine->runMutationAnalysis() at /projects/open-source/yiisoft/html/vendor/infection/infection/src/Engine.php:74
 Infection\Engine->execute() at /projects/open-source/yiisoft/html/vendor/infection/infection/src/Command/RunCommand.php:377
 Infection\Command\RunCommand->executeCommand() at /projects/open-source/yiisoft/html/vendor/infection/infection/src/Command/BaseCommand.php:75
 Infection\Command\BaseCommand->execute() at /projects/open-source/yiisoft/html/vendor/symfony/console/Command/Command.php:298
 Symfony\Component\Console\Command\Command->run() at /projects/open-source/yiisoft/html/vendor/symfony/console/Application.php:1040
 Symfony\Component\Console\Application->doRunCommand() at /projects/open-source/yiisoft/html/vendor/symfony/console/Application.php:301
 Symfony\Component\Console\Application->doRun() at /projects/open-source/yiisoft/html/vendor/symfony/console/Application.php:171
 Symfony\Component\Console\Application->run() at /projects/open-source/yiisoft/html/vendor/roave/infection-static-analysis-plugin/bin/roave-infection-static-analysis-plugin:86
 Roave\InfectionStaticAnalysis\Application\{closure}() at /projects/open-source/yiisoft/html/vendor/roave/infection-static-analysis-plugin/bin/roave-infection-static-analysis-plugin:87
 include() at /projects/open-source/yiisoft/html/vendor/bin/roave-infection-static-analysis-plugin:120
Ocramius commented 1 year ago

I wonder if this is somehow reproducible in psalm itself 🤔

Do you use any plugins perhaps?

vjik commented 1 year ago

Run psalm (./vendor/bin/psalm) is OK. Run pure infecton (./vendor/bin/infection) is OK.

Other plugins don't use. Run ./vendor/bin/roave-infection-static-analysis-plugin throws error.

Ocramius commented 1 year ago

Right, so the problem seems to be that vimeo/psalm is not yet capable of fully recycling its own "codebase" code without problems.

Do you perhaps have a reference to booleaninputtag? Could be something for which a scenario could be designed, perhaps.

vjik commented 1 year ago

https://github.com/yiisoft/html/blob/master/src/Tag/Base/BooleanInputTag.php

Ocramius commented 1 year ago

Probably caused by https://github.com/Roave/infection-static-analysis-plugin/blob/d03ec89483250e7122d15a7b9c62ccac06f42ec9/src/Roave/InfectionStaticAnalysis/Psalm/RunStaticAnalysisAgainstMutant.php#L56

The problem is that this line is necessary, or else SA results will be based on cached analysis 🤔

I can't help in the next few weeks, but I endorse trying to reduce the code size as much as possible, until we reach just a reproducer, and then we bring the problem up to vimeo/psalm 🤔

vjik commented 10 months ago

Any news?

Ocramius commented 10 months ago

Nope, unlikely able to help myself in any foreseeable future

Jean85 commented 6 months ago

I just installed this package in https://github.com/facile-it/doctrine-mysql-come-back/pull/87 to improve my MSI, and I'm getting the same bug, and it doesn't appear when running Psalm alone.

Is there at least some workaround that I can use? Something like purging the whole Psalm cache at each run, since the codebase is so small and it would still be fast?

Ocramius commented 6 months ago

I and @MatteoLattanzio have worked on this bug this week: the problem is not in in this plugin, but rather in @psalm's architecture, or at least when combined with this tool.

Psalm does (obviously) not keep everything in memory, nor does it scan everything ahead of time: instead, it does some level of lazy loading internally, and that mechanism is not perfect.

The issue occurs when there are lookups pointing at sources that are not part of the current analysis.

For example, we had an issue with enum types and custom @psalm-type and @psalm-import-type referencing them, where psalm would not be capable of doing a lookup of part of the type lookup. We could not simplify it down to a reproducible example, which is why this was not fixed as part of our investigation.

Our workaround has been to remove a @psalm-type.

Another approach I'm thinking of is to not invalidate only one of the sources in this tool, but rather to nuke the entire psalm instance for every mutant, which should be bearable for smaller libraries.