Multiple baselines (Taint + Static) overwrite the psalm.xml

vimeo / psalm

A static analysis tool for finding errors in PHP applications

https://psalm.dev

MIT License

5.54k stars 659 forks source link

Multiple baselines (Taint + Static) overwrite the psalm.xml #4810

Open jarstelfox opened 3 years ago

jarstelfox commented 3 years ago

Steps:

Create a psalm.xml with psalm-baseline.xml
Ensure the psalm.xml has errorBaseline="psalm-baseline.xml"
Run Taint Analysis with a separate baseline, psalm-taint-baseline.xml: psalm --taint-analysis --set-baseline=psalm-taint-baseline.xml

Actual:

psalm.xml now has errorBaseline="psalm-taint-baseline.xml"

Expected:

psalm.xml now stays the same. i.e. errorBaseline="psalm-baseline.xml"

Note:

I have tried --use-baseline=psalm-taint-baseline.xml --update-baseline as well as --use-baseline=psalm-taint-baseline.xml --set-baseline=psalm-taint-baseline.xml which both cause the same issue as above.

Psalm Version: 4.3.1

psalm-github-bot[bot] commented 3 years ago

Hey @jarstelfox, can you reproduce the issue on https://psalm.dev ?

ArtemGoutsoul commented 3 years ago

Suggestion: a new psalm config option taintBaseline ? This way all the other config options esp. directory includes / excludes can be shared between both analyses.

ArtemGoutsoul commented 3 years ago

I have psalm_baseline.xml for static code analysis in psalm.xml and I have psalm_taint_baseline.xml for taint analysis.

When I run:

 psalm --use-baseline=psalm_taint_baseline.xml --update-baseline --taint-analysis

the baseline from psalm.xml (psalm_baseline.xml) is going to be updated, instead of psalm_taint_baseline.xml.

So not only config is overwritten when using --set-baseline, but also the actual wrong baseline file is overwritten when using --update-baseline.

leighman commented 3 years ago

We've also had issues getting two baselines to co-exist in the project. It also seems that maybe cli flags --taint-analysis --use-baseline=./taint-baseline.xml get ignored in favour of the baseline in the psalm.xml with the result that all taint errors are reported.