search

microsoft / eslint-plugin-sdl

ESLint Plugin focused on common security issues and misconfigurations.
MIT License
38 stars 27 forks source link

Don't continue on error during E2E test #67

Closed JesseParsons closed 1 month ago

mkacmar commented 1 month ago

Hi Jesse, I think that continuing on error may actually be intentional here - if ESLint ends up returning a non-zero exit code (which is kind of valid and is happening right now) due to rule violations (and misconfiguration as one can see in generated SARIF) we still want to get access to SARIF reports in the next step for further analysis. This may not be the best way to test the plugin E2E going forward since it can hide real issues, but I don't think we can just flip the flag right now.

JesseParsons commented 1 month ago

Hi Jesse, I think that continuing on error may actually be intentional here - if ESLint ends up returning a non-zero exit code (which is kind of valid and is happening right now) due to rule violations (and misconfiguration as one can see in generated SARIF) we still want to get access to SARIF reports in the next step for further analysis. This may not be the best way to test the plugin E2E going forward since it can hide real issues, but I don't think we can just flip the flag right now.

Fair enough, but I think we should consider running it without the --output-file flag or somehow being able to see the results without having to look at the SARIF. Otherwise, this step is not really a gate if ESLint and SARIF upload steps succeed.