Documentation suggestion: multiple MegaLinter runs for blocking and non-blocking scans

[wesley-dean-flexion]

Is your feature request related to a problem? Please describe.

The "problem" we're encountering with getting teams to adopt MegaLinter is folks fearing that they don't want linter warnings to block their development progress.

Describe the solution you'd like

We're using two MegaLinter configuration files: one is for stuff we want to block deployment (e.g., security-related) while the other is for linters.

So, it may be nice if we documented how one could set multiple runs (e.g., GitHub Actions), each with different MEGALINTER_CONFIGURATION environment variables set for blocking (e.g., DISABLE_ERRORS set to false for the REPOSITORY_* scanners) and another for non-blocking (e.g., DISABLE_ERRORS set to true, presumably with the desired linters that aren't in the blocking list).

Describe alternatives you've considered

The feature request is for documentation, maybe to the FAQ page. I'm happy to do the work. My "request" here is for feedback on:

1. would you recommend this approach?
2. if so, do you think it should be documented?
3. if so, does the FAQ page seem like the right place for it?

Additional context

I'm not looking to introduce any more work -- I'm looking for some thoughts. If you don't think this is a good use of time, that's fine with me -- no hurt feelings.

[wesley-dean-flexion]

nevermind... I found another approach that'll work. Apologies for wasting your time.

[echoix]

Would you mind sharing the approach here as future reference?

[wesley-dean-flexion]

@echoix Sure! I Read The Funny Manual and noticed two ways to get from here to there:

Enable / Disable linters one at a time

Linters have *_DISABLE_ERRORS configuration options. So, for example, if we didn't want Black or Pylint blocking deployments (i.e., if we opted not to consider Black as a quality gate necessary to advance in the pipeline), we would add the PYTHON_BLACK_DISABLE_ERRORS and PYTHON_PYLINT_DISABLE_ERRORS directives to the .mega-linter.yml file:

---

# fail when finding errors
DISABLE_ERRORS: false

# Don't let Black block deployments
PYTHON_BLACK_DISABLE_ERRORS: true

# Don't let Pylint block deployments either
PYTHON_PYLINT_DISABLE_ERRORS: true

# DO block deployments from Checkov findings
REPOSITORY_CHECKOV_DISABLE_ERRORS: false

The DISABLE_ERRORS and REPOSITORY_CHECKOV_DISABLE_ERRORS lines are unnecessary (they're set to their default values) but makes it clear what's going on.

Disable errors for non-blocking linters in one place

I saw DISABLE_ERRORS_LINTERS accepts a list of linters to enable, but only result in warnings, not errors (failures). So, to replicate the previous example, it would look like:

---

# fail when finding errors
DISABLE_ERRORS: false

# don't throw errors (failures) for the following linters
DISABLE_ERRORS_LINTERS: [PYTHON_BLACK, PYTHON_PYLINT]

# DO block deployments from Checkov findings
REPOSITORY_CHECKOV_DISABLE_ERRORS: false

[wesley-dean-flexion]

In retrospect, the idea of having MegaLinter error / fail in only certain circumstances seems reasonable (so reasonable that there are multiple ways of setting it up already available) and may have some merit to documenting. My approach (multiple configuration files), while plausible, is unnecessary given what's already there. I suppose one could include it in the documentation for the sake of completion, but knowing what I know now, it wouldn't be my first choice.

I am wondering, though, if the documentation (e.g., FAQ) could be expanded upon, possibly with curated user-contributed content. If there's value in that, I'm happy to help with the execution.

(I reopened this to signal the continuation of the discussion; if it's better to have it closed, I'm fine with that, too. Whatever works for everyone else is cool with me.)

[wesley-dean-flexion]

https://github.com/oxsecurity/megalinter/tree/main#activation-and-deactivation

first example

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

If you think this issue should stay open, please remove the O: stale 🤖 label or comment on the issue.

[nvuillam]

I think the DISABLE_ERRORS_LINTERS is clear enough :)

Maybe we should add it as uncommentable property in the default workflow ?