Improve logging visibility

[victoria-miltcheva]

What does this PR do?

• Sets the default logging level to WARNING instead of ERROR. This way, warnings will be displayed back to the user by default, which helps with troubleshooting issues. They will no longer need to provide the --verbose flag to be able to see warnings anymore.
• Improves the warning message that's logged when a file cannot be loaded (detect_secrets/core/secrets_collection.py)
• Fixes a unit test associated with the above logging change
• Updates the .secrets.baseline file

How to test

1. Create a test.py file in the root detect-secrets directory:

import detect_secrets.main

# Run a scan - default
detect_secrets.main.main(
    [
        "scan",
        "--update",
        ".secrets.baseline",
    ]
)

# Run a scan - verbose
# detect_secrets.main.main(
#     [
#         "--verbose",
#         "scan",
#         "--update",
#         ".secrets.baseline",
#     ]
# )

2. Add the following lines here

    log.warning('Should see warning by default.')
    log.error('Should see error by default.')
    log.debug('Should see debug only with verbose.')

3. Run python test.py to test which logs show up without the --verbose option. Result:

➜  detect-secrets git:(improve-logging-visibility) ✗ python test.py
Should see warning by default.
Should see error by default.

4. Comment out lines 4-10 in test.py and uncomment lines 13-20 to test the --verbose option. Result:

➜  detect-secrets git:(improve-logging-visibility) ✗ python test.py
Should see warning by default.
Should see error by default.
Should see debug only with verbose.
Checking file: .coveragerc
Checking file: .dockerignore
Checking file: .editorconfig
Checking file: .github/ISSUE_TEMPLATE/bug-report.md
Checking file: .github/ISSUE_TEMPLATE/engineering-work-item-template.md
Checking file: .github/workflows/codeql-analysis.yml
Checking file: .gitignore
Checking file: .pre-commit-config.yaml
Checking file: .pre-commit-hooks.yaml
Checking file: .travis.yml
Checking file: CHANGELOG.md
Checking file: CONTRIBUTING.md
Checking file: Dockerfiles/base/00.python-redhat-ubi.Dockerfile
Checking file: Dockerfiles/base/00.python.Dockerfile
Checking file: Dockerfiles/base/01.cli.Dockerfile
Checking file: Dockerfiles/detect-secrets/01.detect-secrets-redhat-ubi.Dockerfile
Checking file: Dockerfiles/detect-secrets/02.detect-secrets-redhat-ubi-custom.Dockerfile
Checking file: Dockerfiles/detect-secrets/02.detect-secrets.Dockerfile
Checking file: Dockerfiles/hook/02.detect-secrets-hook.Dockerfile
Checking file: LICENSE
Checking file: Makefile
Checking file: Makefile.ibm
Checking file: README.md
Checking file: detect_secrets/__init__.py
Checking file: detect_secrets/constants.py
Checking file: detect_secrets/core/__init__.py
Checking file: detect_secrets/core/audit.py
Checking file: detect_secrets/core/baseline.py
Checking file: detect_secrets/core/bidirectional_iterator.py
Checking file: detect_secrets/core/code_snippet.py
Checking file: detect_secrets/core/color.py
Checking file: detect_secrets/core/common.py
Checking file: detect_secrets/core/constants.py
Checking file: detect_secrets/core/log.py
Checking file: detect_secrets/core/potential_secret.py
Checking file: detect_secrets/core/report/__init__.py
Checking file: detect_secrets/core/report/conditions.py
Checking file: detect_secrets/core/report/constants.py
Checking file: detect_secrets/core/report/output.py
Checking file: detect_secrets/core/report/report.py
Checking file: detect_secrets/core/secrets_collection.py
Checking file: detect_secrets/core/usage.py
Checking file: detect_secrets/main.py
Checking file: detect_secrets/plugins/__init__.py
Checking file: detect_secrets/plugins/artifactory.py
Checking file: detect_secrets/plugins/aws.py
Checking file: detect_secrets/plugins/azure_storage_key.py
Checking file: detect_secrets/plugins/base.py
Checking file: detect_secrets/plugins/basic_auth.py
Checking file: detect_secrets/plugins/box.py
Checking file: detect_secrets/plugins/cloudant.py
Checking file: detect_secrets/plugins/common/__init__.py
Checking file: detect_secrets/plugins/common/constants.py
Checking file: detect_secrets/plugins/common/filetype.py
Checking file: detect_secrets/plugins/common/filters.py
Checking file: detect_secrets/plugins/common/ini_file_parser.py
Checking file: detect_secrets/plugins/common/initialize.py
Checking file: detect_secrets/plugins/common/util.py
Checking file: detect_secrets/plugins/common/yaml_file_parser.py
Checking file: detect_secrets/plugins/db2.py
Checking file: detect_secrets/plugins/github_enterprise.py
Checking file: detect_secrets/plugins/github_token.py
Checking file: detect_secrets/plugins/high_entropy_strings.py
Checking file: detect_secrets/plugins/ibm_cloud_iam.py
Checking file: detect_secrets/plugins/ibm_cos_hmac.py
Checking file: detect_secrets/plugins/jwt.py
Checking file: detect_secrets/plugins/keyword.py
Checking file: detect_secrets/plugins/mailchimp.py
Checking file: detect_secrets/plugins/npm.py
Checking file: detect_secrets/plugins/private_key.py
Checking file: detect_secrets/plugins/slack.py
Checking file: detect_secrets/plugins/softlayer.py
Checking file: detect_secrets/plugins/square_oauth.py
Checking file: detect_secrets/plugins/stripe.py
Checking file: detect_secrets/plugins/twilio.py
Checking file: detect_secrets/pre_commit_hook.py
Checking file: detect_secrets/util.py
Checking file: docs/audit.md
Checking file: docs/cheat-sheet.md
Checking file: docs/developer-tool-faq.md
Checking file: docs/scan.md
Checking file: lgtm.yml
Checking file: prepare-upstream-contribution.sh
Checking file: renovate.json
Checking file: requirements-dev.txt
Checking file: scripts/benchmark.py
Checking file: scripts/bumpity.py
Checking file: scripts/gen-pipfile.sh
Checking file: scripts/run-in-pipeline.sh
Checking file: scripts/run_performance_tests.py
Checking file: scripts/version_greater_than_in_cos.py
Checking file: setup.cfg
Checking file: setup.py
Checking file: testing/__init__.py
Checking file: testing/baseline.py
Checking file: testing/factories.py
Checking file: testing/mocks.py
Checking file: testing/util.py
Checking file: tox.ini
Checking file: update.md
Checking file: user-config/.pre-commit-config.yaml
Checking file: user-config/detect-secrets.psm1

[victoria-miltcheva]

I need to do some additional testing on Monday to see if this will be logged each time a scan is run when a plugin is disabled: https://github.com/IBM/detect-secrets/blob/improve-logging-visibility/detect_secrets/plugins/common/initialize.py#L140-L143

If so, that would be really noisy and should only show up in verbose mode...

[victoria-miltcheva]

Note about https://github.com/IBM/detect-secrets/pull/120/commits/ef5796829cd57e0e69564cdbc89b4b80c97d03a1:

I noticed that when a --no-<insert-plugin-name>-scan option is included in the scan arguments to initailize a new baseline file, a certain message always shows up.

For example if you have this in test.py:

detect_secrets.main.main(
    [
        "scan",
        "--update",
        ".secrets.baseline",
        "--no-keyword-scan"
    ]
)

You will see the following log when running a scan to generate a new baseline file:

--keyword-exclude specified, but KeywordDetector not configured! Ignoring..

To prevent this from being logged in each scan, I changed its log level to debug: https://github.com/IBM/detect-secrets/pull/120/files#diff-7f359002f472ddc72105e7e538bae0ccbb3e0e6e7034d821c05b7c34fef0e52aR140