Add static security scanning to CCO CI/CD pipeline

[woodt]

Add bandit scanning to:

[X ] ckan-cloud-provisioning-api [X] ckan-cloud-operator

[X] Confirm pipelines fail when high risk issues are detected.

[woodt]

Working on bandit integration for:

[ ] ckan-cloud-provisioning-api [ ] ckan-cloud-operator

ckan-cloud-provisioning-api seems pretty clean, but a fair number of issues in ckan-cloud-operator. Will create remediation tickets.

May need some assistance on Travis and/or repo access.

[woodt]

Splitting off the ckan-cloud-provisioning-ui work into a separate ticket (Javascript work is quite different than the Python scanning for the other two repos.) See https://github.com/GSA/datagov-ckan-multi/issues/186

[woodt]

Hi @akariv ... can you add me as a contributor to ckan-cloud-operator and ckan-cloud-provisioning-API?

[woodt]

I have access to the required repos. Am working on the travis pipeline updates.

[woodt]

Have packaged up the bandit scanner in a CivicActions "drydock" compatible Docker image, drydockcloud/ci-bandit. Updating the Travis configuration to use this scanner.

[adborden]

https://github.com/drydockcloud/ci-bandit

On Mon, Dec 9, 2019 at 6:59 AM Tom Wood notifications@github.com wrote:

Have packaged up the bandit scanner in a CivicActions "drydock" compatible Docker image, drydockcloud/ci-bandit. Updating the Travis configuration to use this scanner.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/GSA/datagov-ckan-multi/issues/165?email_source=notifications&email_token=AAD4OB26TQMLDOSA3LDDIL3QXZMN3A5CNFSM4JB4BTTKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGJO4TY#issuecomment-563277391, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAD4OB3HB3G53JW227SWC53QXZMN3ANCNFSM4JB4BTTA .

-- Aaron D Borden Lead Engineer | IT Specialist TTS | Data.gov https://www.data.gov

[woodt]

Currently stuck testing this under travis, as the master branch is failing CI.

Step 18/22 : RUN bash -c "source miniconda/etc/profile.d/conda.sh && conda activate ckan-cloud-operator &&             cd ckan-cloud-operator && python3 -m pip install -e ."
 ---> Running in d342b5603abc
Obtaining file:///home/jenkins/ckan-cloud-operator
ERROR: Exception:
Traceback (most recent call last):
  File "/home/jenkins/miniconda/envs/ckan-cloud-operator/lib/python3.7/site-packages/pip/_internal/req/req_install.py", line 441, in check_if_exists
    self.satisfied_by = pkg_resources.get_distribution(str(no_marker))
  File "/home/jenkins/miniconda/envs/ckan-cloud-operator/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 481, in get_distribution
    dist = get_provider(dist)
  File "/home/jenkins/miniconda/envs/ckan-cloud-operator/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 357, in get_provider
    return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0]
  File "/home/jenkins/miniconda/envs/ckan-cloud-operator/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 900, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/home/jenkins/miniconda/envs/ckan-cloud-operator/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 791, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pip._vendor.pkg_resources.ContextualVersionConflict: (python-dateutil 2.8.1 (/home/jenkins/miniconda/envs/ckan-cloud-operator/lib/python3.7/site-packages), Requirement.parse('python-dateutil<2.8.1'), {'ckan-cloud-operator'})

[woodt]

So, although the problem above seems to be failing in the master branch (and there may be other issues), it looks like the security scanning is working in Travis!

[woodt]

Pull requests in https://github.com/ViderumGlobal/ckan-cloud-provisioning-api/pull/8 and https://github.com/datopian/ckan-cloud-operator/pull/96, ready to be reviewed/merged.

[woodt]

@akariv assigned to you to take a peek at the pull requests above.

[akariv]

Done.