0.12.0 fails to verify the certificate of GOCDB API endpoint

[orviz]

Short Description of the issue

While validating 0.12.0 release for CMD, the provider execution failed to verify GOCDB endpoint when --insecure option is not used (see detailed deployment in [1]).

I could not find out a way to pass the CA path with the certificates to trust.

[1] https://jenkins.egi.ifca.es/job/QualityCriteriaValidation/job/cloud-info-provider/93

Environment

• Operating System: CentOS7
• python version: Python 2.7.5
• OpenStack version: Queens

Steps to reproduce

Executing the provider without the insecure flag. Tried with no luck to use --ca-cert option pointing to /etc/grid-security/certificates.

Logs, stacktrace, or other symptoms

Traceback (most recent call last):
  File "/usr/bin/cloud-info-provider-service", line 10, in <module>
    sys.exit(main())
  File "/usr/lib/python2.7/site-packages/cloud_info_provider/core.py", line 144, in main
    output = mgr.driver.format(opts, providers, auth_refresher)
  File "/usr/lib/python2.7/site-packages/cloud_info_provider/formatters/base.py", line 51, in format
    info = available_collectors[tpl].fetch()
  File "/usr/lib/python2.7/site-packages/cloud_info_provider/collectors/compute.py", line 27, in fetch
    **kwargs)
  File "/usr/lib/python2.7/site-packages/cloud_info_provider/collectors/base.py", line 25, in _get_info_from_providers
    result = getattr(i, method)(**provider_kwargs)
  File "/usr/lib/python2.7/site-packages/cloud_info_provider/providers/openstack.py", line 30, in inner
    return f(self, **kwargs)
  File "/usr/lib/python2.7/site-packages/cloud_info_provider/providers/openstack.py", line 192, in get_compute_endpoints
    ret.update(self.get_goc_info(e_id_url, self.insecure))
  File "/usr/lib/python2.7/site-packages/cloud_info_provider/providers/base.py", line 29, in get_goc_info
    insecure)
  File "/usr/lib/python2.7/site-packages/cloud_info_provider/providers/gocdb.py", line 59, in find_in_gocdb
    verify=verify)
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 518, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 639, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 512, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: ("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",)

Summary of proposed changes

The verify = not insecure line only allows boolean values to be passed to the requests.get method:

https://github.com/EGI-Foundation/cloud-info-provider/blob/5c97e11716f6c9195a034c3fc6f438dd5bce84b5/cloud_info_provider/providers/gocdb.py#L55-L59

The code should allow to set verify as a valid path for the CAs.

[enolfc]

We simplified the CA management to just rely on requests and system defaults in 0.11.4 (before for GOCDB the CA path was hardcoded). This change implies that IGTF CAs should be included in the requests bundle. If we really want to allow CAs configuration we should review all code and not just the gocdb.py module.

Not sure how to better approach this :(

[gwarf]

This is documented on https://github.com/EGI-Foundation/cloud-info-provider#cas For me it's fine to leave it like this, but maybe you have another alternative to propose @orviz ?

[gwarf]

But we could also depend on the CA RPM/deb metapackages from UMD and use/hardcode the related specific path.

[orviz]

The current approach is the less intrusive, but in practice (since it is primarily used in EGI Fedcloud) having it hardcoded would prevent from recurring questions from users that do not read the documentation (just like me :P).

If you ask me I prefer the current one, I can update the ansible role accordingly, and the release notes should state this clearly.

[gwarf]

So it's part of a previous release: https://github.com/EGI-Foundation/cloud-info-provider/releases/tag/0.11.5. We can clarify those release notes, but not sure we should include it in the current/new one.

So do we all agree that we keep things like this in this repo and that you update the ansible role?

[gwarf]

I've updated https://github.com/EGI-Foundation/cloud-info-provider/releases/tag/0.11.5

[orviz]

thanks @gwarf the ansible role tackles now these steps.

The last CMD validation job has been successfully executed so we can close the issue.

[gwarf]

Thanks!