Closed lavcraft closed 5 years ago
Infra-Red
commented
5 years ago Hi @lavcraft ! Could you please share the versions of logsearch and logsearch-for-cloudfoundry releases that you have deployed? For me it looks like inconsistency between release versions.
lavcraft
commented
5 years ago Hi @Infra-Red. Sure, I use the latest versions
releases:
- name: "logsearch-for-cloudfoundry"
version: "210.2.0"
url: "https://bosh.io/d/github.com/cloudfoundry-community/logsearch-for-cloudfoundry?v=210.2.0"
sha1: "f26ed0a65b7eda4e52f96520003fb4838f218885"
- name: "logsearch"
version: "210.2.0"
url: "https://bosh.io/d/github.com/cloudfoundry-community/logsearch-boshrelease?v=210.2.0"
sha1: "f23c694271eb3d7a357fc826511195fb4a04d2b8"With next stemcell bosh-vsphere-esxi-ubuntu-xenial-go_agent 97.3* ubuntu-xenial
Infra-Red
commented
5 years ago @lavcraft Looks fine. Can you ssh to the maintenance VM and check that python3 package is in correct state:
bosh -d logsearch ssh maintenance
sudo -i
ls /var/vcap/packages/python3/lib/python3.6/site-packages/You should see the the requests module files in this folder.
lavcraft
commented
5 years ago Yes, i found requests package
maintenance/285bc4e1-c332-45a0-ba5d-33e44f208b9f:~$ ls /var/vcap/packages/python3/lib/python3.6/site-packages/
README.txt __pycache__ certifi certifi-2019.3.9.dist-info chardet chardet-3.0.4.dist-info easy_install.py idna idna-2.8.dist-info pip pip-9.0.1.dist-info pkg_resources requests requests-2.21.0.dist-info setuptools setuptools-28.8.0.dist-info urllib3 urllib3-1.24.2.dist-infoBut, i see only ruby package declaration in upload-kibana-objects job spec. Not requests Errand upload-kibana-objects run on separate VM. Not in mainatance..
axelaris
commented
5 years ago It seems that you're using outdated version of logsearch-deployment.yml file. In latest versions all errands are colocated with maintenance job.
lavcraft
commented
5 years ago @axelaris i think about it, but upload-kibana-objects errand locate in logsearch-for-cloudfoundry. I use logsearch-deployment.yml with modification for logsearch-for-cloudfoundry release. Where i can find example deploymeny.yml with logsearch-for-cloudfoundry ? I cant find templates from deployment docs templates/stub.logsearch-for-cf.standalone-kibana-with-auth.yml
Infra-Red
commented
5 years ago @lavcraft You should use the following ops file for logsearch-for-cloudfoundry extension: https://github.com/cloudfoundry-community/logsearch-boshrelease/blob/v210.2.0/deployment/operations/cloudfoundry.yml
lavcraft
commented
5 years ago @Infra-Red many thanks! Looks fine
lavcraft
commented
5 years ago Okay, it works. But the next problem was appear: How to disable SSL certificate verification during kibana objects uploading? Because we are using self-signed certificate inside VPN subnet.
Stderr Traceback (most recent call last):
File "/var/vcap/packages/python3/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
chunked=chunked)
File "/var/vcap/packages/python3/lib/python3.6/site-packages/urllib3/connectionpool.py", line 343, in _make_request
self._validate_conn(conn)
File "/var/vcap/packages/python3/lib/python3.6/site-packages/urllib3/connectionpool.py", line 839, in _validate_conn
conn.connect()
File "/var/vcap/packages/python3/lib/python3.6/site-packages/urllib3/connection.py", line 344, in connect
ssl_context=context)
File "/var/vcap/packages/python3/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 347, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/var/vcap/packages/python3/lib/python3.6/ssl.py", line 401, in wrap_socket
_context=self, _session=session)
File "/var/vcap/packages/python3/lib/python3.6/ssl.py", line 808, in __init__
self.do_handshake()
File "/var/vcap/packages/python3/lib/python3.6/ssl.py", line 1061, in do_handshake
self._sslobj.do_handshake()
File "/var/vcap/packages/python3/lib/python3.6/ssl.py", line 683, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:748)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/vcap/packages/python3/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/var/vcap/packages/python3/lib/python3.6/site-packages/urllib3/connectionpool.py", line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "/var/vcap/packages/python3/lib/python3.6/site-packages/urllib3/util/retry.py", line 399, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='logs.$MY_SYSTEM_DOMAIN, port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:748)'),))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/vcap/jobs/upload-kibana-objects/bin/import-objects", line 13, in <module>
logs = session.get('https://logs.{}'.format(os.environ['CF_SYSTEM_DOMAIN']))
File "/var/vcap/packages/python3/lib/python3.6/site-packages/requests/sessions.py", line 546, in get
return self.request('GET', url, **kwargs)
File "/var/vcap/packages/python3/lib/python3.6/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/var/vcap/packages/python3/lib/python3.6/site-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/var/vcap/packages/python3/lib/python3.6/site-packages/requests/adapters.py", line 514, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='logs.$MY_SYSTEM_DOMAIN', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:748)'),))
1 errand(s)@Infra-Red can you recommend sth to resolve this problem?
Infra-Red
commented
5 years ago Hi @lavcraft ! I've added option to skip ssl validation in the following commit: https://github.com/cloudfoundry-community/logsearch-for-cloudfoundry/commit/0fd508a506d01ced5c84ac89b96ed444f949b9b7.
lavcraft
commented
5 years ago @Infra-Red very appreciated! But, verify flag really doesn't required in post/put https://github.com/cloudfoundry-community/logsearch-for-cloudfoundry/blob/c4f9843a4de26820647ae78069c4905f3aa7af71/jobs/upload-kibana-objects/templates/bin/import-objects#L39,L45 ?
lavcraft
commented
5 years ago @Infra-Red thanks! I am waiting for release with 0fd508a 👍
axelaris
commented
5 years ago Released
Hi. I am trying to run upload-kibana-objects errand. But it was fail with next error: