SHolzhauer / elastic-tip

Elastic TIP is a python tool which automates the process of aggregating Threat Intelligence and ingesting the intelligence into a common format into Elasticsearch with the main goal of being used by the Security solution.
GNU General Public License v3.0
27 stars 5 forks source link

Elastic Cloud #3

Closed peasead closed 2 years ago

peasead commented 3 years ago

Is your feature request related to a problem? Please describe. I am trying to use this to upload to Elastic Cloud, when I try to run this, I get the following error. I've tried with https and without, with and without --tls. I am not sure if the Elastic Cloud port is related to the issue.

python3 tip/elastic_tip_cli.py run -e https://[elastic-cloud-endpoint]:9243 -u [user] -p [password] -m * --tls

Describe the solution you'd like I would like to be able to use this in Elastic Cloud.

Additional context

Module LICENSE.md does not exist
<Elasticsearch([{'host': 'https://[elastic-cloud-endpoint]:9243', 'use_ssl': True}])>
<Elasticsearch([{'host': 'https://[elastic-cloud-endpoint]:9243', 'use_ssl': True}])>
Verifying TIP
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 159, in _new_conn
    conn = connection.create_connection(
  File "/usr/local/lib/python3.8/site-packages/urllib3/util/connection.py", line 61, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/usr/local/Cellar/python@3.8/3.8.6_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/socket.py", line 918, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 8] nodename nor servname provided, or not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/elasticsearch/connection/http_urllib3.py", line 241, in perform_request
    response = self.pool.urlopen(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 724, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/site-packages/urllib3/util/retry.py", line 379, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/site-packages/urllib3/packages/six.py", line 735, in reraise
    raise value
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 976, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 308, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 171, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x1084d3a30>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "tip/elastic_tip_cli.py", line 185, in <module>
    tip_cli.cli()
  File "tip/elastic_tip_cli.py", line 27, in cli
    self._run_cli()
  File "tip/elastic_tip_cli.py", line 93, in _run_cli
    self._tip.run()
  File "/Users/[user]/Documents/elastic-tip/tip/elastic_tip.py", line 72, in run
    self.verify_tip()
  File "/Users/[user]/Documents/elastic-tip/tip/elastic_tip.py", line 106, in verify_tip
    if self._es.indices.exists(index=self.index):
  File "/usr/local/lib/python3.8/site-packages/elasticsearch/client/utils.py", line 84, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/elasticsearch/client/indices.py", line 294, in exists
    return self.transport.perform_request("HEAD", _make_path(index), params=params)
  File "/usr/local/lib/python3.8/site-packages/elasticsearch/transport.py", line 351, in perform_request
    status, headers_response, data = connection.perform_request(
  File "/usr/local/lib/python3.8/site-packages/elasticsearch/connection/http_urllib3.py", line 254, in perform_request
    raise ConnectionError("N/A", str(e), e)
elasticsearch.exceptions.ConnectionError: ConnectionError(<urllib3.connection.HTTPSConnection object at 0x1084d3a30>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known) caused by: NewConnectionError(<urllib3.connection.HTTPSConnection object at 0x1084d3a30>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known)
SHolzhauer commented 3 years ago

@peasead I haven't used elastic cloud before, any specifics I should keep in mind or would help before delving in?

peasead commented 3 years ago

It does use a different default port than locally; so that may be causing the issue. Maybe an argument that would allow to set a port, but default 9200?

Beyond that, it's an Elasticsearch endpoint with TLS and username/passphrase.

SHolzhauer commented 3 years ago

@peasead you can now specify the port to connect to, hope this works!

peasead commented 3 years ago

Wow...sorry this is so late. Hope its still relevant: 9243.

peasead commented 2 years ago

Merged.