gbossert
commented
2 years ago Find below the steps I executed to check the proxy support in SEKOIA.IO App for Splunk works as expected.
1) I created a squid server that support Transparant HTTPs proxying. 2) with the browser I checked the squid works as expected 3) I !! switched off the docker of the proxy !! 4) I downloaded and installed a fresh Splunk Enterprise 8.2.6 instance 5) through the marketplace, I searched than installed the SEKOIA.IO app
6) I configured the app with the following info
- my SEKOIA.IO API key
- proxy url: http://127.0.0.1:3128
- ipv4 lookup: search query:
index=*and Field:ipv4 - clicked on save and got redirected to the SEKOIA.IO Dashboard
7) The dashboard shows no indicator is downloaded (remember I switch off the proxy)
8) in the
var/log/splunk/splunkd.logI see numerous error raised by the python scripts that fails to connect to the proxy server:
04-22-2022 15:47:25.824 +0200 INFO ExecProcessor [33826 ExecProcessor] - message from "/home/georges/Applications/splunk/bin/python3.7 /home/georges/Applications/splunk/etc/apps/sekoia.io/bin/sekoia_indicators.py" Getting new events with the SEKOIA.IO modular input
04-22-2022 15:47:25.824 +0200 INFO ExecProcessor [33826 ExecProcessor] - message from "/home/georges/Applications/splunk/bin/python3.7 /home/georges/Applications/splunk/etc/apps/sekoia.io/bin/sekoia_indicators.py" Fetch the indicators for input sekoia_indicators://feed
04-22-2022 15:47:25.832 +0200 INFO ExecProcessor [33826 ExecProcessor] - message from "/home/georges/Applications/splunk/bin/python3.7 /home/georges/Applications/splunk/etc/apps/sekoia.io/bin/sekoia_indicators.py" Fetch indicators from feed_id=d6092c37-d8d7-45c3-8aff-c4dc26030608
04-22-2022 15:47:25.835 +0200 ERROR ExecProcessor [33826 ExecProcessor] - message from "/home/georges/Applications/splunk/bin/python3.7 /home/georges/Applications/splunk/etc/apps/sekoia.io/bin/sekoia_indicators.py" Traceback (most recent call last):
04-22-2022 15:47:25.835 +0200 ERROR ExecProcessor [33826 ExecProcessor] - message from "/home/georges/Applications/splunk/bin/python3.7 /home/georges/Applications/splunk/etc/apps/sekoia.io/bin/sekoia_indicators.py" File "/home/georges/Applications/splunk/etc/apps/sekoia.io/bin/../lib/py3/urllib3/connection.py", line 170, in _new_conn
04-22-2022 15:47:25.835 +0200 ERROR ExecProcessor [33826 ExecProcessor] - message from "/home/georges/Applications/splunk/bin/python3.7 /home/georges/Applications/splunk/etc/apps/sekoia.io/bin/sekoia_indicators.py" (self._dns_host, self.port), self.timeout, **extra_kw
04-22-2022 15:47:25.835 +0200 ERROR ExecProcessor [33826 ExecProcessor] - message from "/home/georges/Applications/splunk/bin/python3.7 /home/georges/Applications/splunk/etc/apps/sekoia.io/bin/sekoia_indicators.py" File "/home/georges/Applications/splunk/etc/apps/sekoia.io/bin/../lib/py3/urllib3/util/connection.py", line 96, in create_connection
04-22-2022 15:47:25.835 +0200 ERROR ExecProcessor [33826 ExecProcessor] - message from "/home/georges/Applications/splunk/bin/python3.7 /home/georges/Applications/splunk/etc/apps/sekoia.io/bin/sekoia_indicators.py" raise err
04-22-2022 15:47:25.835 +0200 ERROR ExecProcessor [33826 ExecProcessor] - message from "/home/georges/Applications/splunk/bin/python3.7 /home/georges/Applications/splunk/etc/apps/sekoia.io/bin/sekoia_indicators.py" File "/home/georges/Applications/splunk/etc/apps/sekoia.io/bin/../lib/py3/urllib3/util/connection.py", line 86, in create_connection
04-22-2022 15:47:25.835 +0200 ERROR ExecProcessor [33826 ExecProcessor] - message from "/home/georges/Applications/splunk/bin/python3.7 /home/georges/Applications/splunk/etc/apps/sekoia.io/bin/sekoia_indicators.py" sock.connect(sa)
04-22-2022 15:47:25.835 +0200 ERROR ExecProcessor [33826 ExecProcessor] - message from "/home/georges/Applications/splunk/bin/python3.7 /home/georges/Applications/splunk/etc/apps/sekoia.io/bin/sekoia_indicators.py" ConnectionRefusedError: [Errno 111] Connection refused
04-22-2022 15:47:25.835 +0200 ERROR ExecProcessor [33826 ExecProcessor] - message from "/home/georges/Applications/splunk/bin/python3.7 /home/georges/Applications/splunk/etc/apps/sekoia.io/bin/sekoia_indicators.py"
04-22-2022 15:47:25.835 +0200 ERROR ExecProcessor [33826 ExecProcessor] - message from "/home/georges/Applications/splunk/bin/python3.7 /home/georges/Applications/splunk/etc/apps/sekoia.io/bin/sekoia_indicators.py" During handling of the above exception, another exception occurred:
[...]
04-22-2022 15:47:25.836 +0200 ERROR ExecProcessor [33826 ExecProcessor] - message from "/home/georges/Applications/splunk/bin/python3.7 /home/georges/Applications/splunk/etc/apps/sekoia.io/bin/sekoia_indicators.py" requests.exceptions.ProxyError: HTTPSConnectionPool(host='api.sekoia.io', port=443): Max retries exceeded with url: /v2/inthreat/collections/d6092c37-d8d7-45c3-8aff-c4dc26030608/objects?match%5Btype%5D=indicator&limit=300 (Caused by ProxyError('Cannot connect to proxy.', NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f5c8304d210>: Failed to establish a new connection: [Errno 111] Connection refused')))
9) I restart my squid docker
10) ProxyError in the splunk log stops and instead numerous logs show indicators are downloaded (for example, messages with message "WARNING Unsupported path …" shows the connection was succesfull
11) Almost 10min after the end of log errors, logs show "Saved KVStore Batch of" messages and the first indicators start to appear on the dashboard
As reported by a user, the setup page fails to conclude its process when a proxy is required for Internet Access.
Splunk Enterprise Version: 8.2.6 SEKOIA.IO for Splunk Version: 1.1.2