search

hRun / SA-haveibeenpwned

Splunk add-on providing a custom search command to query Troy Hunt's haveibeenpwned API (https://haveibeenpwned.com/api/v3/) for known breaches of your domains or mail addresses.
https://splunkbase.splunk.com/app/5050/
Apache License 2.0
6 stars 4 forks source link

HTTPS request failed: Request-sent #6

Closed ruizmark closed 2 years ago

ruizmark commented 2 years ago

Hello!

I am getting this error when running a query with SA-haveibeenpwned

RuntimeWarning at "/opt/splunk/etc/apps/SA-haveibeenpwned/bin/haveibeenpwned.py", line 194 : HTTPS request failed: Request-sent

Splunk 9.0.0.1 Python 3.7.11

I have confirmed the following

image

I did some research to see if I can fix this but no luck. I have some thoughts thinking it has be an SSL error but I am unsure.

Here are sources I used from Splunk base but I cant seem to fix the issue.

SSL error while trying to connect to splunk web from python in CentOS-7 - https://community.splunk.com/t5/Security/SSL-error-while-trying-to-connect-to-splunk-web-from-python-in/m-p/295077

SSL Certificate issue - https://community.splunk.com/t5/Splunk-Enterprise/SSL-Certificate-issue/m-p/577055

How to get Splunk to run my Python shell script? - https://community.splunk.com/t5/All-Apps-and-Add-ons/How-to-get-Splunk-to-run-my-Python-shell-script/m-p/223138

Also I can curl haveibeenpwned.com from the server CLI with no issues but if I curl with ./splunk cmd I am not getting out to https://haveibeenpwned.com

Curl with ./splunk cmd image

Curl -vk with ./splunk cmd (insecure) image

hRun commented 2 years ago

Hi there!

My first suspicion was something about the SSL certificates as well. That's backed by the curl screenshots, as they show that when ignoring certificates the requests go through. You could rule out certificates as the roo cause by slightly modifying the haveibeenpwned.py script (if possible): Add "import ssl" to the imports and to each call to http_client.HTTPSConnection, add "context=ssl._create_unverified_context()" as the last argument. You won't need to restart Splunk afterwards, just give it a minute and dispatch your search again.

However the actual error message seems to hint at something different and having done some googling, I suspect the error to stem from some concurrency (a request is sent while the previous hasn't been fully handled yet). Not sure why this never occured while testing. The used HTTP library would be to blame in that case which I think I chose over the more popular "requests" library due to recommendation by splunk and to reduce the number of required requests. Not sure if I can reproduce your error, but I'll try to implement a version of the script that uses a new and better HTTP library for you to test. Please just have some patience with me as I'm doing this in my very limited free time.

Cheers, hRun

ruizmark commented 2 years ago

I tried the script with "context=ssl._create_unverified_context()" but still no luck with that. At first my network team wanted me to use our proxy for this but they have allowed me to bypass the proxy. Once I bypassed the proxy the app work perfectly fine!

Thanks for your support hRun!