SSLError - EOF occurred in violation of protocol

[NickS-2022]

Anybody know what could be causing this?

File "/home/azureuser/sentinel/lib/python3.8/site-packages/requests/adapters.py", line 517, in send raise SSLError(e, request=request) requests.exceptions.SSLError: HTTPSConnectionPool(host='sentinelus.azure-api.net', port=443): Max retries exceeded with url: /<-redacted-this-is-my-workspace-id>/threatintelligence:upload-indicators?api-version=2022-07-01 (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1131)')))

The App has the appropriate API permissions and has the Sentinel Contributor role for the LAW. In Azure AD I'm seeing successful authentications to the App. The Threat Intelligence Upload Indicators API is an installed Connector in Sentinel.

In the logs I see the script connecting to MISP ok and downloading indicators ok. I get the same error when using the Read (-r) flag.

[lnfernux]

I'm not 'the guy' for this, but I'll try to help if I can.

Need some more information @NickS-2022:

• Does this happen when running in an Azure Function App or on a virtual machine?
• Are you running the same versions of components using the upload indicators API as with the old graph api version, or have you updated in accordance with requirements.txt? (Just curious because of this https://stackoverflow.com/questions/18578439/using-requests-with-tls-doesnt-give-sni-support/18579484#18579484 post, which notes that some of the requirements we have should be installed/updated).
• What do you mean with "read" flag, the new upload indicators version of this script does not have a read flag that I know of?

Personally I haven't seen the issue before, but for sake of narrowing it down:

• Wrong details for the SPN (app reg) would throw a different error, so likely not that
• Missing permissions on the workspace would throw a missing permissions error, so likely not that
• You can always try to reinstall the solution in Sentinel, but afaik you can push to the api without error even if the solution isn't installed (pretty sure I did this by mistake).
• Since you can download indicators from MISP and the MISP logs aren't throwing any errors (please do check the MISP audit logs for authencations by the misp key and see that it is able to connect without issue), then we can probably rule out MISP as the issue
• Any config-file you're supposed to edit shouldn't do anything with different with the Microsoft API endpoint as far as my knowledge of it goes
• This should leave only configuration errors (missing updates, packages) on the server itself, but having said that, I might be wrong about that :)

[NickS-2022]

Hi,

• This is running on an Ubuntu virtual machine.
• It is the same machine that was running the Graph API version, but I did run the command to install any new dependencies. Just double-check by running again and it says everything is satisfied.
• I wasn't sure if the -r was still supported, but I tried it anyway. I also get the same error if I don't add that flag.
• I can see from the logs that is authenticating ok to the MISP server.

[NickS-2022]

As a check I swapped to using the Graph API (Set Graph_Api to True, swapped to the graph api scope URL and commented out the workspace ID) and the indicators uploaded successfully. So that validates most of the settings. Double-checked that the App has Microsoft Sentinel Contributor on the log analytics workspace and that the correct workspace ID was being used. Authentication to the API appears to be successful. Might just trying starting with a fresh VM.

[cudeso]

Hello,

I had similar problems when the threshold for API requests with the Upload Indicators API was set to high. Can you check that

ms_max_indicators_request = 100     # Upload Indicators API: Throttle max: 100 indicators per request
ms_max_requests_minute = 100        # Upload Indicators API: Throttle max: 100 requests per minute

are indeed set?

[NickS-2022]

Yes those are both set. They were 100 and I've tried taking them down to 10, but no difference.

As a test i deliberately used the wrong App secret and got a different error: "get_access_token" which is as expected.

I then tried changing the workspace ID and still got the "SSLError - Eof" error.

I even tried a different Sentinel workspace, but got the same error.

So I have now rebuilt the server using a newer version of Ubuntu (22.02 rather than 20.02). This time I have a different error and will post that in a different thread. Worth noting this is using Python 3.10 and not Python 3.8.

[NickS-2022]

After rebuilding using 22.02 and Python 3.10 I still have the same error, once I got past another error that seems to have slipped into the latest release (a missing entry in the template config file.

It's a slightly different error trace so I've included it all: Unknown error: the response is not in JSON. Something is broken server-side, please send us everything that follows (careful with the auth key): Request headers: {'User-Agent': 'PyMISP 2.4.173 - Python 3.10', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/json', 'Connection': 'keep-alive', 'Cookie': 'xxxxxxxxxxxxxxxx', 'Content-Length': '367', 'content-type': 'applic ation/json'} Request body: {"returnFormat": "stix2", "page": 5, "limit": 10, "withAttachments": 0, "metadata": 0, "publish_timestamp": "3d", "published": 1, "enforceWarninglist": 0, "to_ids": 1, "includeEventUuid": 0, "includeEventTags": 0, "sgReferenceOnly": 0, "includ eContext": 0, "headerless": 0, "includeSightings": 0, "includeDecayScore": 0, "includeCorrelations": 0, "excludeDecayed": 0} Response (if any): {"name":"An Internal Error Has Occurred.","message":"An Internal Error Has Occurred.","url":"\/events\/restSearch"} ----------------CLEAR existing_indicators_hash--------------------------- Traceback (most recent call last): File "/home/azureuser/sentinel/lib/python3.10/site-packages/urllib3/connectionpool.py", line 467, in _make_request self._validate_conn(conn) File "/home/azureuser/sentinel/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1092, in _validate_conn conn.connect() File "/home/azureuser/sentinel/lib/python3.10/site-packages/urllib3/connection.py", line 642, in connect sock_and_verified = _ssl_wrap_socket_and_match_hostname( File "/home/azureuser/sentinel/lib/python3.10/site-packages/urllib3/connection.py", line 783, in _ssl_wrap_socket_and_match_hostname ssl_sock = ssl_wrapsocket( File "/home/azureuser/sentinel/lib/python3.10/site-packages/urllib3/util/ssl.py", line 469, in ssl_wrap_socket ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls, serverhostname) File "/home/azureuser/sentinel/lib/python3.10/site-packages/urllib3/util/ssl.py", line 513, in _ssl_wrap_socket_impl return ssl_context.wrap_socket(sock, server_hostname=server_hostname) File "/usr/lib/python3.10/ssl.py", line 513, in wrap_socket return self.sslsocket_class._create( File "/usr/lib/python3.10/ssl.py", line 1071, in _create self.do_handshake() File "/usr/lib/python3.10/ssl.py", line 1342, in do_handshake self._sslobj.do_handshake() ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:997)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/azureuser/sentinel/lib/python3.10/site-packages/urllib3/connectionpool.py", line 790, in urlopen response = self._make_request( File "/home/azureuser/sentinel/lib/python3.10/site-packages/urllib3/connectionpool.py", line 491, in _make_request raise new_e urllib3.exceptions.SSLError: TLS/SSL connection has been closed (EOF) (_ssl.c:997)

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/azureuser/sentinel/lib/python3.10/site-packages/requests/adapters.py", line 486, in send resp = conn.urlopen( File "/home/azureuser/sentinel/lib/python3.10/site-packages/urllib3/connectionpool.py", line 844, in urlopen retries = retries.increment( File "/home/azureuser/sentinel/lib/python3.10/site-packages/urllib3/util/retry.py", line 515, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='sentinelus.azure-api.net', port=443): Max retries exceeded with url: /e4d86d10-4ea1-4599-90c2-2ec0b8916ff5/threatintelligence:upload-indicators?api-version=2022-07-01 (Caused by SSLEr ror(SSLZeroReturnError(6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:997)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/azureuser/sentinel/lib/python3.10/site-packages/urllib3/connectionpool.py", line 790, in urlopen response = self._make_request( File "/home/azureuser/sentinel/lib/python3.10/site-packages/urllib3/connectionpool.py", line 491, in _make_request raise new_e urllib3.exceptions.SSLError: TLS/SSL connection has been closed (EOF) (_ssl.c:997)

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/azureuser/sentinel/lib/python3.10/site-packages/requests/adapters.py", line 486, in send resp = conn.urlopen( File "/home/azureuser/sentinel/lib/python3.10/site-packages/urllib3/connectionpool.py", line 844, in urlopen retries = retries.increment( File "/home/azureuser/sentinel/lib/python3.10/site-packages/urllib3/util/retry.py", line 515, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='sentinelus.azure-api.net', port=443): Max retries exceeded with url: /e4d86d10-4ea1-4599-90c2-2ec0b8916ff5/threatintelligence:upload-indicators?api-version=2022-07-01 (Caused by SSLEr ror(SSLZeroReturnError(6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:997)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/azureuser/misp-nda-cc/script.py", line 264, in main() File "/home/azureuser/misp-nda-cc/script.py", line 254, in main request_manager.upload_indicators(parsed_indicators) File "/home/azureuser/misp-nda-cc/RequestManager.py", line 239, in upload_indicators response = requests.post(request_url, headers=self.headers, json=request_body) File "/home/azureuser/sentinel/lib/python3.10/site-packages/requests/api.py", line 115, in post return request("post", url, data=data, json=json, kwargs) File "/home/azureuser/sentinel/lib/python3.10/site-packages/requests/api.py", line 59, in request return session.request(method=method, url=url, kwargs) File "/home/azureuser/sentinel/lib/python3.10/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, send_kwargs) File "/home/azureuser/sentinel/lib/python3.10/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, kwargs) File "/home/azureuser/sentinel/lib/python3.10/site-packages/requests/adapters.py", line 517, in send raise SSLError(e, request=request) requests.exceptions.SSLError: HTTPSConnectionPool(host='sentinelus.azure-api.net', port=443): Max retries exceeded with url: /xxxxxxxxxxxxxxxxxxxxxxxxxxxx/threatintelligence:upload-indicators?api-version=2022-07-01 (Caused by SSLError( SSLZeroReturnError(6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:997)')))

[cudeso]

Hello. The line of code ----------------CLEAR existing_indicators_hash--------------------------- does not exist in the Upload Indicators branch > https://github.com/cudeso/misp2sentinel/blob/upload_indicators_api/RequestManager.py It does exist in the main branch though. I think you're branch is not set to "upload_indicators_api".

Check with git branch. Switch branches with git checkout upload_indicators_api

[NickS-2022]

So this error message was caused by the Azure Firewall blocking access to the sentinelus.azure-api.net. I had checked this, but the function we use to review blocked connections was not picking these. Probably MS changing the log data resulting in a broken function. It was only when I actually checked the firewall rules that I spotted the URL was not listed.