Bug: removing the content of the ca certs setting for an analyzer breaks HTTPS connections

Request Type

Bug

Work Environment

Question	Answer
OS version (server)	Ubuntu 20.04 Server
OS version (client)	Ubuntu 20.04 Client
Cortex version / git hash	3.1.6-1
Package Type	Binary
Browser type & version	n/a

Problem Description

If one sets the content of the CA Certs to a non-empty value and then removes it later on, said analyzer will no longer work if it depends on verified HTTPS connections. Specifically, this is an issue with the Mnemonic PDNS analyzer but also affects any other analyzer which uses Python requests.

Steps to Reproduce

set a CA Certs settings for any analyzer (or global)
run analysis which requires HTTPS connections to work -> all fine
remove the content of the CA Certs setting so that it is empty
any subsequent analysis which requires HTTPS will now fail because certificates cannot be verified by Python's request library.

Possible Solutions

Do not write the content of the ca certs settings to disk if it's empty in the database (aka unset):

diff --git a/app/org/thp/cortex/services/JobRunnerSrv.scala b/app/org/thp/cortex/services/JobRunnerSrv.scala
index 1731eec..3e332d9 100644
--- a/app/org/thp/cortex/services/JobRunnerSrv.scala
+++ b/app/org/thp/cortex/services/JobRunnerSrv.scala
@@ -130,8 +130,10 @@ class JobRunnerSrv @Inject() (
           .deepMerge(proxy_http)
           .deepMerge(proxy_https)
         (worker.config \ "cacerts").asOpt[String].foreach { cacerts =>
-          val cacertsFile = jobFolder.resolve("input").resolve("cacerts")
-          Files.write(cacertsFile, cacerts.getBytes)
+          if (cacerts.length > 0) {
+            val cacertsFile = jobFolder.resolve("input").resolve("cacerts")
+            Files.write(cacertsFile, cacerts.getBytes)
+          }
         }
         artifact +
           ("dataType"   -> JsString(job.dataType())) +

Alternatively, the worker config service could remove empty settings from the database when changes are saved.

Complementary information

Log excerpt from Cortex log:

2022-07-22 18:13:26,768 [DEBUG] from org.thp.cortex.services.JobRunnerSrv in application-analyzer-14 - Write worker input: {"data":"<masked>","dataType":"ip","tlp":1,"pap":1,"message":"","parameters":{},"config":{"proxy_https":null,"cacerts":"","max_pap":2,"jobTimeout":30,"check_tlp":true,"service":"public","proxy_http":null,"max_tlp":1,"auto_extract_artifacts":true,"jobCache":10,"check_pap":true}}
[...]
2022-07-22 18:13:28,235 [INFO] from org.thp.cortex.services.DockerJobRunnerSrv in application-analyzer-14 - Execute container 30cf490c9bc4950d473db6e8f9e88e7ca0961946309b8e26720552427a6af34f
  timeout: 30 minutes
  image  : cortexneurons/mnemonic_pdns_public:3
  volume : /tmp/cortex-job-BpyvJoIBM-rfMA6ZZviX-1408747954675906914:/job
  env    : REQUESTS_CA_BUNDLE=/job/input/cacerts

Worker output:

Traceback (most recent call last):
  File "/worker/MnemonicPDNS/pdns.py", line 87, in <module>
    PDNSv3().run()
  File "/worker/MnemonicPDNS/pdns.py", line 62, in run
    response = self.get_pdns(content)
  File "/worker/MnemonicPDNS/pdns.py", line 30, in get_pdns
    r = requests.get(url, params=self.params, headers=self.headers)
  File "/usr/local/lib/python3.10/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 563, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='api.mnemonic.no', port=443): Max retries exceeded with url: /pdns/v3/<masked>?aggregate=true&limit=0&includeAnonymous=true (Caused by SSLError(SSLError(136, '[X509: NO_CERTIFICATE_OR_CRL_FOUND] no certificate or crl found (_ssl.c:4123)')))

TheHive-Project / Cortex