eclecticiq / OpenTAXII

TAXII server implementation in Python from EclecticIQ
BSD 3-Clause "New" or "Revised" License
189 stars 89 forks source link

OpenTAXII feeds / collections not recognised by Qradar Threat Intelligence App Platform #55

Closed CQRuu closed 6 years ago

CQRuu commented 7 years ago

Hi,

When I tried to poll from my OpenTAXII server using Qradar's Threat Intelligence App platform, it throws an error - Failed to get list of collections from "https://domain:port/services/discovery". This is even though my collection ("collection") has 12 items. May I know if there's anyone who managed to connect their TAXII server with Qradar SIEM platform?

traut commented 7 years ago

We've tested pushing from QRadar but not polling. Could you please show your OpenTAXII configuration? What taxii-collections --path https://domain:port/services/discovery tells you (CLI call with Cabby client)?

CQRuu commented 7 years ago

Hi traut,

Here's the response:

misp@misp:~/misp-t2/bin/MISP-Taxii-Server$ taxii-collections --path http://domain:port/services/discovery -v 2017-03-21 22:47:51,670 cabby.dispatcher INFO: Sending Collection_Information_Request to http://domain:port/services/discovery 2017-03-21 22:47:51,670 cabby.dispatcher DEBUG: Request:

2017-03-21 22:47:51,725 requests.packages.urllib3.connectionpool DEBUG: Starting new HTTP connection (1): domain 2017-03-21 22:47:51,733 requests.packages.urllib3.connectionpool DEBUG: http://domain:port "POST /services/discovery HTTP/1.1" 200 439 2017-03-21 22:47:51,734 cabby.dispatcher DEBUG: Response:

Message not supported by this service 2017-03-21 22:47:51,735 cabby.cli.commons ERROR: FAILURE: Message not supported by this service Traceback (most recent call last): File "/usr/local/lib/python3.4/dist-packages/cabby/cli/commons.py", line 164, in run_client run_func(client, args.uri, args) File "/usr/local/lib/python3.4/dist-packages/cabby/cli/collections.py", line 14, in _runner collections.extend(client.get_collections(uri=path)) File "/usr/local/lib/python3.4/dist-packages/cabby/client11.py", line 283, in get_collections service_type=const.SVC_COLLECTION_MANAGEMENT) File "/usr/local/lib/python3.4/dist-packages/cabby/abstract.py", line 202, in _execute_request taxii_binding=self.taxii_binding) File "/usr/local/lib/python3.4/dist-packages/cabby/dispatcher.py", line 80, in send_taxii_request raise UnsuccessfulStatusError(obj) cabby.exceptions.UnsuccessfulStatusError: FAILURE: Message not supported by this service

And I also tried https (i changed my MISP environment to run with ssl):

misp@misp:~/misp-t2/bin/MISP-Taxii-Server$ taxii-collections --path https://domain:port/services/discovery -v 2017-03-21 22:26:11,879 cabby.dispatcher INFO: Sending Collection_Information_Request to https://domain:port/services/discovery 2017-03-21 22:26:11,880 cabby.dispatcher DEBUG: Request:

2017-03-21 22:26:11,883 requests.packages.urllib3.connectionpool DEBUG: Starting new HTTPS connection (1): domain 2017-03-21 22:26:42,544 cabby.cli.commons ERROR: EOF occurred in violation of protocol (_ssl.c:600) Traceback (most recent call last): File "/usr/local/lib/python3.4/dist-packages/requests/packages/urllib3/connectionpool.py", line 600, in urlopen chunked=chunked) File "/usr/local/lib/python3.4/dist-packages/requests/packages/urllib3/connectionpool.py", line 345, in _make_request self._validate_conn(conn) File "/usr/local/lib/python3.4/dist-packages/requests/packages/urllib3/connectionpool.py", line 844, in _validate_conn conn.connect() File "/usr/local/lib/python3.4/dist-packages/requests/packages/urllib3/connection.py", line 326, in connect sslcontext=context) File "/usr/local/lib/python3.4/dist-packages/requests/packages/urllib3/util/ssl.py", line 324, in ssl_wrap_socket return context.wrap_socket(sock, server_hostname=server_hostname) File "/usr/lib/python3.4/ssl.py", line 365, in wrap_socket _context=self) File "/usr/lib/python3.4/ssl.py", line 601, in init self.do_handshake() File "/usr/lib/python3.4/ssl.py", line 828, in do_handshake self._sslobj.do_handshake() ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:600)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/lib/python3.4/dist-packages/requests/adapters.py", line 423, in send timeout=timeout File "/usr/local/lib/python3.4/dist-packages/requests/packages/urllib3/connectionpool.py", line 630, in urlopen raise SSLError(e) requests.packages.urllib3.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:600)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/lib/python3.4/dist-packages/cabby/cli/commons.py", line 164, in run_client run_func(client, args.uri, args) File "/usr/local/lib/python3.4/dist-packages/cabby/cli/collections.py", line 14, in _runner collections.extend(client.get_collections(uri=path)) File "/usr/local/lib/python3.4/dist-packages/cabby/client11.py", line 283, in get_collections service_type=const.SVC_COLLECTION_MANAGEMENT) File "/usr/local/lib/python3.4/dist-packages/cabby/abstract.py", line 202, in _execute_request taxii_binding=self.taxii_binding) File "/usr/local/lib/python3.4/dist-packages/cabby/dispatcher.py", line 64, in send_taxii_request response = session.post(url, data=request_body, stream=True) File "/usr/local/lib/python3.4/dist-packages/requests/sessions.py", line 535, in post return self.request('POST', url, data=data, json=json, kwargs) File "/usr/local/lib/python3.4/dist-packages/requests/sessions.py", line 488, in request resp = self.send(prep, send_kwargs) File "/usr/local/lib/python3.4/dist-packages/requests/sessions.py", line 609, in send r = adapter.send(request, **kwargs) File "/usr/local/lib/python3.4/dist-packages/requests/adapters.py", line 497, in send raise SSLError(e, request=request) requests.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:600)

Kindly advise please? Thank you!!

traut commented 7 years ago

@CQRuu sorry, could you give me more details? Ishttp://domain:port/services/discovery an OpenTAXII instance or Qradar SIEM platform?

CQRuu commented 7 years ago

It is an opentaxii instance, i created the TAXII Server based on the guide here: https://opentaxii.readthedocs.io/en/latest/running.html

traut commented 7 years ago

ok, so you're trying to fetch data using Cabby's client.

taxii-collections --path http://domain:port/services/discovery -v

this ↑ is not correct, because you're sending collection request to discovery service. Did you mean to write --discovery http://domain:port/services/discovery?

taxii-collections --path https://domain:port/services/discovery -v

same mistake with paths here but additionally you also getting SSL error. I suspect you do not have a valid SSL certificate configured in your web server

CQRuu commented 7 years ago

my code response was based on your comment earlier

Could you please show your OpenTAXII configuration? What taxii-collections --path https://domain:port/services/discovery tells you (CLI call with Cabby client)?

I don't suppose Qradar needs the discovery path, because all it needs is to 1) find collections/data set in OpenTaxii 2) get feeds from the collections. But it is failing in step 1), because the collections in TAXII Server cannot be discovered... somehow. (Qradar is able to discover the data feeds / collections in hailataxii)

my response when calling taxii-discovery on my TAXII Server:

`=== Service Instance === Service Type: INBOX Service Version: urn:taxii.mitre.org:services:1.1 Protocol Binding: urn:taxii.mitre.org:protocol:https:1.0 Service Address: https://domain:port/services/inbox Message Binding: urn:taxii.mitre.org:message:xml:1.0 Message Binding: urn:taxii.mitre.org:message:xml:1.1 Inbox Service AC: ['urn:stix.mitre.org:xml:1.1.1', 'urn:stix.mitre.org:xml:1.2'] Available: True Message: Custom Inbox Service Description B

=== Service Instance === Service Type: DISCOVERY Service Version: urn:taxii.mitre.org:services:1.1 Protocol Binding: urn:taxii.mitre.org:protocol:http:1.0 Service Address: http://domain:port/services/discovery Message Binding: urn:taxii.mitre.org:message:xml:1.0 Message Binding: urn:taxii.mitre.org:message:xml:1.1 Available: True Message: Custom Discovery Service description

=== Service Instance === Service Type: DISCOVERY Service Version: urn:taxii.mitre.org:services:1.1 Protocol Binding: urn:taxii.mitre.org:protocol:https:1.0 Service Address: https://domain:port/services/discovery Message Binding: urn:taxii.mitre.org:message:xml:1.0 Message Binding: urn:taxii.mitre.org:message:xml:1.1 Available: True Message: Custom Discovery Service description

=== Service Instance === Service Type: COLLECTION_MANAGEMENT Service Version: urn:taxii.mitre.org:services:1.1 Protocol Binding: urn:taxii.mitre.org:protocol:http:1.0 Service Address: http://domain:port/services/collection_management Message Binding: urn:taxii.mitre.org:message:xml:1.0 Message Binding: urn:taxii.mitre.org:message:xml:1.1 Available: True Message: Custom Collection Management Service description

=== Service Instance === Service Type: COLLECTION_MANAGEMENT Service Version: urn:taxii.mitre.org:services:1.1 Protocol Binding: urn:taxii.mitre.org:protocol:https:1.0 Service Address: https://domain:port/services/collection_management Message Binding: urn:taxii.mitre.org:message:xml:1.0 Message Binding: urn:taxii.mitre.org:message:xml:1.1 Available: True Message: Custom Collection Management Service description

=== Service Instance === Service Type: POLL Service Version: urn:taxii.mitre.org:services:1.1 Protocol Binding: urn:taxii.mitre.org:protocol:http:1.0 Service Address: http://domain:port/services/poll Message Binding: urn:taxii.mitre.org:message:xml:1.0 Message Binding: urn:taxii.mitre.org:message:xml:1.1 Available: True Message: Custom poll Service description

=== Service Instance === Service Type: POLL Service Version: urn:taxii.mitre.org:services:1.1 Protocol Binding: urn:taxii.mitre.org:protocol:https:1.0 Service Address: https://domain:port/services/poll Message Binding: urn:taxii.mitre.org:message:xml:1.0 Message Binding: urn:taxii.mitre.org:message:xml:1.1 Available: True Message: Custom poll Service description `

traut commented 7 years ago

@CQRuu hah, my mistake :)

why don't you provide QRadar with Collection Service URL?

CQRuu commented 7 years ago

Do you mean letting Qradar try to poll from http://domain:port/services/discovery ?

I tried, but Qradar responded with:

[com.ibm.ThreatIntelligence] [ERROR] - Failed to get list of collections from http://domain:port/services/discovery; 'Discovery_Response'

and also

[com.ibm.ThreatIntelligence] [ERROR] - Failed to get list of collections from http://domain:port/services/collection_management;

traut commented 7 years ago

No, why would you want to pull from http://domain:port/services/discovery? It is a path to Discovery service. You're getting clear error from QRadar and from OpenTAXII when you're trying to do that.

You have Collection Management Service configured on http://domain:port/services/collection_management, so just use it

CQRuu commented 7 years ago

Hi Traut,

Tried that too, but:

2017-03-29 11:57:33,160 [com.ibm.ThreatIntelligence] [INFO] - Found 2 TAXII servers 2017-03-29 11:57:33,175 [com.ibm.ThreatIntelligence] [INFO] - Verified authentication token response status code: 200 2017-03-29 11:57:33,175 [com.ibm.ThreatIntelligence] [INFO] - Verification of authentication token succeeded 2017-03-29 11:57:48,518 [com.ibm.ThreatIntelligence] [INFO] - Updated configuration: {"auth_token": "token", "proxies": {"http": "proxy:port"}, "ca_bundle": null} 2017-03-29 11:58:19,918 [com.ibm.ThreatIntelligence] [INFO] - Sending Discovery request to http://domain:port/services/collection_management 2017-03-29 11:58:22,209 [com.ibm.ThreatIntelligence] [ERROR] - Failed to get list of collections from http://domain:port/services/collection_management; 'Discovery_Response'

Update: This is what I see on my logs of my TAXII Server:

{"service_id": "collection_management", "level": "warning", "timestamp": "2017-03-29T12:10:41.619948Z", "message_version": "urn:taxii.mitre.org:message:xml:1.1", "logger": "opentaxii.taxii.services.collection_management.CollectionManagementService", "event": "Message not supported", "message_id": "message-id", "message_type": "Discovery_Request"}

traut commented 7 years ago

wait, now you're the opposite thing is happening --

Sending Discovery request to http://domain:port/services/collection_management

this does not make any sense.

Discovery requests are sent to Discovery Service Collection Management requests are sent to Collection Management Service

what URL exactly QRadar asks you for?

CQRuu commented 7 years ago

Hi Traut,

Sorry to have confused you.

Qradar is just looking for a TAXII Endpoint where it can identify available data sets / data feeds, then connect to it. something like http://hailataxii.com/taxii-discovery-service (the URL), guest.abuse_ch, or system.Default etc (the data feed), and all the items within those feeds. Qradar expects and accepts this:

<taxii_11:Collection_Information_Response xmlns:soltra="http://taxii.soltra.com/messages/taxii_extension_xml_binding-1.1" xmlns:taxii="http://taxii.mitre.org/messages/taxii_xml_binding-1" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:tdq="http://taxii.mitre.org/query/taxii_default_query-1" message_id="messageid" in_response_to="requestid"><taxii_11:Collection collection_name="admin.FSISAC-Feeds-Only" collection_type="DATA_FEED" available="true"><taxii_11:Description>admin.FSISAC-Feeds-Only</taxii_11:Description><taxii_11:Content_Binding binding_id="urn:stix.mitre.org:xml:1.0"/><taxii_11:Polling_Service><taxii_11:Protocol_Binding>urn:taxii.mitre.org:protocol:https:1.0</taxii_11:Protocol_Binding><taxii_11:Address><<ip:port>>/taxii-data</taxii_11:Address><taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.1</taxii_11:Message_Binding></taxii_11:Polling_Service></taxii_11:Collection><taxii_11:Collection collection_name="system.Default" collection_type="DATA_FEED" available="true"><taxii_11:Description>system.Default</taxii_11:Description><taxii_11:Content_Binding binding_id="urn:stix.mitre.org:xml:1.0"/><taxii_11:Polling_Service><taxii_11:Protocol_Binding>urn:taxii.mitre.org:protocol:https:1.0</taxii_11:Protocol_Binding><taxii_11:Address><<ip:port>>/taxii-data</taxii_11:Address><taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.1</taxii_11:Message_Binding></taxii_11:Polling_Service></taxii_11:Collection></taxii_11:Collection_Information_Response>

So to map it to my situation:

  1. Qradar is trying to find collections from another taxii server given its url)
  2. OpenTAXII is the taxii server with all my data, and it goes by http://domain:port/services/discovery

On other computers, I can do cabby commands like taxii-poll, taxii-discovery, taxii-push with no issues.

However, on Qradar, it doesn't do cabby and it doesn't recognise the "DATA_SET" that OpenTAXII provides: `

12 urn:taxii.mitre.org:protocol:http:1.0 <>/services/poll urn:taxii.mitre.org:message:xml:1.0 urn:taxii.mitre.org:message:xml:1.1 urn:taxii.mitre.org:protocol:https:1.0 <>/services/poll urn:taxii.mitre.org:message:xml:1.0 urn:taxii.mitre.org:message:xml:1.1 urn:taxii.mitre.org:protocol:http:1.0 <>/services/collection-management urn:taxii.mitre.org:message:xml:1.0 urn:taxii.mitre.org:message:xml:1.1 urn:taxii.mitre.org:protocol:https:1.0 <>/services/collection-management urn:taxii.mitre.org:message:xml:1.0 urn:taxii.mitre.org:message:xml:1.1 ...`
traut commented 7 years ago

Ah, so it is QRadar issue. You give it a Discovery URL and it decides to send collection requests there. Please open a bug report with them (and post a link here when you do).

I guess this was built with Soltra Edge in mind - Soltra Edge maps every service to the same URL. Obviously, this is not a requirement.

Correct way for QRadar would be to:

traut commented 7 years ago

also, you can change collection type to DATA_FEED - that's just a flag on a collection. See type field in example collections config.

CQRuu commented 7 years ago

Hi Traut,

Ahh I see. Thanks for the advice :)!

VibhavariMandal commented 7 years ago

Hi,

I am facing ssl error on trying to send a discovery request with https binding, I am using libtaxii as client, what could be the issue? doesnt opentaxii support https? how to make this certificate validation work? stuck on this issue for quite long time,..any leads would be helpful.

root@vatd:~# discovery_client -u https://10.213.18.230:9001/services/discovery-a Request:

Message Type: Discovery_Request Message ID: 4007619629629755912

Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/libtaxii-1.1.111-py2.7.egg/libtaxii/scripts/init.py", line 375, in call url.port) File "/usr/lib/python2.7/site-packages/libtaxii-1.1.111-py2.7.egg/libtaxii/clients.py", line 346, in call_taxii_service2 response = urllib.request.urlopen(req) File "/usr/lib64/python2.7/urllib2.py", line 154, in urlopen return opener.open(url, data, timeout) File "/usr/lib64/python2.7/urllib2.py", line 429, in open response = self._open(req, data) File "/usr/lib64/python2.7/urllib2.py", line 447, in _open '_open', req) File "/usr/lib64/python2.7/urllib2.py", line 407, in _call_chain result = func(*args) File "/usr/lib/python2.7/site-packages/libtaxii-1.1.111-py2.7.egg/libtaxii/clients.py", line 374, in https_open return self.do_open(self.get_connection, req) File "/usr/lib64/python2.7/urllib2.py", line 1200, in do_open raise URLError(err) URLError: <urlopen error EOF occurred in violation of protocol (_ssl.c:590)>

traut commented 7 years ago

@VibhavariMandal OpenTAXII does not do SSL verification, it must be running behind a webserver as recommended in documentation. This webserver will take care of HTTPS setup

// also, this error is unrelated to the issue discussed in this thread

VibhavariMandal commented 7 years ago

@traut thanks for response & sorry for posting on a wrong thread

traut commented 6 years ago

closing this, since no new details on the original issue

mcamachotw commented 6 years ago

I tried to write here, but I did not do it properly, and I definitely read the last thing, to change from data_set to data_feed, but in my case it still does not work, I do not know if @CQRuu managed to solve this case and how did they do it? The opentaxii server works well and shows the collections from the server itself, the problem is wanting to pull the collections of Qradar appears "Failed to get list of collections". I would really appreciate any clue to advance on this topic. Regards,

arcsector commented 6 years ago

@mcamachotw Please post your entire process/setup so that we can help you accurately identify the problem. Can you get a list of collections from the opentaxii server on a non-local machine (try sending a collection_management query from a different computer)? Are you running a nginx reverse proxy on gunicorn like gunicorn's homepage suggests? Are you running in dev mode? Do you have a firewall between the listener and the Qradar machine?

mcamachotw commented 6 years ago

@arcsector Thanks in advance for reply me, I edit my previous message, because now appear me this error in qradar "There is a problem connecting to the TAXII server. Verify that the TAXII server is available. Failed to connect to the server due to SSL problems. This might be caused by an invalid client certificate, an unknown Certificate Authority, or a problem with the server."

the logs in server opentaxii says

{"timestamp": "2018-02-22T04:32:20.861390Z", "logger": "opentaxii.server", "event": "opentaxii.server_configured", "level": "info"} ("'bWlndWVsOmluZm8='", 16) ("'bWlndWVsOmluZm8='", 16)

Could you please help me?, Thanks in advance. @VibhavariMandal, did you fix the issue with SSL?

TiagoSantos84 commented 6 years ago

Hi,

I have a similiar issue with QRADAR. If I give the QRadar taxii feed: https://host:port/services/collection-management I get the error: There is a problem connecting to the TAXII server. Verify that the TAXII server is available. Get list of collections failed If I run on cli: taxii-collections --path https://host:port/services/collection-management I get:

2018-03-02 11:40:00,202 INFO: Sending Collection_Information_Request to https://host:9000/services/collection-management
2018-03-02 11:40:00,206 INFO: Starting new HTTPS connection (1): host
=== Data Collection Information ===
  Collection Name: collection
  Collection Type: DATA_SET
  Available: True
  Collection Description: None
  Supported Content: All
  === Polling Service Instance ===
    Poll Protocol: urn:taxii.mitre.org:protocol:http:1.0
    Poll Address: http://localhost:9000/services/poll
    Message Binding: urn:taxii.mitre.org:message:xml:1.0
    Message Binding: urn:taxii.mitre.org:message:xml:1.1
  === Polling Service Instance ===
    Poll Protocol: urn:taxii.mitre.org:protocol:https:1.0
    Poll Address: https://localhost:9000/services/poll
    Message Binding: urn:taxii.mitre.org:message:xml:1.0
    Message Binding: urn:taxii.mitre.org:message:xml:1.1
  === Subscription Service ===
    Protocol Binding: urn:taxii.mitre.org:protocol:http:1.0
    Address: http://localhost:9000/services/collection-management
    Message Binding: urn:taxii.mitre.org:message:xml:1.0
    Message Binding: urn:taxii.mitre.org:message:xml:1.1
  === Subscription Service ===
    Protocol Binding: urn:taxii.mitre.org:protocol:https:1.0
    Address: https://localhost:9000/services/collection-management
    Message Binding: urn:taxii.mitre.org:message:xml:1.0
    Message Binding: urn:taxii.mitre.org:message:xml:1.1
  === Receiving Inbox Service ===
    Protocol Binding: urn:taxii.mitre.org:protocol:http:1.0
    Address: http://localhost:9000/services/inbox
    Message Binding: urn:taxii.mitre.org:message:xml:1.0
    Message Binding: urn:taxii.mitre.org:message:xml:1.1
    Supported Content: urn:stix.mitre.org:xml:1.1.1
    Supported Content: urn:stix.mitre.org:xml:1.2
  === Receiving Inbox Service ===
    Protocol Binding: urn:taxii.mitre.org:protocol:https:1.0
    Address: https://localhost:9000/services/inbox
    Message Binding: urn:taxii.mitre.org:message:xml:1.0
    Message Binding: urn:taxii.mitre.org:message:xml:1.1
    Supported Content: urn:stix.mitre.org:xml:1.1.1
    Supported Content: urn:stix.mitre.org:xml:1.2
==================================

I can see this "log" in taxii server:


'cm9vdDpyb290' 12
{"message_version": "urn:taxii.mitre.org:message:xml:1.1", "message_id": "96449269-de79-4201-82bf-5e67c297e904", "service_id": "collection_management", "message_type": "Discovery_Request", "timestamp": "2018-03-02T12:02:27.068582Z", "logger": "opentaxii.taxii.services.collection_management.CollectionManagementService", "level": "warning", "event": "Message not supported"}
{"exception": "Traceback (most recent call last):\n  File \"/usr/local/lib/python3.5/dist-packages/opentaxii/taxii/services/abstract.py\", line 101, in get_message_handler\n    return self.handlers[message.message_type]\nKeyError: 'Discovery_Request'\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"/usr/local/lib/python3.5/dist-packages/flask/app.py\", line 1612, in full_dispatch_request\n    rv = self.dispatch_request()\n  File \"/usr/local/lib/python3.5/dist-packages/flask/app.py\", line 1598, in dispatch_request\n    return self.view_functions[rule.endpoint](**req.view_args)\n  File \"/usr/local/lib/python3.5/dist-packages/opentaxii/middleware.py\", line 76, in wrapper\n    return _process_with_service(service)\n  File \"/usr/local/lib/python3.5/dist-packages/opentaxii/middleware.py\", line 154, in _process_with_service\n    response_message = service.process(request.headers, taxii_message)\n  File \"/usr/local/lib/python3.5/dist-packages/opentaxii/taxii/services/abstract.py\", line 77, in process\n    handler = self.get_message_handler(message)\n  File \"/usr/local/lib/python3.5/dist-packages/opentaxii/taxii/services/abstract.py\", line 110, in get_message_handler\n    in_response_to=message.message_id)\n  File \"/usr/local/lib/python3.5/dist-packages/opentaxii/taxii/exceptions.py\", line 48, in raise_failure\n    tb=tb)\n  File \"/usr/lib/python3/dist-packages/six.py\", line 685, in reraise\n    raise value.with_traceback(tb)\n  File \"/usr/local/lib/python3.5/dist-packages/opentaxii/taxii/services/abstract.py\", line 101, in get_message_handler\n    return self.handlers[message.message_type]\nopentaxii.taxii.exceptions.FailureStatus: 'Discovery_Request'", "level": "warning", "logger": "opentaxii.middleware", "timestamp": "2018-03-02T12:02:27.068987Z", "event": "Status exception"}

On the other hand if I try with on Qradar: https://hosttaxii:9000/services/discovery

I get: There is a problem connecting to the TAXII server. Verify that the TAXII server is available. Get list of collections failed.

However with on cli: taxii-discovery --path https://misp2.ciberdefesa.pt:9000/services/discovery

I get:

2018-03-02 12:09:48,984 INFO: Sending Discovery_Request to https://hosttaxii:9000/services/discovery
2018-03-02 12:09:48,988 INFO: Starting new HTTPS connection (1): hosttaxii
2018-03-02 12:09:49,016 INFO: 8 services discovered
=== Service Instance ===
  Service Type: COLLECTION_MANAGEMENT
  Service Version: urn:taxii.mitre.org:services:1.1
  Protocol Binding: urn:taxii.mitre.org:protocol:http:1.0
  Service Address: http://localhost:9000/services/collection-management
  Message Binding: urn:taxii.mitre.org:message:xml:1.0
  Message Binding: urn:taxii.mitre.org:message:xml:1.1
  Available: True
  Message: Collection Management Service
(...) SIP (...)
arcsector commented 6 years ago

@TiagoSantos84 can you use something like fiddler or wireshark to capture Qradar's full post request? This would be helpful because then we can determine if the TAXII server is misbehaving or if the message is in an invalid format.

TiagoSantos84 commented 6 years ago

@arcsector , I had the same SSL issue in qradar. Try to run the server giving the .pem (with public and chain certificates) and .key (private key). It worked for me. Did you solve that problem? Sorry, I was writing for you... and I didn't see that probably you already solved it...

TiagoSantos84 commented 6 years ago

@arcsector, thank you for your fast repply!

I'm figuring out how to get the full post request. probably tcpdump can do it..

Thank you.

TiagoSantos84 commented 6 years ago
........POST /services/discovery HTTP/1.1
Host: hosttaxii:9000
X-TAXII-Accept: urn:taxii.mitre.org:message:xml:1.1
Content-Length: 336
X-TAXII-Services: urn:taxii.mitre.org:services:1.1
Accept-Encoding: gzip, deflate
User-agent: Qradar TI App/1.4.0.20180117152642
Accept: application/xml
X-TAXII-Content-Type: urn:taxii.mitre.org:message:xml:1.1
Connection: keep-alive
X-TAXII-Protocol: urn:taxii.mitre.org:protocol:http:1.0
Content-type: application/xml
Authorization: Basic cm9vdDpyb290

12:25:50.030326 IP (tos 0x0, ttl 63, id 51379, offset 0, flags [DF], proto TCP (6), length 388)
    siemlab1.host.pt.49518 > hosttaxii.pt.cslistener: Flags [P.], cksum 0x19f7 (incorrect -> 0xa909), seq 507:843, ack 1, win 11, options [nop,nop,TS val 785103382 ecr 16564979], length 336
E.....@.?.Y@
..s
....n#(3.t..+.............
........
<taxii_11:Discovery_Request xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xsi:schemaLocation="http://taxii.mitre.org/messages/taxii_xml_binding-1.1 http://taxii.mitre.org/messages/taxii_xml_binding-1.1"  message_id="692c8b66-8490-440f-b3b5-4efe78122582" />

.... This is the post message from siem to taxiiserver,...

traut commented 6 years ago

@TiagoSantos84 and what's the reply from the server? is there a follow-up taxii poll request?

TiagoSantos84 commented 6 years ago

@traut,

when I issue the request: http://hostaxii:9000/services/discovery

I get:

..../..2HTTP/1.1 200 OK
Server: gunicorn/19.7.1
Date: Fri, 02 Mar 2018 13:49:06 GMT
Connection: close
Content-Type: application/xml
Content-Length: 5290
X-TAXII-Services: urn:taxii.mitre.org:services:1.1
X-TAXII-Content-Type: urn:taxii.mitre.org:message:xml:1.1
X-TAXII-Protocol: urn:taxii.mitre.org:protocol:http:1.0

13:48:37.553562 IP (tos 0x0, ttl 63, id 12124, offset 0, flags [DF], proto TCP (6), length 5342)
    hostaxii.cslistener > siemlab1.pt.37490: Flags [P.], cksum 0x2d51 (incorrect -> 0xed92), seq 322:5612, ack 844, win 243, options [nop,nop,TS val 17806864 ecr 790070834], length 5290
E.../\@.?..=
...
..s#(.r............-Q.....
..../..2<taxii_11:Discovery_Response xmlns:taxii="http://taxii.mitre.org/messages/taxii_xml_binding-1" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:tdq="http://taxii.mitre.org/query/taxii_default_query-1" message_id="4322411540731582924" in_response_to="46c6d8f0-47ef-4cc3-9923-7dc4eb6d439a">
  <taxii_11:Service_Instance service_type="COLLECTION_MANAGEMENT" service_version="urn:taxii.mitre.org:services:1.1" available="true">
    <taxii_11:Protocol_Binding>urn:taxii.mitre.org:protocol:http:1.0</taxii_11:Protocol_Binding>
    <taxii_11:Address>http://localhost:9000/services/collection-management</taxii_11:Address>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.0</taxii_11:Message_Binding>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.1</taxii_11:Message_Binding>
    <taxii_11:Message>Collection Management Service</taxii_11:Message>
  </taxii_11:Service_Instance>
  <taxii_11:Service_Instance service_type="COLLECTION_MANAGEMENT" service_version="urn:taxii.mitre.org:services:1.1" available="true">
    <taxii_11:Protocol_Binding>urn:taxii.mitre.org:protocol:https:1.0</taxii_11:Protocol_Binding>
    <taxii_11:Address>https://localhost:9000/services/collection-management</taxii_11:Address>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.0</taxii_11:Message_Binding>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.1</taxii_11:Message_Binding>
    <taxii_11:Message>Collection Management Service</taxii_11:Message>
  </taxii_11:Service_Instance>
  <taxii_11:Service_Instance service_type="DISCOVERY" service_version="urn:taxii.mitre.org:services:1.1" available="true">
    <taxii_11:Protocol_Binding>urn:taxii.mitre.org:protocol:http:1.0</taxii_11:Protocol_Binding>
    <taxii_11:Address>http://localhost:9000/services/discovery</taxii_11:Address>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.0</taxii_11:Message_Binding>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.1</taxii_11:Message_Binding>
    <taxii_11:Message>Discovery Service</taxii_11:Message>
  </taxii_11:Service_Instance>
  <taxii_11:Service_Instance service_type="DISCOVERY" service_version="urn:taxii.mitre.org:services:1.1" available="true">
    <taxii_11:Protocol_Binding>urn:taxii.mitre.org:protocol:https:1.0</taxii_11:Protocol_Binding>
    <taxii_11:Address>https://localhost:9000/services/discovery</taxii_11:Address>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.0</taxii_11:Message_Binding>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.1</taxii_11:Message_Binding>
    <taxii_11:Message>Discovery Service</taxii_11:Message>
  </taxii_11:Service_Instance>
  <taxii_11:Service_Instance service_type="INBOX" service_version="urn:taxii.mitre.org:services:1.1" available="true">
    <taxii_11:Protocol_Binding>urn:taxii.mitre.org:protocol:http:1.0</taxii_11:Protocol_Binding>
    <taxii_11:Address>http://localhost:9000/services/inbox</taxii_11:Address>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.0</taxii_11:Message_Binding>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.1</taxii_11:Message_Binding>
    <taxii_11:Content_Binding binding_id="urn:stix.mitre.org:xml:1.1.1"/>
    <taxii_11:Content_Binding binding_id="urn:stix.mitre.org:xml:1.2"/>
    <taxii_11:Message>Inbox Service</taxii_11:Message>
  </taxii_11:Service_Instance>
  <taxii_11:Service_Instance service_type="INBOX" service_version="urn:taxii.mitre.org:services:1.1" available="true">
    <taxii_11:Protocol_Binding>urn:taxii.mitre.org:protocol:https:1.0</taxii_11:Protocol_Binding>
    <taxii_11:Address>https://localhost:9000/services/inbox</taxii_11:Address>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.0</taxii_11:Message_Binding>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.1</taxii_11:Message_Binding>
    <taxii_11:Content_Binding binding_id="urn:stix.mitre.org:xml:1.1.1"/>
    <taxii_11:Content_Binding binding_id="urn:stix.mitre.org:xml:1.2"/>
    <taxii_11:Message>Inbox Service</taxii_11:Message>
  </taxii_11:Service_Instance>
  <taxii_11:Service_Instance service_type="POLL" service_version="urn:taxii.mitre.org:services:1.1" available="true">
    <taxii_11:Protocol_Binding>urn:taxii.mitre.org:protocol:http:1.0</taxii_11:Protocol_Binding>
    <taxii_11:Address>http://localhost:9000/services/poll</taxii_11:Address>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.0</taxii_11:Message_Binding>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.1</taxii_11:Message_Binding>
    <taxii_11:Message>Poll Service</taxii_11:Message>
  </taxii_11:Service_Instance>
  <taxii_11:Service_Instance service_type="POLL" service_version="urn:taxii.mitre.org:services:1.1" available="true">
    <taxii_11:Protocol_Binding>urn:taxii.mitre.org:protocol:https:1.0</taxii_11:Protocol_Binding>
    <taxii_11:Address>https://localhost:9000/services/poll</taxii_11:Address>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.0</taxii_11:Message_Binding>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.1</taxii_11:Message_Binding>
    <taxii_11:Message>Poll Service</taxii_11:Message>
  </taxii_11:Service_Instance>
</taxii_11:Discovery_Response>

If: http://hostaxii.pt:9000/services/collection-management


..../.  .HTTP/1.1 200 OK
Server: gunicorn/19.7.1
Date: Fri, 02 Mar 2018 13:54:02 GMT
Connection: close
Content-Type: application/xml
Content-Length: 438
X-TAXII-Services: urn:taxii.mitre.org:services:1.1
X-TAXII-Content-Type: urn:taxii.mitre.org:message:xml:1.1
X-TAXII-Protocol: urn:taxii.mitre.org:protocol:http:1.0

13:53:33.382875 IP (tos 0x0, ttl 63, id 56267, offset 0, flags [DF], proto TCP (6), length 490)
    hostaxii.pt.cslistener > siemlab1.pt.40152: Flags [P.], cksum 0x4bed (correct), seq 321:759, ack 856, win 243, options [nop,nop,TS val 17880822 ecr 790366659], length 438
E.....@.?.E.
...
..s#(..q..k..K.....K......
..../.  .<taxii_11:Status_Message xmlns:taxii="http://taxii.mitre.org/messages/taxii_xml_binding-1" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:tdq="http://taxii.mitre.org/query/taxii_default_query-1" message_id="109741544258536812" in_response_to="57680687-914e-47a8-8255-120230199e40" status_type="FAILURE">
  <taxii_11:Message>Message not supported by this service</taxii_11:Message>
</taxii_11:Status_Message>

13:53:33.382889 IP (tos 0x0, ttl 63, id 56268, offset 0, flags [DF], proto TCP (6), length 52)
    hostaxii.pt.cslistener > siemlab1.pt.40152: Flags [F.], cksum 0x07a0 (correct), seq 759, ack 856, win 243, options [nop,nop,TS val 17880822 ecr 790366659], length 0
E..4..@.?.Gw
...
..s#(..q..!..K............
..../.  .
13:53:33.384651 IP (tos 0x0, ttl 63, id 56269, offset 0, flags [DF], proto TCP (6), length 52)
   hostaxii.pt.cslistener > siemlab1.pt.40152: Flags [.], cksum 0x0752 (correct), seq 760, ack 857, win 243, options [nop,nop,TS val 17880822 ecr 790366736], length 0
E..4..@.?.Gv
...
..s#(..q.."..K......R.....
..../.
.
traut commented 6 years ago

@TiagoSantos84 so all of that looks ok. If you set Discovery URL — it gets TAXII Discovery Request, processes it and responds with TAXII Discovery Response. So far so good. Are there any follow up requests from QRadar after that?

also, note that you haven't changed domain of your OpenTAXII server, so it replied with URLs that have localhost in them. This might be an issue if you're requesting data from a different machine

TiagoSantos84 commented 6 years ago

@traut Yes, it's true. I change the configuration in config.yaml. I put hostaxii in domain:

---
domain: "hostaxii:9000"
support_basic_auth: yes

However, I still get the error..

If I lunch gunicorn without SSL, I get the error from Qradar due to the SSL problem...

traut commented 6 years ago

@TiagoSantos84 did you restart the service after changing the config? If yes, please provide again all request/responses you see on the wire if you use discovery url.

TiagoSantos84 commented 6 years ago

@traut ok...

request:

/oI     .%..POST /services/discovery HTTP/1.1
Host: hosttaxii.pt:9000
X-TAXII-Accept: urn:taxii.mitre.org:message:xml:1.1
Content-Length: 336
X-TAXII-Services: urn:taxii.mitre.org:services:1.1
Accept-Encoding: gzip, deflate
User-agent: Qradar TI App/1.4.0.20180117152642
Accept: application/xml
X-TAXII-Content-Type: urn:taxii.mitre.org:message:xml:1.1
Connection: keep-alive
X-TAXII-Protocol: urn:taxii.mitre.org:protocol:http:1.0
Content-type: application/xml
Authorization: Basic cm9vdDpyb290

<taxii_11:Discovery_Request xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xsi:schemaLocation="http://taxii.mitre.org/messages/taxii_xml_binding-1.1 http://taxii.mitre.org/messages/taxii_xml_binding-1.1"  message_id="598a4dd5-9fb3-4f4b-b0ca-9669c262950d" />

answer:

.%../oI HTTP/1.1 200 OK
Server: gunicorn/19.7.1
Date: Fri, 02 Mar 2018 15:24:58 GMT
Connection: close
Content-Type: application/xml
Content-Length: 5378
X-TAXII-Services: urn:taxii.mitre.org:services:1.1
X-TAXII-Protocol: urn:taxii.mitre.org:protocol:http:1.0
X-TAXII-Content-Type: urn:taxii.mitre.org:message:xml:1.1

..s#(...(...l......-......
.%../oI <taxii_11:Discovery_Response xmlns:taxii="http://taxii.mitre.org/messages/taxii_xml_binding-1" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:tdq="http://taxii.mitre.org/query/taxii_default_query-1" message_id="8553896905246595459" in_response_to="598a4dd5-9fb3-4f4b-b0ca-9669c262950d">
  <taxii_11:Service_Instance service_type="COLLECTION_MANAGEMENT" service_version="urn:taxii.mitre.org:services:1.1" available="true">
    <taxii_11:Protocol_Binding>urn:taxii.mitre.org:protocol:http:1.0</taxii_11:Protocol_Binding>
    <taxii_11:Address>http://hosttaxii.pt:9000/services/collection-management</taxii_11:Address>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.0</taxii_11:Message_Binding>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.1</taxii_11:Message_Binding>
    <taxii_11:Message>Collection Management Service</taxii_11:Message>
  </taxii_11:Service_Instance>
  <taxii_11:Service_Instance service_type="COLLECTION_MANAGEMENT" service_version="urn:taxii.mitre.org:services:1.1" available="true">
    <taxii_11:Protocol_Binding>urn:taxii.mitre.org:protocol:https:1.0</taxii_11:Protocol_Binding>
    <taxii_11:Address>https://hosttaxii.pt:9000/services/collection-management</taxii_11:Address>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.0</taxii_11:Message_Binding>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.1</taxii_11:Message_Binding>
    <taxii_11:Message>Collection Management Service</taxii_11:Message>
  </taxii_11:Service_Instance>
  <taxii_11:Service_Instance service_type="DISCOVERY" service_version="urn:taxii.mitre.org:services:1.1" available="true">
    <taxii_11:Protocol_Binding>urn:taxii.mitre.org:protocol:http:1.0</taxii_11:Protocol_Binding>
    <taxii_11:Address>http://hosttaxii.pt:9000/services/discovery</taxii_11:Address>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.0</taxii_11:Message_Binding>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.1</taxii_11:Message_Binding>
    <taxii_11:Message>Discovery Service</taxii_11:Message>
  </taxii_11:Service_Instance>
  <taxii_11:Service_Instance service_type="DISCOVERY" service_version="urn:taxii.mitre.org:services:1.1" available="true">
    <taxii_11:Protocol_Binding>urn:taxii.mitre.org:protocol:https:1.0</taxii_11:Protocol_Binding>
    <taxii_11:Address>https://hosttaxii.pt:9000/services/discovery</taxii_11:Address>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.0</taxii_11:Message_Binding>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.1</taxii_11:Message_Binding>
    <taxii_11:Message>Discovery Service</taxii_11:Message>
  </taxii_11:Service_Instance>
  <taxii_11:Service_Instance service_type="INBOX" service_version="urn:taxii.mitre.org:services:1.1" available="true">
    <taxii_11:Protocol_Binding>urn:taxii.mitre.org:protocol:http:1.0</taxii_11:Protocol_Binding>
    <taxii_11:Address>http://hosttaxii.pt:9000/services/inbox</taxii_11:Address>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.0</taxii_11:Message_Binding>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.1</taxii_11:Message_Binding>
    <taxii_11:Content_Binding binding_id="urn:stix.mitre.org:xml:1.1.1"/>
    <taxii_11:Content_Binding binding_id="urn:stix.mitre.org:xml:1.2"/>
    <taxii_11:Message>Inbox Service</taxii_11:Message>
  </taxii_11:Service_Instance>
  <taxii_11:Service_Instance service_type="INBOX" service_version="urn:taxii.mitre.org:services:1.1" available="true">
    <taxii_11:Protocol_Binding>urn:taxii.mitre.org:protocol:https:1.0</taxii_11:Protocol_Binding>
    <taxii_11:Address>https://hosttaxii.pt:9000/services/inbox</taxii_11:Address>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.0</taxii_11:Message_Binding>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.1</taxii_11:Message_Binding>
    <taxii_11:Content_Binding binding_id="urn:stix.mitre.org:xml:1.1.1"/>
    <taxii_11:Content_Binding binding_id="urn:stix.mitre.org:xml:1.2"/>
    <taxii_11:Message>Inbox Service</taxii_11:Message>
  </taxii_11:Service_Instance>
  <taxii_11:Service_Instance service_type="POLL" service_version="urn:taxii.mitre.org:services:1.1" available="true">
    <taxii_11:Protocol_Binding>urn:taxii.mitre.org:protocol:http:1.0</taxii_11:Protocol_Binding>
    <taxii_11:Address>http://hosttaxii.pt:9000/services/poll</taxii_11:Address>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.0</taxii_11:Message_Binding>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.1</taxii_11:Message_Binding>
    <taxii_11:Message>Poll Service</taxii_11:Message>
  </taxii_11:Service_Instance>
  <taxii_11:Service_Instance service_type="POLL" service_version="urn:taxii.mitre.org:services:1.1" available="true">
    <taxii_11:Protocol_Binding>urn:taxii.mitre.org:protocol:https:1.0</taxii_11:Protocol_Binding>
    <taxii_11:Address>https://hosttaxii.pt:9000/services/poll</taxii_11:Address>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.0</taxii_11:Message_Binding>
    <taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.1</taxii_11:Message_Binding>
    <taxii_11:Message>Poll Service</taxii_11:Message>
  </taxii_11:Service_Instance>
</taxii_11:Discovery_Response>

after that I can't see anything in clear text... probably because:

<taxii_11:Address>**https:**//hosttaxii.pt:9000/services/poll</taxii_11:Address>

And I get from QRadar:

There is a problem connecting to the TAXII server. Verify that the TAXII server is available. Failed to connect to the server due to SSL problems. This might be caused by an invalid client certificate, an unknown Certificate Authority, or a problem with the server.

traut commented 6 years ago

@TiagoSantos84 so you have a clear error description from QRadar there. You need to fix your SSL or use HTTP

TiagoSantos84 commented 6 years ago

@traut,

To use http I luch gunicorn as: /usr/local/bin/gunicorn opentaxii.http:app --bind 0.0.0.0:9000

and then I receive the SSL error from qradar ... probably because I receive from services something like: <taxii_11:Address>https://hosttaxii.pt:9000/services/inbox</taxii_11:Address>

If I change to SSL: Run gunicorn as: gunicorn --certfile=misptaxii.pem --keyfile=misptaxii.key opentaxii.http:app --bind 0.0.0.0:9000 I can't see anything with tcpdump

certfile (misptaxii.pem) content is the chain and public key. And Qradar gives me this: There is a problem connecting to the TAXII server. Verify that the TAXII server is available. Get list of collections failed

What is my mistake?

traut commented 6 years ago

@TiagoSantos84 do you have a valid certificate verified by proper CA?

TiagoSantos84 commented 6 years ago

@traut Yes I do!

traut commented 6 years ago

@TiagoSantos84 in this case, I think this is QRadar's issue. Maybe it doesn't like your custom port 9000 with HTTPs schema, or it doesn't like your SSL cert for some reason. As far as I can see, OpenTAXII replied correctly to received requests and according to your configuration. Unfortunately, my knowledge ends here.

traut commented 6 years ago

just to quickly check your SSL, what does curl -v https://hosttaxii.pt:9000/services/inbox tell you?

TiagoSantos84 commented 6 years ago

@traut

* About to connect() to hosttaxii.pt port 9000 (#0)
*   Trying 10.***.***.**...
* Connected to hosttaxii.pt (10.***.***.**) port 9000 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
* Server certificate:
*       subject: CN=*.hosttaxii.pt,C=PT,E=******@******.pt,L=Lisboa,O=Est*** ***** ******,OU=Cen******,ST=Portugal
*       start date: Sep 12 15:58:53 2017 GMT
*       expire date: Sep 12 15:58:52 2018 GMT
*       common name: *.hosttaxii.pt
*       issuer: CN=ECCE 001,OU=ECEstado,O=SCEE,C=PT
> GET /services/inbox HTTP/1.1
> User-Agent: curl/7.29.0
> Host: hosttaxii.pt:9000
> Accept: */*
>
< HTTP/1.1 405 METHOD NOT ALLOWED
< Server: gunicorn/19.7.1
< Date: Fri, 02 Mar 2018 15:46:13 GMT
< Connection: close
< Content-Type: text/html
< Allow: OPTIONS, POST
< Content-Length: 178
<
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>405 Method Not Allowed</title>
<h1>Method Not Allowed</h1>
<p>The method is not allowed for the requested URL.</p>
* Closing connection 0

just change some values to not put details.

traut commented 6 years ago

@TiagoSantos84 that looks ok to me. sorry, no idea on why would QRadar wouldn't like that. Maybe ask that community?

mcamachotw commented 6 years ago

Hi @traut / @TiagoSantos84 I still have the same problem, I still do not know how to solve it, I did not even understand the part of not using https, I prefer to use http first to rule out the error of the certificate, but I do not know in which file this option is defined, would you mind telling me please , @ I have created a ticket for IBM to know what the error could be, but first I would like to discard using http instead of https, If you can clarify in which part I define to use http and in what file I will thank you Regards,

traut commented 6 years ago

@mcamachotw opentaxii config file and your gunicorn config. make sure to read the docs

@TiagoSantos84 also, I noticed you configured your Discovery Service to support both HTTP and HTTPs but you run gunicorn with HTTPs. That looks like a misconfiguration. Please check your Discovery Service configuration

TiagoSantos84 commented 6 years ago

@traut How can I do that? Change the "Discovery Service configuration"?

traut commented 6 years ago

@TiagoSantos84 if you already configured your services, you need to do that in DB. If you have fresh clean DB, adjust YAML configuration files and sync configs into DB (docs for 0.1.9)

TiagoSantos84 commented 6 years ago

@traut Are you refering to this?

    - id: discovery
      type: discovery
      address: /services/discovery
      description: Discovery Service
      advertised_services:
        - inbox
        - discovery
        - collection_management
        - poll
      protocol_bindings:
        - urn:taxii.mitre.org:protocol:http:1.0
        - urn:taxii.mitre.org:protocol:https:1.0

By removing the line:

traut commented 6 years ago

@TiagoSantos84 yep

TiagoSantos84 commented 6 years ago

@traut how to do a comment in the .yaml? with --- ? or #?

TiagoSantos84 commented 6 years ago

@traut Thank you so much!! I comment the https, just to try and debug if it is necessary but it worked at the first time!! Now I already can choose parameters in Qradar! :)

traut commented 6 years ago

@TiagoSantos84 👍