search

MISP / MISP-Taxii-Server

An OpenTAXII Configuration for MISP
BSD 3-Clause "New" or "Revised" License
81 stars 30 forks source link

Error 404 on taxii-discovery and taxii-push #97

Open derwilliwonka opened 3 years ago

derwilliwonka commented 3 years ago

Hello everybody, I am stuck with this problem for quite some time now and I don't know what else I could try. I installed MISP and MISP-Taxii as per install guide except that MISP-Taxii is installed inside a venv(for some unknown reason this was the only way to install MISP-Taxii without issues). But everytime I try to push some data from otx alienvault or from hailataxii or if I try to just use the taxii-discovery command I get the following response:

This is the command I use: taxii-discovery --host localhost --port 9000 --discovery services/discovery

This is the output I get:

2021-06-17 14:28:32,098 cabby.dispatcher INFO: Sending Discovery_Request to http://localhost:9000/services/discovery

2021-06-17 14:28:32,098 cabby.dispatcher DEBUG: Request:

xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" >xmlns:tdq="http://taxii.mitre.org/query/taxii_default_query-1" message_id="f641525e-3c58-4a6a-9db6-96e81acf3e1e"/> 2021-06-17 14:28:32,100 urllib3.connectionpool DEBUG: Starting new HTTP connection (1): localhost:9000 2021-06-17 14:28:32,105 urllib3.connectionpool DEBUG: http://localhost:9000 "POST /services/discovery HTTP/1.1" 404 232 2021-06-17 14:28:32,105 cabby.dispatcher DEBUG: Response: 404 Not Found

Not Found

The requested URL was not found on the server. If you entered the URL manually please check your spelling and try >again.

2021-06-17 14:28:32,105 cabby.cli.commons ERROR: HTTP Error: status code 404 Traceback (most recent call last): File "/home/mispuser/venv/lib/python3.8/site-packages/cabby/cli/commons.py", line 174, in run_client run_func(client, args.uri, args) File "/home/mispuser/venv/lib/python3.8/site-packages/cabby/cli/discovery.py", line 7, in _runner services = client.discover_services(uri=path) File "/home/mispuser/venv/lib/python3.8/site-packages/cabby/abstract.py", line 308, in discover_services response = self._discovery_request(uri) File "/home/mispuser/venv/lib/python3.8/site-packages/cabby/client11.py", line 27, in _discovery_request response = self._execute_request(request, uri=uri) File "/home/mispuser/venv/lib/python3.8/site-packages/cabby/abstract.py", line 200, in _execute_request return do_request() File "/home/mispuser/venv/lib/python3.8/site-packages/cabby/abstract.py", line 191, in do_request return dispatcher.send_taxii_request( File "/home/mispuser/venv/lib/python3.8/site-packages/cabby/dispatcher.py", line 54, in send_taxii_request stream, headers = request_stream(session, url, request_body, timeout) File "/home/mispuser/venv/lib/python3.8/site-packages/cabby/dispatcher.py", line 91, in request_stream raise_http_error(response.status_code, response.raw) File "/home/mispuser/venv/lib/python3.8/site-packages/cabby/dispatcher.py", line 34, in raise_http_error raise HTTPError(status_code) cabby.exceptions.HTTPError: HTTP Error: status code 404

This is what the Server says: 2021-06-17T14:28:32.104688Z [werkzeug] info: 127.0.0.1 - - [17/Jun/2021 14:28:32] "POST /services/discovery HTTP/1.1" 404 -

Similiar for taxii-push but if requested I will provide this one too. Maybe I fucked up the configuration? This is config.yaml:

domain: "localhost:9000" support_basic_auth: yes

persistence_api: class: opentaxii.persistence.sqldb.SQLDatabaseAPI parameters: db_connection: mysql://taxii:some_password@localhost/taxiipersist create_tables: yes

auth_api: class: opentaxii.auth.sqldb.SQLDatabaseAPI parameters: db_connection: mysql://taxii:some_password@localhost/taxiiauth create_tables: yes secret: ChangedKey

logging: opentaxii: info root: info

hooks: misp_taxii_hooks.hooks

Sample configuration for misp_taxii_server

zmq: host: localhost port: 50000

misp: url: "http://localhost" api: "MISP-API-KEY" verifySSL: False

taxii: auth: username: taxii password: some_password collections:

  • collection

Note: I tried 2 different variants for the api key. A) sdjfhsjkdfjksdfjksd and B) "sdjfhsjkdfjksdfjksd". The secret Key is literally named "ChangedKey" but to be honest I really don't know what this key is used for and in which way I have to change it properly.

This is the data-configuration.yaml:

services:

  • id: inbox type: inbox address: /services/inbox description: Inbox Service destination_collection_required: yes accept_all_content: no authentication_required: yes supported_content:

    • urn:stix.mitre.org:xml:1.1.1
    • urn:stix.mitre.org:xml:1.2 protocol_bindings:
    • urn:taxii.mitre.org:protocol:http:1.0
    • urn:taxii.mitre.org:protocol:https:1.0

  • id: discovery type: discovery address: /services/discovery description: Discovery Service advertised_services:

    • inbox
    • discovery
    • collection_management
    • poll protocol_bindings:
    • urn:taxii.mitre.org:protocol:http:1.0
    • urn:taxii.mitre.org:protocol:https:1.0

  • id: collection_management type: collection_management address: /services/collection-management description: Collection Management Service protocol_bindings:

    • urn:taxii.mitre.org:protocol:http:1.0
    • urn:taxii.mitre.org:protocol:https:1.0

  • id: poll type: poll address: /services/poll description: Poll Service subscription_required: no max_result_count: 100 max_result_size: 10 authentication_required: yes protocol_bindings:

    • urn:taxii.mitre.org:protocol:http:1.0
    • urn:taxii.mitre.org:protocol:https:1.0

collections:

  • name: my_collection available: true accept_all_content: true type: DATA_SET

    service_ids:

    • inbox
    • collection_management
    • poll

accounts:

  • username: guest password: guest permissions: my_collection: read
  • username: taxii password: some_password is_admin: yes

I checked if opentaxii-sync-data set up the databases accordingly and yep it does. This is the output for taxiipersist and taxiiauth

MariaDB [taxiipersist]> show tables; +-----------------------------+ | Tables_in_taxiipersist | +-----------------------------+ | collection_to_content_block | | content_blocks | | data_collections | | inbox_messages | | result_sets | | service_to_collection | | services | | subscriptions | +-----------------------------+ 8 rows in set (0.000 sec)

MariaDB [taxiiauth]> select * from accounts; +----+----------+--------------------------------------------------------------------------------------------------------+----------+---------------------------+ | id | username | password_hash | is_admin | _permissions | +----+----------+--------------------------------------------------------------------------------------------------------+----------+---------------------------+ | 1 | guest | pbkdf2:sha256:260000$2R5bTOOKtVtiUn8J$b52806f3e0d94d0c60b25d378583008a5d161aed671c1125736d77bfebdfe683 | 0 | {"my_collection": "read"} | | 2 | taxii | pbkdf2:sha256:SOMEPRETTYLONGHASHVAL | 1 | {} | +----+----------+--------------------------------------------------------------------------------------------------------+----------+---------------------------+ 2 rows in set (0.000 sec)

I also checked if the port is blocked by something else but doesn't seem so. This is the output for the network check:

ss -anp | grep :9000

tcp LISTEN 0 128 127.0.0.1:9000 0.0.0.0:* users:(("python",pid=7826,fd=4),("python",pid=7826,fd=3),("opentaxii-run-d",pid=7824,fd=3))

I checked what the other processes(pid=7826) are but they belong to the venv of opentaxii and they stop using the port as soon as I stop the server. Are they blocking the port? Are they the reason why I always get a 404?

Does anybody have an idea how to solve this problem or what else I could check? Did I fuck up the installation at some point?

csoccti commented 2 years ago

Hi, did you ever manage to solve the problem?