MISP - integration - invalid GET instead of POST request is being sent

[Kaloszer]

What happened

Added MISP connector, tried to test it. Fail :(

This issue might be relevant to this PR https://github.com/intelowlproject/IntelOwl/pull/2164

As it mentions this error message. Not sure whether this was merged to the version I'm on though.

Environment

1. OS: Ubuntu 22.04 LTS
2. IntelOwl version: v6.0.4
3. MISP v2.4.195

What did you expect to happen

MISP is able to retrieve information from IP

How to reproduce your issue

Setup MISP try to request information about an IP Same VNET, internal IP connectivity ok. Test connection health OK

Error messages and logs

Happens both for Connector and Analyzer

{
    "id": 7,
    "user": {
        "username": "intelowl"
    },
    "tags": [],
    "comments": [],
    "pivots_to_execute": [],
    "analyzers_to_execute": [
        "MISP"
    ],
    "analyzers_requested": [
        "MISP"
    ],
    "connectors_to_execute": [],
    "connectors_requested": [],
    "visualizers_to_execute": [],
    "playbook_requested": null,
    "playbook_to_execute": null,
    "investigation": null,
    "permissions": {
        "kill": true,
        "delete": true,
        "plugin_actions": true
    },
    "analyzer_reports": [
        {
            "name": "MISP",
            "process_time": 0.49,
            "status": "FAILED",
            "end_time": "2024-08-20T10:51:36.404298Z",
            "parameters": {
                "debug": true,
                "limit": 50,
                "timeout": 5,
                "metadata": false,
                "from_days": 90,
                "published": false,
                "ssl_check": false,
                "strict_search": true,
                "filter_on_type": true,
                "enforce_warninglist": true,
                "self_signed_certificate": false
            },
            "type": "analyzer",
            "id": 49,
            "report": {},
            "errors": [
                "(400, {'name': 'Restsearch queries using GET and no parameters are not allowed. If you have passed parameters via a JSON body, make sure you use POST requests.', 'message': 'Restsearch queries using GET and no parameters are not allowed. If you have passed parameters via a JSON body, make sure you use POST requests.', 'url': '/events/restSearch'})"
            ],
            "start_time": "2024-08-20T10:51:35.914034Z",
            "description": "scan an observable on a custom MISP instance"
        }
    ],
    "connector_reports": [],
    "pivot_reports": [],
    "visualizer_reports": [],
    "is_sample": false,
    "md5": "29ab9d23ec079e98ebb02a7e2f43b378",
    "observable_name": "178.32.53.124",
    "observable_classification": "ip",
    "file_name": "",
    "file_mimetype": "",
    "status": "failed",
    "runtime_configuration": {
        "pivots": {},
        "analyzers": {},
        "connectors": {},
        "visualizers": {}
    },
    "received_request_time": "2024-08-20T10:36:46.060672Z",
    "finished_analysis_time": "2024-08-20T10:51:36.481039Z",
    "process_time": 890.42,
    "tlp": "AMBER",
    "errors": [],
    "warnings": [],
    "scan_mode": 2,
    "scan_check_time": "1 00:00:00"
}

[Kaloszer]

I'll check whether develop branch has this resolved tommorow and note results.

[mlodic]

thanks for reporting. We are using the official pymisp library so this surprises me honestly. @g4ze will investigate it shortly

[mlodic]

we have been trying to address it here: https://github.com/intelowlproject/IntelOwl/pull/2481 but we still need to test it with a misp instance. I am not sure whether that works. Most probably is something related to the pymisp library itself. I saw other similar bug reports in the pymisp library that are still unsolved.

[Kaloszer]

we have been trying to address it here: #2481 but we still need to test it with a misp instance. I am not sure whether that works. Most probably is something related to the pymisp library itself. I saw other similar bug reports in the pymisp library that are still unsolved.

Would just modifying said files and doing the ./start test up -- --build work? To test it I mean

EDIT: Nope - but I can see that the pymisp library wasnt updated to ~.915 - not sure how to force it to upgrade as i tried ./start test build -- --no-cache ./start test up -- --build

But it still complains that:

intelowl_celery_worker_default  | The version of PyMISP recommended by the MISP instance (2.4.195) is newer than the one you're using now (2.4.190). Please upgrade PyMISP.
intelowl_celery_worker_default  | Something went wrong (400): {'name': 'Restsearch queries using GET and no parameters are not allowed. If you have passed parameters via a JSON body, make sure you use POST requests.', 'message': 'Restsearch queries using GET and no parameters are not allowed. If you have passed parameters via a JSON body, make sure you use POST requests.', 'url': '/events/restSearch'}

Any tips?

[mlodic]

I have just tried with a local Instance of PyMISP created from here, v 2.4.192 with pymisp version 2.4.195. I created a sample event, published with a sample ioc. I looked for that IOC with the MISP analyzer, without any additional configuration. Everything worked, the IOC was found and got me the results back.

Basically, that tells me that there's something wrong in yout environment.

Can you please tell me how you configured your MISP analyzer? (if there's private data, either obfuscate it or you can contact me directly via Twitter if you like)

@g4ze can you help us sharing your configuration too considering you are getting the same error?

[g4ze]

I got the same error months ago, I don't have misp setup now...

[Kaloszer]

@mlodic I don't think I have added anything other than the default setup for MISP + defaults: https://github.com/MISP/MISP/blob/2.4/app/files/feed-metadata/defaults.json

What I think is amiss here is that my docker PyMISP is at (2.4.190) and not (2.4.195). I'm kind of green in the docker area so not sure how to force it to update, should I just rebuild the VM and reinitialize the project from the PR branch?

[mlodic]

if you go in the develop branch, you can ./start test up -- --build, it would use the most recent version released yesterday (2.4.196). If you tried yesterday from the develop branch, the pymisp version was the 2.4.190 so this may align with what you said. I have just made a commit in the develop branch with the new version. Anyway, once you build with the test option, you can customize the project-requirements as you wish and then rebuild again

[Kaloszer]

@mlodic Yep, tried it seems that it is updated now, however I'm seeing the same thing.

My MISP setup is not exposed so it's over http - SSL flag is disabled in IntelOwl.

IntelOwl

Something went wrong (400): {'name': 'Restsearch queries using GET and no parameters are not allowed. If you have passed parameters via a JSON body, make sure you use POST requests.', 'message': 'Restsearch queries using GET and no parameters are not allowed. If you have passed parameters via a JSON body, make sure you use POST requests.', 'url': '/events/restSearch'}

MISP error log:

2024-08-22 11:54:18 Error: [BadRequestException] Restsearch queries using GET and no parameters are not allowed. If you have passed parameters via a JSON body, make sure you use POST requests.
Request URL: /events/restSearch
Stack Trace:
#0 [internal function]: AppController->restSearch()
#1 /var/www/MISP/app/Lib/cakephp/lib/Cake/Controller/Controller.php(499): ReflectionMethod->invokeArgs()
#2 /var/www/MISP/app/Lib/cakephp/lib/Cake/Routing/Dispatcher.php(193): Controller->invokeAction()
#3 /var/www/MISP/app/Lib/cakephp/lib/Cake/Routing/Dispatcher.php(167): Dispatcher->_invoke()
#4 /var/www/MISP/app/webroot/index.php(101): Dispatcher->dispatch()
#5 {main}

Audit logs seem to show that no actual content is being passed? Odd:

Am I just being a dummy and using the tool wrong?

MISP config as follows in IntelOwl:

Healthcheck reports OK

[mlodic]

try to set the parameter ssl_check of the MISP analyzer/connector to False. This seems the only difference with my test environment.

[mlodic]

ah no I read now that maybe you have already flagged it....ok so I have no idea, I can't replicate the problem. Everything you are doing is fine.

[Kaloszer]

@mlodic - dumb question, how would I go about logging what is being send to pyMisp and then from pyMISP to MISP? Seems that logging stuff does not show up in the docker log that I can see? I'd then try to post the message with postman and see what gives. If it's the same then it must be my MISP instance playing tricks