failed to get data from MISP after upgrade to 2.4.144

[baymax1908]

OS: Centos 7 MISP: 2.4.144 pymisp: 2.4.144

The script is working fine. After upgrade to MISP 2.4.144. It failed.

Test script:

misp_verifycert = True relative_path = 'users/statistics/users' body = []

from pymisp import ExpandedPyMISP

misp = ExpandedPyMISP(misp_url, misp_key, misp_verifycert) misp.direct_call(relative_path, body)

Error message:

Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/pymisp/api.py", line 3432, in _check_response response_json = response.json() File "/usr/local/lib/python3.6/site-packages/requests/models.py", line 900, in json return complexjson.loads(self.text, **kwargs) File "/usr/lib64/python3.6/site-packages/simplejson/init.py", line 518, in loads return _default_decoder.decode(s) File "/usr/lib64/python3.6/site-packages/simplejson/decoder.py", line 370, in decode obj, end = self.raw_decode(s) File "/usr/lib64/python3.6/site-packages/simplejson/decoder.py", line 400, in raw_decode return self.scan_once(s, idx=_w(s, idx).end()) simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/pymisp/api.py", line 171, in init misp_version = self.misp_instance_version File "/usr/local/lib/python3.6/site-packages/pymisp/api.py", line 242, in misp_instance_version return self._check_json_response(response) File "/usr/local/lib/python3.6/site-packages/pymisp/api.py", line 3400, in _check_json_response r = self._check_response(response, expect_json=True) File "/usr/local/lib/python3.6/site-packages/pymisp/api.py", line 3442, in _check_response raise PyMISPUnexpectedResponse(error_msg) pymisp.exceptions.PyMISPUnexpectedResponse: Unexpected response (size: 3335) from server:

Notice (8): Undefined index: perm_galaxy_editor [APP/Controller/ServersController.php, line 1808]
Code Context
            'perm_sync' => $this->userRole['perm_sync'],
            'perm_sighting' => $this->userRole['perm_sighting'],
            'perm_galaxy_editor' => $this->userRole['perm_galaxy_editor'],
$versionArray = array(
'major' => (int) 2,
'minor' => (int) 4,
'hotfix' => (int) 144
)
ServersController::getVersion() - APP/Controller/ServersController.php, line 1808
ReflectionMethod::invokeArgs() - [internal], line ??
Controller::invokeAction() - APP/Lib/cakephp/lib/Cake/Controller/Controller.php, line 499
Dispatcher::_invoke() - APP/Lib/cakephp/lib/Cake/Routing/Dispatcher.php, line 193
Dispatcher::dispatch() - APP/Lib/cakephp/lib/Cake/Routing/Dispatcher.php, line 167
[main] - APP/webroot/index.php, line 92

{"version":"2.4.144","perm_sync":true,"perm_sighting":true,"perm_galaxy_editor":null,"request_encoding":["gzip"]}

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "new.py", line 11, in misp = ExpandedPyMISP(misp_url, misp_key, misp_verifycert) File "/usr/local/lib/python3.6/site-packages/pymisp/api.py", line 181, in init raise PyMISPError(f'Unable to connect to MISP ({self.root_url}). Please make sure the API key and the URL are correct (http/https is required): {e}') pymisp.exceptions.PyMISPError: Unable to connect to MISP (FQDN). Please make sure the API key and the URL are correct (http/https is required): Unexpected response (size: 3335) from server:

Notice (8): Undefined index: perm_galaxy_editor [APP/Controller/ServersController.php, line 1808]
Code Context
            'perm_sync' => $this->userRole['perm_sync'],
            'perm_sighting' => $this->userRole['perm_sighting'],
            'perm_galaxy_editor' => $this->userRole['perm_galaxy_editor'],
$versionArray = array(
'major' => (int) 2,
'minor' => (int) 4,
'hotfix' => (int) 144
)
ServersController::getVersion() - APP/Controller/ServersController.php, line 1808
ReflectionMethod::invokeArgs() - [internal], line ??
Controller::invokeAction() - APP/Lib/cakephp/lib/Cake/Controller/Controller.php, line 499
Dispatcher::_invoke() - APP/Lib/cakephp/lib/Cake/Routing/Dispatcher.php, line 193
Dispatcher::dispatch() - APP/Lib/cakephp/lib/Cake/Routing/Dispatcher.php, line 167
[main] - APP/webroot/index.php, line 92

{"version":"2.4.144","perm_sync":true,"perm_sighting":true,"perm_galaxy_editor":null,"request_encoding":["gzip"]}

[baymax1908]

I can get the data by postman. By python, seems it didn't parse the result correctly.

[Rafiot]

Yeah, MISP is returning a stacktrace, so PyMISP fails to parse the json.

@mokaddem @iglocska that's something for you, I think?

[baymax1908]

Fixed this issue by changing the getVersion() function to old version.

public function getVersion() { if (!$this->userRole['perm_auth']) { throw new MethodNotAllowedException('This action requires API access.'); } $versionArray = $this->Server->checkMISPVersion(); $this->set('response', array('version' => $versionArray['major'] . '.' . $versionArray['minor'] . '.' . $versionArray['hotfix'], 'perm_sync' => $this->userRole['perm_sync'], 'perm_sighting' => $this->userRole['perm_sighting'])); $this->set('_serialize', 'response'); }

[Rafiot]

The test suite is passing on the github actions so I'm not sure what you're talking about.

[mokaddem]

@baymax1908 could you check the diagnostic page in MISP? Especially the database schema diagnostic. Thanks!