socfortress / CoPilot

SOCFortress CoPilot
https://www.socfortress.co
GNU Affero General Public License v3.0
194 stars 40 forks source link

Graylog Connector: Sometimes http status code is 200 but there is no returned content #277

Closed ngocngoan closed 3 weeks ago

ngocngoan commented 1 month ago

Describe the bug When I send an HTTP POST request to the Graylog API /api/entitylists/preferences/saved-searches, then the error happens:

500: Failed to send POST request to /api/entitylists/preferences/saved-searches with error: Expecting value: line 1 column 1 (char 0)

To Reproduce Steps to reproduce the behavior:

  1. Import function send_post_request() from Graylog universal module
  2. Send HTTP POST request by instrustion response = await send_post_request( endpoint="/api/entitylists/preferences/saved-searches", data=preferences.model_dump(by_alias=True, exclude_unset=True, mode="json"), )
  3. The error shows in the log

Failed to update saved search preference: 500: Failed to send POST request to /api/entitylists/preferences/saved-searches with error: Expecting value: line 1 column 1 (char 0)

  1. See the following images:

Screenshots

Screenshot 2024-08-29 163558

Screenshot 2024-08-29 163608

Screenshot 2024-08-29 163617

  1. The reason

The response text is empty string. It is the reason why the response.json() command generates the above error.

Screenshots

Screenshot 2024-08-29 170322

Expected behavior

Update line 168 from response.json() to response.json() if response.text else None

Container Logs

Traceback (most recent call last):
  File "/path/to/lib/python3.12/site-packages/requests/models.py", line 974, in json
    return complexjson.loads(self.text, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/path/to/lib/python3.12/site-packages/simplejson/__init__.py", line 514, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/path/to/lib/python3.12/site-packages/simplejson/decoder.py", line 386, in decode
    obj, end = self.raw_decode(s)
               ^^^^^^^^^^^^^^^^^^
  File "/path/to/lib/python3.12/site-packages/simplejson/decoder.py", line 416, in raw_decode
    return self.scan_once(s, idx=_w(s, idx).end())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/path/to/lib/python3.12/site-packages/requests/models.py", line 978, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
taylorwalton commented 1 month ago

Hey @ngocngoan Thanks for the in depth troubleshooting and recommended fix, that really helps! I understand that you are experiencing an issue when sending a POST to Graylog using the send_post_request function.

I have no problem implementing your recommend fix, and thanks for providing that. I am curious though as to what intention you have with fetching /api/entitylists/preferences/saved-searches from Graylog as we are not currently leveraging this Graylog endpoint within CoPilot. Could you think of a use case where it would make sense to add some frontend support for the API endpoint of /api/entitylists/preferences/saved-searches?