Error: Confluence returns a format that is not JSON as expected

[GMous]

Confluence version: 8.2.3 Server Edition markdown-to-confluence installed with pip, version: 0.1.10

Confluence returns: "<Response [200]>"

Exception message:

2023-12-06 13:47:11,617 - INFO - _synchronize_page [87] - Synchronizing page: test.md
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/requests/models.py", line 971, in json
    return complexjson.loads(self.text, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/opt/homebrew/lib/python3.11/site-packages/md2conf/__main__.py", line 85, in <module>
    Application(api, ConfluenceDocumentOptions(args.generated_by)).synchronize(
  File "/opt/homebrew/lib/python3.11/site-packages/md2conf/application.py", line 34, in synchronize
    self.synchronize_page(path)
  File "/opt/homebrew/lib/python3.11/site-packages/md2conf/application.py", line 41, in synchronize_page
    self._synchronize_page(page_path, dict())
  File "/opt/homebrew/lib/python3.11/site-packages/md2conf/application.py", line 94, in _synchronize_page
    self._update_document(document, base_path)
  File "/opt/homebrew/lib/python3.11/site-packages/md2conf/application.py", line 104, in _update_document
    self.api.update_page(document.id.page_id, content)
  File "/opt/homebrew/lib/python3.11/site-packages/md2conf/api.py", line 343, in update_page
    page = self.get_page(page_id)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/md2conf/api.py", line 320, in get_page
    data = typing.cast(Dict[str, JsonType], self._invoke(path, query))
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/md2conf/api.py", line 188, in _invoke
    return response.json()
           ^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/requests/models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

[hunyadi]

Can you elaborate on what Confluence is returning? HTTP 200 OK is an expected HTTP status code. Is there no response body? If there is a body, what does it contain? (From the error message, it looks like it's not JSON.)

[GMous]

I'm not very familiar with python and confluence API, but I've just inserted "print(response)" in api.py before line 188 and I got string "<Response [200]>".

[hunyadi]

When you do a print(response), Python would print the string representation of the object held in the variable response as defined by the object type. Developers of the Requests library have likely opted to do a compact representation that prints the status code, and nothing else. What would be interesting to know is what the actual response payload is. Normally, this should be a JSON string, which is de-serialized automatically into a Python object when calling response.json(). Since the function call to response.json() is triggering an error, you would need to call print on response.text, which could produce the response body as-is, revealing what Confluence is returning.

[GMous]

Thanks for info. I've checked it and found that response.text contains HTML, indicating that my user doesn't have access to the page. However, this page is in my own user space and doesn't have any restrictions. Additionally, I've tried creating a simple custom script using the atlassian-python-api module, and it works with the same initial data (token, space key, page id).

[hunyadi]

Looking at the implementation of the class AtlassianRestAPI in the library atlassian-python-api, it looks way more complex than the basic authentication scheme available in md2conf. I would need to understand which exact authentication path is taken in AtlassianRestAPI such that it could be reproduced in the corresponding API wrapper in md2conf. For example, I could include requests-oauthlib in md2conf but we would need to identify first if this is indeed what is lacking in your case.

[GMous]

I've used authentication with a personal access token in the following way: confluence = Confluence(url=confluence_url, token=api_token)

It seems that it simply adds an "Authorization" header with the value "Bearer {token}".format(token=token).

[hunyadi]

If you don't pass a user name, md2conf will also call

session.headers.update({"Authorization": f"Bearer {self.api_key}"})

Perhaps you have a stray setting that assigns the environment variable CONFLUENCE_USER_NAME and thus invokes the HTTP basic authentication path in md2conf as opposed to the desired HTTP bearer authentication path (as in AtlassianRestAPI).

[GMous]

I was digging deep into debugging when I realised that I had missed the default value of the wiki PATH parameter. I feel so silly. Really sorry for your time. I've just added -p / to parameters and it works now.

[hunyadi]

That's great news! Don't worry; the important point is that you got it working in the end.