massdosage
commented
1 week ago This looks similar to #974 but the status code, title and detail are different.
Open massdosage opened 1 week ago
massdosage
commented
1 week ago This looks similar to #974 but the status code, title and detail are different.
jdknight
commented
1 week ago To start off, a workaround to use a more recent version of this extension, you should be able to force original requests with other new features of this extension using:
confluence_api_mode = 'v1'For sure though that it will not address the active issue mentioned. Having a 400 error being reported over a 500 one (i.e. https://github.com/sphinx-contrib/confluencebuilder/issues/974) is a bit more concerning. This extension might be performing calls in a way it should not. Just a matter of trying to reproduce the err scenario.
One step that might help is generating a report. This should help ensure development testing utilizes a similar configuration scenario.
Based on the provided output and inspecting the code, the issue appears to be specific to v2 (Cloud) property update scenario. The implementation in this extension appears to following the API call -- maybe this extension is providing a bad key/prop-ID resulting in the 400 error.
If possible/willing, if you can try running with confluence_publish_debug configured to capture headers:
confluence_publish_debug = 'headers'And just confirm on a failing PUT request that you see two unique IDs for the page and property (e.g. in the following example, 123456789 for the page and 987654321 for the property). This could rule out this extension forming a bad URL. Maybe checking if "Content-Length" is a non-zero.
PUT https://example.atlassian.net/wiki/api/v2/pages/123456789/properties/987654321 User-Agent: Sphinx Confluence Builder Accept-Encoding: gzip, deflate Accept: application/json; charset=utf-8 Connection: keep-alive X-Atlassian-Token: no-check Content-Length: 291 Content-Type: application/json ...
This extension does not have a graceful way to dump the data sent in requests (maybe we should add that).
Another question to ask is if this is a consistent or a sporadically failure? Could help it point towards a possible concurrency issue versus just a bad request.
In the mean time, I will try to perform some publication requests that try to reproduce the content property update scenario and see if this is a consistent failure (that this extension failed to properly test) or try to trigger a scenario and check to see what workarounds exist.
Using v1 API should be fine for users to fallback to now (if stuck). Ideally we want to improve v2 support as soon as we can before the cut-off date for v1 at the end of this year. I appreciate the reports and hopefully we can address this in a timely fashion.
massdosage
commented
1 week ago Thanks for the information, some answers:
I will follow the steps you suggest to try and gather more information and provide it here when I get a chance (may only be next week).
Thanks for the prompt and detailed response!
massdosage
commented
4 days ago I ran the report as requested, here is the output from that:
------------[ cut here ]------------
(system)
platform: Linux-5.13.0-1022-aws-x86_64-with-glibc2.36
python: 3.11.9 (main, Jun 13 2024, 06:17:53) [GCC 12.2.0]
sphinx: 7.3.7
docutils: 0.20.1
requests: 2.32.3
builder: 2.4.0
(configuration)
confluence_cleanup_from_root: True
confluence_cleanup_purge: True
confluence_disable_notifications: True
confluence_ignore_titlefix_on_index: True
confluence_page_generation_notice: True
confluence_page_hierarchy: True
confluence_parent_page: (set)
confluence_publish: False
confluence_publish_dryrun: True
confluence_publish_prefix: REDACTED (It's of the form `word word -`)
confluence_server_pass: (set)
confluence_server_url: https://<removed> (cloud)
confluence_server_user: (set)
confluence_sourcelink: {'type': 'gitlab', 'host': 'gitlab.REDACTED.com', 'owner': 'REDACTED', 'repo': 'REDACTED', 'version': '0.1.0'}
confluence_space_key: (set; upper)
------------[ cut here ]------------I also set confluence_publish_debug = 'headers' and this is the output from the failing call (there are 200 OKs before this from other pages):
(debug) Request]
PUT https://REDACTED.atlassian.net/wiki/api/v2/pages/REDACTED_83/properties/REDACTED_92
User-Agent: Sphinx Confluence Builder
Accept-Encoding: gzip, deflate
Accept: application/json; charset=utf-8
Connection: keep-alive
X-Atlassian-Token: no-check
Content-Length: 177
Content-Type: application/json
Authorization: Basic REDACTED_TOKEN=
(debug) Response]
Code: 400
Date: Mon, 01 Jul 2024 11:06:18 GMT
Content-Type: application/json;charset=utf-8
Server: AtlassianEdge
Atl-Confluence-Via: h:confluence-REDACTED.net
X-Content-Type-Options: nosniff
X-Xss-Protection: 1; mode=block
Atl-Traceid: REDACTED
Strict-Transport-Security: max-age=63072000; includeSubDomains; preload
Report-To: {"endpoints": [{"url": "https://REDACTED.cloudfront.net"}], "group": "endpoint-1", "include_subdomains": true, "max_age": 600}
Nel: {"failure_fraction": 0.001, "include_subdomains": true, "max_age": 600, "report_to": "endpoint-1"}
Vary: Accept-Encoding
Content-Encoding: gzip
Transfer-Encoding: chunked
sphinxcontrib.confluencebuilder error:
---
Unsupported Confluence API call
An unsupported Confluence API call has been made. See the following
details for more information:
REQ: PUT
RSP: 400
URL: https://REDACTED.atlassian.net/wiki/
API: api/v2/pages/REDACTED_83/properties
DATA: {
"errors": [
{
"status": 400,
"code": "INVALID_REQUEST_PARAMETER",
"title": "Cannot update content property. Property : JsonContentProperty{id='JsonContentPropertyId{id=REDACTED_92}', key='content-appearance-published', content=ExpandedReference{idProperties={id=ContentId{id=REDACTED_83}, status=current, version=CollapsedReference{idProperties={number=213}, referentClass=class com.atlassian.confluence.api.model.content.Version}}, of=some(Content{id='ContentId{id=REDACTED_83}', type=page, title='REDACTED', status=current, space=CollapsedReference{idProperties={key=DATA}, referentClass=class com.atlassian.confluence.api.model.content.Space}, history=CollapsedReference{idProperties={contentParent=CollapsedReference{idProperties={id=ContentId{id=REDACTED_83}, status=current, version=CollapsedReference{idProperties={number=213}, referentClass=class com.atlassian.confluence.api.model.content.Version}}, referentClass=class com.atlassian.confluence.api.model.content.Content}}, referentClass=class com.atlassian.confluence.api.model.content.History}, version=CollapsedReference{idProperties={number=213}, referentClass=class com.atlassian.confluence.api.model.content.Version}, ancestors=null (CollapsedList), container=CollapsedReference{idProperties={key=DATA}, referentClass=class com.atlassian.confluence.api.model.content.Space}, childTypes=null (CollapsedMap)})}, version=Version{by=null, when=null, message='', number=2, syncRev='null', syncRevSource='null', ncsStepVersion='null', ncsStepVersionSource='null', confRev='null', content=EmptyReference{referentClass=class com.atlassian.confluence.api.model.content.Content}}} , isSuccessful: false, isAuthorized: true, isValid: false, hasErrors: true, Errors: \"jsonproperty.invalid.id\"",
"detail": null
}
]
}
---Thanks for the information so far. I tried reproducing a similar configuration set to see if it would help yield anything but no luck yet. I see in the report it does indicate an older builder (builder: 2.4.0), but I am going to assume that you may have generated a report on a local instance instance of a CI instance (which is fine; just mentioning just in case).
Looking at the outputs of the original and new responses look to show the issue is failing on a page's content property update. Specifically, trying to update a property content-appearance-published.
{
"status": 400,
"code": "INVALID_REQUEST_PARAMETER",
"title": "Cannot update content property. Property :
JsonContentProperty{
id='JsonContentPropertyId{id=REDACTED_92}',
key='content-appearance-published',
content=ExpandedReference{
idProperties={
id=ContentId{id=REDACTED_83},
status=current,
version=CollapsedReference{
idProperties={number=213},
referentClass=class com.atlassian.confluence.api.model.content.Version}
},
of=some(Content{
id='ContentId{id=REDACTED_83}',
type=page,
title='REDACTED',
status=current,
space=CollapsedReference{idProperties={key=DATA},
referentClass=class com.atlassian.confluence.api.model.content.Space
},
history=CollapsedReference{
idProperties={
contentParent=CollapsedReference{
idProperties={
id=ContentId{id=REDACTED_83},
status=current,
version=CollapsedReference{
idProperties={number=213},
referentClass=class com.atlassian.confluence.api.model.content.Version
}
},
referentClass=class com.atlassian.confluence.api.model.content.Content
}
},
referentClass=class com.atlassian.confluence.api.model.content.History
},
version=CollapsedReference{
idProperties={number=213},
referentClass=class com.atlassian.confluence.api.model.content.Version
},
ancestors=null (CollapsedList),
container=CollapsedReference{
idProperties={key=DATA},
referentClass=class com.atlassian.confluence.api.model.content.Space
},
childTypes=null (CollapsedMap)
})
},
version=Version{
by=null,
when=null,
message='',
number=2,
syncRev='null',
syncRevSource='null',
ncsStepVersion='null',
ncsStepVersionSource='null',
confRev='null',
content=EmptyReference{
referentClass=class com.atlassian.confluence.api.model.content.Content
}
}
},
isSuccessful: false,
isAuthorized: true,
isValid: false,
hasErrors: true,
Errors: \"jsonproperty.invalid.id\"",
"detail": null
}
The details of this error show a similar structure to that of the PUT request. When updating the content-appearance-published property, a request like the following is made:
{
'key': 'content-appearance-published',
'value': 'default',
'version': {
'number': 5
},
'id': '123456789'
}What I find interesting is that I do not see a hint related to the value configuration (e.g. default) in the 400 details output. It very well could be that a dump would not include this information, but hard to know having access to Confluence's code to check.
A current suspicion is that maybe the extension is re-pushing the content-appearance-publish property that it originally fetched back to Confluence (ideally, should not happen if there is "no change"), and that the value of content-appearance-publish now is either in an odd state or cannot be reused between the GET and PUT events.
Would you mind trying to open the following URL for the site and page where the error occurs?
https://YOUR_SITE.atlassian.net/wiki/api/v2/pages/YOUR_PAGE_ID/properties?key=content-appearance-publishedYou should see an output like this:
{
"results": [
{
"value": "default",
"key": "content-appearance-published",
"id": "123456789",
"version": {
"number": 5,
"message": "",
"minorEdit": false,
"authorId": "123456789:<guid>",
"createdAt": "2024-07-01T15:30:11.904Z"
}
}
],
"_links": {
"base": "https://YOUR_SITE.atlassian.net/wiki"
}
}And I'm curious to know if the output of value is something not expected (most likely will be sane, but want to rule this out).
The only other suggestion I have at this time is maybe try running with the development installation and try to set the following option:
confluence_publish_debug = 'headers_and_data'This should output the contents of the PUT request made which might real if this extension is sending a malformed request out.
massdosage
commented
3 days ago This is all running on a CI instance.
I tried hitting the URL https://YOUR_SITE.atlassian.net/wiki/api/v2/pages/YOUR_PAGE_ID/properties?key=content-appearance-published directly as suggested and it returned output pretty much exactly like what you said above. value was set to default.
Then, instead of installing sphinxcontrib-confluencebuilder==2.5.2 I did pip install git+https://github.com/sphinx-contrib/confluencebuilder.git to get the latest version and then the build succeeded. So I guess whatever the issue is, it's been fixed by whatever has been changed between 2.5.2 and what is now in main. So this will probably work in the next release which is good news!
jdknight
commented
2 days ago Gah ~ well.... glad the development revision works. Maybe it was the https://github.com/sphinx-contrib/confluencebuilder/commit/8bd8137178d65bd35040020fdaa1fb724aa8e2a0 commit? Sorry for the issues.
Will flag as "available-next-release" with the assumption that the now working state was related to a fix in the development branch. We should be building a release soon (maybe this or next weekend; no guarantees yet) to wrap this up. In the interim, if you do see any other issues, please let me know.
massdosage
commented
2 days ago No worries, thanks for looking into this and all the prompt responses. If you want to ping me here when the release comes out I can then try it out and report back if its fixed. No rush on our side as we'll just stick with 2.4.0 for now.
We use Sphinx and this extension to convert a number of
.mdand.rstfiles to HTML and then upload the files to Confluence. We have used this across a wide variety of projects with success for a number of years. We recently moved to the2.5.0/1/2versions and consistently get the below error which goes away if we downgrade to version2.4.0. Looking at the CHANGELOG my guess is there is some bug talking to one of thev2APIs. Happy to provide more information or run certain follow up steps if that would help you debug the root cause.