search

kovetskiy / mark

Sync your markdown files with Confluence pages.
https://samizdat.dev
Apache License 2.0
1.02k stars 153 forks source link

Sync failure on image #94

Closed mantzas closed 2 years ago

mantzas commented 3 years ago

Hi,

when creating a page with image attachment the process works perfectly fine. After updating the image attachment and running sync again we face a lot of times failures with the following output: 2021-07-09 16:54:30.450 FATAL Confluence API returned unexpected status: 500 , output: "" The workaround is to delete the attachment in Confluence and sync again. Is this something other face also? Thanks

kovetskiy commented 3 years ago

Hi, I haven't seen this behavior before, could you provide steps to reproduce it?

mantzas commented 3 years ago

The steps are pretty simple and that's why I am puzzled by it.

The command used in both sync is (obviously I have changed the arguments):

mark -u username -p password -b https://confluence.company.com -f README.md

and when I sync the second time, with and existing image attachment, it fails with:

2021-07-14 12:58:42.057 INFO  processing README.md
2021-07-14 12:58:45.384 INFO  page will be stored under path: Page1 > Page2 > Page3 > Page4
2021-07-14 12:58:45.479 INFO  updating attachment: "doc/architecture/image.png"
2021-07-14 12:58:46.447 FATAL unable to create/update attachments
                              └─ unable to update attachment "doc/architecture/image.png"
                                 └─ Confluence REST API for creating attachments returned 0 json objects, expected at least 1

this is the header of the markdown:

<!-- Space: DT -->
<!-- Title: Page1 -->
<!-- Parent: Page2 -->
<!-- Parent: Page3 -->
<!-- Attachment: doc/architecture/image.png -->

and this is how the image is embedded in the markdown:

<img alt="image" src="doc/architecture/image.png" width="1187" />

Hope this helps

mantzas commented 3 years ago

BTW this happens even if i use the markdown way of inlining images

![Image](doc/architecture/image.png)
kovetskiy commented 3 years ago

You are using a hosted version of confluence right? Which version you are on?

mantzas commented 3 years ago

Powered by Atlassian Confluence 6.15.2

rafadc commented 3 years ago

I am having the same problem. I run an example response with --trace. Could it be related to versioning of the attachments and the payload being different? When you run it the second time it looks to me like you are trying to create a second version instead of creating a completely new attachment. 

{
   "id":"att3037331492",
   "type":"attachment",
   "status":"current",
   "title":"whatever.png",
   "version":{
      "by":{
         "type":"known",
         "accountId":"5c9cc150691f0c0d01e35efa",
         "accountType":"atlassian",
         "email":"myname@whatever.com",
         "publicName":"Rafael de Castro",
         "profilePicture":{
            "path":"/wiki/aa-avatar/5c9cc150691f0c0d01e35efa",
            "width":48,
            "height":48,
            "isDefault":false
         },
         "displayName":"Rafael de Castro",
         "isExternalCollaborator":false,
         "_expandable":{
            "operations":"",
            "personalSpace":""
         },
         "_links":{
            "self":"https://whatever.atlassian.net/wiki/rest/api/user?accountId=5c9cc150691f0c0d01e35efa"
         }
      },
      "when":"2021-08-24T07:39:01.051Z",
      "friendlyWhen":"just a moment ago",
      "message":"mark:checksum: 7060f2fedd164e0426f0f53868686dd102945b9d02530e780efa1418f8d64477",
      "number":2,
      "minorEdit":false,
      "contentTypeModified":false,
      "_expandable":{
         "collaborators":"",
         "content":"/rest/api/content/att3037331492"
      },
      "_links":{
         "self":"https://whatever.atlassian.net/wiki/rest/api/content/att3037331492/version/2"
      }
   },
   "container":{
      "id":"3035595539",
      "type":"page",
      "status":"current",
      "title":"Drivers convoy",
      "macroRenderedOutput":{

      },
      "extensions":{
         "position":1190
      },
      "_expandable":{
         "container":"/rest/api/space/EN",
         "metadata":"",
         "restrictions":"/rest/api/content/3035595539/restriction/byOperation",
         "history":"/rest/api/content/3035595539/history",
         "body":"",
         "version":"",
         "descendants":"/rest/api/content/3035595539/descendant",
         "space":"/rest/api/space/EN",
         "childTypes":"",
         "operations":"",
         "schedulePublishDate":"",
         "children":"/rest/api/content/3035595539/child",
         "ancestors":""
      },
      "_links":{
         "self":"https://whatever.atlassian.net/wiki/rest/api/content/3035595539",
         "tinyui":"/x/E4Pvt",
         "editui":"/pages/resumedraft.action?draftId=3035595539",
         "webui":"/spaces/EN/pages/3035595539/Drivers+convoy"
      }
   },
   "macroRenderedOutput":{

   },
   "metadata":{
      "comment":"mark:checksum: 7060f2fedd164e0426f0f53868686dd102945b9d02530e780efa1418f8d64477",
      "mediaType":"image/png",
      "labels":{
         "results":[

         ],
         "start":0,
         "limit":200,
         "size":0,
         "_links":{
            "next":"/rest/api/content/att3037331492/label?next=true&limit=200&start=200",
            "self":"https://whatever.atlassian.net/wiki/rest/api/content/att3037331492/label"
         }
      },
      "_expandable":{
         "currentuser":"",
         "comments":"",
         "simple":"",
         "properties":"",
         "frontend":"",
         "likes":""
      }
   },
   "extensions":{
      "mediaType":"image/png",
      "fileSize":115132,
      "comment":"mark:checksum: 7060f2fedd164e0426f0f53868686dd102945b9d02530e780efa1418f8d64477",
      "mediaTypeDescription":"PNG Image",
      "fileId":"e5c99f3b-e4d0-4df8-aef4-59a3761a5735"
   },
   "_expandable":{
      "childTypes":"",
      "operations":"",
      "schedulePublishDate":"",
      "children":"/rest/api/content/att3037331492/child",
      "restrictions":"/rest/api/content/att3037331492/restriction/byOperation",
      "history":"/rest/api/content/att3037331492/history",
      "ancestors":"",
      "body":"",
      "descendants":"/rest/api/content/att3037331492/descendant",
      "space":"/rest/api/space/EN"
   },
   "_links":{
      "context":"/wiki",
      "self":"https://whatever.atlassian.net/wiki/rest/api/content/att3037331492",
      "download":"/download/attachments/3035595539/whatever.png?version=2&modificationDate=1629790741051&cacheVersion=1&api=v2",
      "collection":"/rest/api/content",
      "webui":"/pages/viewpageattachments.action?pageId=3035595539&preview=%2F3035595539%2F3037331492%2Fwhatever.png",
      "base":"https://whatever.atlassian.net/wiki"
   }
}
rafadc commented 3 years ago

Ok. I am debugging this a bit and looks like sometimes the API returns a 

{ "results": [attachment] }

and sometimes

{ attachment }

straight on.

The docs on the hosted version do not specify the return format.

I'll try to investigate why this is happening.

rafadc commented 3 years ago

Looks like the return for the endpoint for updating data is different. In hosted this version of the function worked for me

func (api *API) UpdateAttachment(
    pageID string,
    attachID string,
    name string,
    comment string,
    path string,
) (AttachmentInfo, error) {
    var info AttachmentInfo

    form, err := getAttachmentPayload(name, comment, path)
    if err != nil {
        return AttachmentInfo{}, err
    }

    var result AttachmentInfo

    resource := api.rest.Res(
        "content/"+pageID+"/child/attachment/"+attachID+"/data", &result,
    )

    resource.Payload = form.buffer
    resource.Headers = http.Header{}

    resource.SetHeader("Content-Type", form.writer.FormDataContentType())
    resource.SetHeader("X-Atlassian-Token", "no-check")

    request, err := resource.Post()
    if err != nil {
        return info, err
    }

    if request.Raw.StatusCode != 200 {
        return info, newErrorStatusNotOK(request)
    }

    info = result

    return info, nil
}

I created a docker image at https://hub.docker.com/r/rafadc/mark in case @mantzas you want to check if this is what is happening to you.

TBH I got no clue on how to do this check to make this upstream. Should I check if we are on the cloud to get a different response?

Dosenwerfer commented 3 years ago

I am also getting this issue ("Confluence REST API for creating attachments returned 0 json objects, expected at least 1"), but it is not only limited to image attachments. It also happens with text attachments. I assume it applies to any type of attachment.

Confluence version: 7.13.0 Mark version: 6.4

It just seems to be an API response interpretation/logging issue though since the changed attachment actually correctly updates in Confluence despite the "FATAL" console error. Accordingly, this doesn't happen anymore when running a second mark pass because then the attachment is detected as unchanged since it was correctly updated in the first pass.

I didn't check if rafadc's snippet fixes this because I currently don't have the resources to build my own mark version.

mac89 commented 2 years ago

It's an inconsistent issue. On my CI it tends to fail more often than not. Locally it works more of the time. One time it fails on one attachment, the next time on another.

> Sync README.md
2021-12-08 15:39:09.619 INFO  processing README.md
2021-12-08 15:39:11.049 INFO  page will be stored under path: Production and Maintenance > Production and Maintenance (P&M) > di-sam-api: DB schema
2021-12-08 15:39:11.378 INFO  page successfully updated: ******/spaces/SM/pages/2220294285/di-sam-api%3A+DB+schema
******/spaces/SM/pages/2220294285/di-sam-api%3A+DB+schema

> Sync SWAGGER.md
2021-12-08 15:39:11.405 INFO  processing SWAGGER.md
2021-12-08 15:39:11.736 INFO  page will be stored under path: Production and Maintenance > Production and Maintenance (P&M) > di-sam-api: Swagger Spec
******/spaces/SM/pages/2206403591/di-sam-api%3A+Swagger+Spec
2021-12-08 15:39:12.069 INFO  page successfully updated: ******/spaces/SM/pages/2206403591/di-sam-api%3A+Swagger+Spec

> Sync _schema.md
2021-12-08 15:39:12.124 INFO  processing _schema.md
2021-12-08 15:39:12.431 INFO  page will be stored under path: Production and Maintenance > Production and Maintenance (P&M) > di-sam-api: DB schema > _schema.md
2021-12-08 15:39:12.721 INFO  page successfully updated: ******/spaces/SM/pages/2220261678/_schema.md
******/spaces/SM/pages/2220261678/_schema.md

> Sync asset.md
2021-12-08 15:39:12.742 INFO  processing asset.md
2021-12-08 15:39:13.240 INFO  page will be stored under path: Production and Maintenance > Production and Maintenance (P&M) > di-sam-api: DB schema > asset.md
2021-12-08 15:39:13.298 INFO  updating attachment: "asset.svg"
2021-12-08 15:39:13.799 FATAL unable to create/update attachments
                              └─ unable to update attachment "asset.svg"
                                 └─ Confluence REST API for creating attachments returned 0 json objects, expected at least 1
2021-12-08 15:43:44.209 INFO  processing README.md
2021-12-08 15:43:45.748 INFO  page will be stored under path: Production and Maintenance > Production and Maintenance (P&M) > di-sam-api: DB schema
2021-12-08 15:43:45.808 INFO  updating attachment: "schema.svg"
2021-12-08 15:43:46.317 FATAL unable to create/update attachments
                              └─ unable to update attachment "schema.svg"
                                 └─ Confluence REST API for creating attachments returned 0 json objects, expected at least 1
kovetskiy commented 2 years ago

Please check version 6.7 it should be fixed

mac89 commented 2 years ago

Please check version 6.7 it should be fixed

Looks to be working. Great work!

mantzas commented 2 years ago

Seems to be working great. Thanks, closing the issue.