search

alcionai / corso

Free, Secure, and Open-Source Backup for Microsoft 365
https://corsobackup.io
Apache License 2.0
181 stars 28 forks source link

[Bug]: Entire mailbox fails on single attachment #5419

Open szivic opened 4 weeks ago

szivic commented 4 weeks ago

What happened?

When a backup fails on getting a single mail attachment, there is no output for that mailbox backup and the entire mailbox backup fails.

We run a backup for 50 mailboxes, one of them fails on an attachment, the other 49 run fine. One of those mailboxes has an email with an attachment that cannot be found. This causes the “unable to complete backup” error (on every run), thus leaving us without any data from that mailbox and no way to restore anything from it.

Below the graph logs to get the attachments for the email. It shows 9 attachments, the last two seem to be empty.
When viewing the email in the mailbox, I found 11 attachments: 7 png-files (these were visible in the graph call), 2 vcf cards and 2 (empty) eml files. I’ve been unable to retrieve the two empty attachments through the graph api, log below.

Instead of erroring out the mailbox completely, wouldn’t it be possible to skip the item that causes the error so that the backup can complete? That would mean only losing a small amount of data (the specific attachment or the email) instead of being unable to retrieve the complete mailbox.

Corso Version?

Corso v0.19.0

Where are you running Corso?

Relevant log output

Error on the mailbox with the attachment issues:

“Error: Unable to complete backup Cause: fetching item: fetching item: getting mail attachment by id: An internal server error occurred. The operation failed., Property: [Calc:PersonId] PersonId, PropertyErrorCode: CorruptedData, PropertyErrorDescription: .: An internal server error occurred. The operation failed., Property: [Calc:PersonId] PersonId, PropertyErrorCode: CorruptedData, PropertyErrorDescription: .”

The full error:

{"level":"error","ts":1724315862.5491636,"caller":"graph/logging.go:60","msg":"graph api error: 500 Internal Server Error","disable_assist_backup":"false","request_start_time":"2024-08-22T08:37:27.743862074Z","resp_status_code":"500","incremental":"true","can_use_previous_backup":"true","collection_source":"operations","service":"Exchange”,”url":"https://graph.microsoft.com/v1.0/users/{user-id}/messages/{message-id}/attachments/{attachment-id} $expand=microsoft.graph.itemattachment/item","clues_trace”:”{trace},”resp_status":"500 Internal Server Error","method":"GET","assist_bases":"1","merge_bases":"0","request_content_len":"0","snapshot_type":"item data","resp_content_len":"-1","len_bases":"0","category":"email","attachment_id”:”{attachment-id}”,”attachment_odatatype":"#microsoft.graph.itemAttachment","request_retry_iter":"0","resource_owner_name”:”{owner},”categories":"[Contacts Emails Events]","tenant_id”:”{tenant_id}”,”resource_owner_selected”:”{owner}”,”collection_count":"16","resource_owner_id”:”{owner_id},”backup_id”:”{id},”can_use_metadata":"true","response":"HTTP/2.0 500 Internal Server Error\r\nCache-Control: private\r\nClient-Request-Id: {id}\r\nContent-Type: application/json; odata.metadata=minimal; odata.streaming=true; IEEE754Compatible=false; charset=utf-8\r\nDate: Thu, 22 Aug 2024 08:37:41 GMT\r\nRequest-Id: {id}\r\nStrict-Transport-Security: max-age=31536000\r\nVary: Accept-Encoding\r\nX-Ms-Ags-Diagnostic: {\"ServerInfo\":{\"DataCenter\":\"West Europe\",\"Slice\":\"E\",\"Ring\":\"5\",\"ScaleUnit\":\"003\",\"RoleInstance\":\"AM1PEPF0003BA69\"}}\r\n\r\n{\"error\":{\"code\":\"ErrorInternalServerError\",\"message\":\"An internal server error occurred. The operation failed., Property: [Calc:PersonId] PersonId, PropertyErrorCode: CorruptedData, PropertyErrorDescription: .\"}}"}

https://graph.microsoft.com/v1.0/users/{user-id}/messages/{message-id} in ms graph explorer:

{
    "@odata.context": "https://graph.microsoft.com/v1.0/$metadata#users('{user-id}')/messages/$entity",
    "@odata.etag": “W/\”{bla}””,
    "id": “{id}”,
    "createdDateTime": "2024-05-10T09:40:25Z",
    "lastModifiedDateTime": "2024-05-10T09:42:29Z",
    "changeKey": “{key}”,
    "categories": [],
    "receivedDateTime": "2024-05-10T09:40:27Z",
    "sentDateTime": "2024-05-10T09:40:01Z",
    "hasAttachments": true,
    "internetMessageId": “{id}”,
    "subject": “subject”,
    "bodyPreview": “preview”,
    "importance": "normal",
    "parentFolderId": “{id}”,
    "conversationId": “{id}”,
    "conversationIndex": “{index}”,
    "isDeliveryReceiptRequested": null,
    "isReadReceiptRequested": false,
    "isRead": true,
    "isDraft": false,
    "webLink": "https://outlook.office365.com/owa/?ItemID=<id>&exvsurl=1&viewmodel=ReadMessageItem",
    "inferenceClassification": "focused",
    "body": {
        "contentType": "html",
        "content": "<html>{content-of-mail}</html>"
    },
    "sender": {
        "emailAddress": {
            "name": “{name}”,
            "address": “{address}”
        }
    },
    "from": {
        "emailAddress": {
            "name": “{name}",
            "address": “{address}”
        }
    },
    "toRecipients": [
        {
            "emailAddress": {
                "name": “{name}”,
                "address": “{address}”
            }
        },
        {
            "emailAddress": {
                "name": “{name}”,
                "address": “{address}”
            }
        }
    ],
    "ccRecipients": [],
    "bccRecipients": [],
    "replyTo": [],
    "flag": {
        "flagStatus": "notFlagged"
    }
}

https://graph.microsoft.com/v1.0/users/{user-id}/messages/{message-id}/attachments shows 9 attachments. 

{
    "@odata.context": "https://graph.microsoft.com/v1.0/$metadata#users('{user-id}')/messages('{message-id}')/attachments",
    "value": [
        {
            "@odata.type": "#microsoft.graph.fileAttachment",
            "@odata.mediaContentType": "image/png",
            "id": “{id}”,
            "lastModifiedDateTime": "2024-05-10T09:40:25Z",
            "name": "image001.png",
            "contentType": "image/png",
            "size": 174792,
            "isInline": true,
            "contentId": "image001.png@01DAA2CE.5D600EF0",
            "contentLocation": null,
            "contentBytes": “content”
        },
        {
            "@odata.type": "#microsoft.graph.fileAttachment",
            "@odata.mediaContentType": "image/png",
            "id": “{id}”,
            "lastModifiedDateTime": "2024-05-10T09:40:25Z",
            "name": "image002.png",
            "contentType": "image/png",
            "size": 916,
            "isInline": true,
            "contentId": "image002.png@01DAA2CE.5D600EF0",
            "contentLocation": null,
            "contentBytes": “content"
        },
        {
            "@odata.type": "#microsoft.graph.fileAttachment",
            "@odata.mediaContentType": "image/png",
            "id": “{id}”,
            "lastModifiedDateTime": "2024-05-10T09:40:25Z",
            "name": "image003.png",
            "contentType": "image/png",
            "size": 635,
            "isInline": true,
            "contentId": "image003.png@01DAA2CE.5D600EF0",
            "contentLocation": null,
            "contentBytes": “content”
        },
        {
            "@odata.type": "#microsoft.graph.fileAttachment",
            "@odata.mediaContentType": "image/png",
            "id": “{id}”,
            "lastModifiedDateTime": "2024-05-10T09:40:25Z",
            "name": "image004.png",
            "contentType": "image/png",
            "size": 643,
            "isInline": true,
            "contentId": "image004.png@01DAA2CE.5D600EF0",
            "contentLocation": null,
            "contentBytes": “content”
        },
        {
            "@odata.type": "#microsoft.graph.fileAttachment",
            "@odata.mediaContentType": "image/png",
            "id": “{id}”,
            "lastModifiedDateTime": "2024-05-10T09:40:25Z",
            "name": "image005.png",
            "contentType": "image/png",
            "size": 21942,
            "isInline": true,
            "contentId": "image005.png@01DAA2CE.5D600EF0",
            "contentLocation": null,
            "contentBytes": “content”
        },
        {
            "@odata.type": "#microsoft.graph.fileAttachment",
            "@odata.mediaContentType": "image/jpeg",
            "id": “{id}”,
            "lastModifiedDateTime": "2024-05-10T09:40:25Z",
            "name": "image006.jpg",
            "contentType": "image/jpeg",
            "size": 142513,
            "isInline": true,
            "contentId": "image006.jpg@01DAA2CE.5D600EF0",
            "contentLocation": null,
            "contentBytes": “content”
        },
        {
            "@odata.type": "#microsoft.graph.fileAttachment",
            "@odata.mediaContentType": "image/png",
            "id": “{id}”,
            "lastModifiedDateTime": "2024-05-10T09:40:26Z",
            "name": "image007.png",
            "contentType": "image/png",
            "size": 74391,
            "isInline": true,
            "contentId": "image007.png@01DAA2CE.8CA558A0",
            "contentLocation": null,
            "contentBytes": “content”
        },
        {
            "@odata.type": "#microsoft.graph.itemAttachment",
            "id": “{id}”,
            "lastModifiedDateTime": "2024-05-10T09:40:26Z",
            "name": "",
            "contentType": null,
            "size": 1304,
            "isInline": false
        },
        {
            "@odata.type": "#microsoft.graph.itemAttachment",
            "id": “{id}”,
            "lastModifiedDateTime": "2024-05-10T09:40:26Z",
            "name": "",
            "contentType": null,
            "size": 1586,
            "isInline": false
        }
    ]
}

https://graph.microsoft.com/v1.0/users/{user-id}/messages/{message-id}/attachments/{attachment-id}?$expand=microsoft.graph.itemattachment/item for the last two attachments:

{
    "error": {
        "code": "ErrorInternalServerError",
        "message": "An internal server error occurred. The operation failed., Property: [Calc:PersonId] PersonId, PropertyErrorCode: CorruptedData, PropertyErrorDescription: ."
    }
}
brywhi commented 1 week ago

I am also experiencing this exact same issue.