Large file upload fails when uploading to a (mail) message

[dvag-joerg-winter]

Hi,

using graph v6.1.0 we get:

com.microsoft.kiota.ApiException: generalException

at com.microsoft.kiota.ApiExceptionBuilder.withMessage(ApiExceptionBuilder.java:45)
at com.microsoft.graph.core.requests.upload.UploadResponseHandler.handleResponse(UploadResponseHandler.java:61)
at com.microsoft.graph.core.requests.upload.UploadSliceRequestBuilder.put(UploadSliceRequestBuilder.java:69)
at com.microsoft.graph.core.tasks.LargeFileUploadTask.uploadSlice(LargeFileUploadTask.java:207)
at com.microsoft.graph.core.tasks.LargeFileUploadTask.upload(LargeFileUploadTask.java:131)

for this code:

    private fun uploadAttachment(messageId: String, inputStream: InputStream, filename: String): UploadResult<FileAttachment>? {
        val largeAttachment = AttachmentItem().apply {
            attachmentType = AttachmentType.File
            name = filename
            size = inputStream.available().toLong()
            odataType = "#microsoft.graph.attachmentItem"
        }

        val uploadSessionPostRequestBody = CreateUploadSessionPostRequestBody().apply {
            attachmentItem = largeAttachment
        }
        val uploadSession = graphClientUser
            .messages()
            .byMessageId(messageId)
            .attachments()
            .createUploadSession()
            .post(uploadSessionPostRequestBody)

        val callback = IProgressCallback { current, max ->
            log.info("Uploaded $filename $current bytes of $max total bytes")
        }

        uploadSession?.let {
            val largeFileUploadTask = LargeFileUploadTask(
                graphClient.requestAdapter,
                it,
                inputStream,
                largeAttachment.size!!,
                FileAttachment::createFromDiscriminatorValue
            )
            try {
                val uploadResult = largeFileUploadTask.upload(1, callback)
                log.info("Upload of $filename successful: ${uploadResult?.isUploadSuccessful}")
                return uploadResult
            } catch (e: Exception) {
                log.error(e.message)
                throw e
            }
        }
        throw RuntimeException("no session")
    }

failing line is val uploadResult = largeFileUploadTask.upload(1, callback) We tried to adhere to this migration example here: https://github.com/microsoftgraph/msgraph-sdk-java/blob/59581c120f2b0459512f4b19e7f2725d2699ea9b/docs/upgrade-to-v6.md#large-file-upload-enhancements

Our case uploads to a (mail) message, the previous graph release 5.58.0 was successful, using the same mailbox

debugging, we see no more than this:

the message we upload to, was created before this upload: graphClient.user(...).mailFolders().byMailFolderId(configurationProperties.mailbox.draftId).messages().post(message)

(authentication is application registration, not delegated)

[Ndiritu]

Thanks for the detailed breakdown @dvag-joerg-winter.

Linking this to a previously created issue https://github.com/microsoftgraph/msgraph-sdk-java/issues/1806

[dvag-joerg-winter]

Hi @Ndiritu from my point of view, the issue #1806 is very different. The large file upload (to a mail message) returns this response:

Response{protocol=h2, code=400, message=, url=https://outlook.office365.com/api/gv1.0/users('71e99be6-f1cc-4aa0-ae49-cd8a78f16e52@930d042f-8145-48e6-871e-7659c17b56da')/messages('AAMkAGI5YTY0ZTM4LTViYTktNGM0NC04MTk3LWFjZjBlMTg1MmU2ZABGAAAAAABnZOgv6hYTS4T2ovW9N_JpBwAdlr4xzuUoQoGxNCP3KeEiAAAAAAEPAABufLUX4I7ZQam6nocOyooYAADeZNXCAAA=')/AttachmentSessions('AAMkAGI5YTY0ZTM4LTViYTktNGM0NC04MTk3LWFjZjBlMTg1MmU2ZABGAAAAAABnZOgv6hYTS4T2ovW9N_JpBwAdlr4xzuUoQoGxNCP3KeEiAAAAAAEBAABufLUX4I7ZQam6nocOyooYAADeZN2RAAA=')?authtoken=OMITTED

btw, same error also with version 6.2.0

Is this relevant for this error maybe ? https://learn.microsoft.com/en-us/graph/outlook-large-attachments?tabs=java

Do you know of any example code we can look at for our usecase ?

[dvag-joerg-winter]

@Ndiritu when I remove the auth-Token as stated in the answeres here https://stackoverflow.com/questions/63933868/uploading-a-large-attachment-using-microsoft-graph-got-invalid-token-error

..then I get this new error

java.lang.NullPointerException: Cannot invoke "okhttp3.MediaType.toString()" because the return value of "okhttp3.ResponseBody.contentType()" is null

    at com.microsoft.graph.core.requests.upload.UploadResponseHandler.handleResponse(UploadResponseHandler.java:58)

[dvag-joerg-winter]

@Ndiritu can you re-open this issue ?

[Ndiritu]

Sure, will take another look

[dvag-joerg-winter]

We could present the integration-test we have for this, if you want, in a Teams meeting, if that makes sense for you. My code-search in all of Github does not yield any good example of such (large)uploads to a message, unfortunately.

[dvag-joerg-winter]

@Ndiritu Hi, specifically it seems to me, that this "opaque and authenticated uploadUrl" (from the created uploadSession) could be the problem, as mentioned here: https://learn.microsoft.com/en-us/graph/outlook-large-attachments?tabs=java

I can see that the put request has indeed a parameter "authtoken" with a value https://outlook.office365.com/api/gv1.0/users('71e99be6-f1cc-4aa0-ae49-cd8a78f16e52@930d042f-8145-48e6-871e-7659c17b56da')/messages('AAMkAGI5YTY0ZTM4LTViYTktNGM0NC04MTk3LWFjZjBlMTg1MmU2ZABGAAAAAABnZOgv6hYTS4T2ovW9N_JpBwAdlr4xzuUoQoGxNCP3KeEiAAAAAAEPAABufLUX4I7ZQam6nocOyooYAADecef2AAA=')/AttachmentSessions('AAMkAGI5YTY0ZTM4LTViYTktNGM0NC04MTk3LWFjZjBlMTg1MmU2ZABGAAAAAABnZOgv6hYTS4T2ovW9N_JpBwAdlr4xzuUoQoGxNCP3KeEiAAAAAAEBAABufLUX4I7ZQam6nocOyooYAADecfOxAAA=')?authtoken=](https://outlook.office365.com/api/gv1.0/users(%2771e99be6-f1cc-4aa0-ae49-cd8a78f16e52@930d042f-8145-48e6-871e-7659c17b56da%27)/messages(%27AAMkAGI5YTY0ZTM4LTViYTktNGM0NC04MTk3LWFjZjBlMTg1MmU2ZABGAAAAAABnZOgv6hYTS4T2ovW9N_JpBwAdlr4xzuUoQoGxNCP3KeEiAAAAAAEPAABufLUX4I7ZQam6nocOyooYAADecef2AAA=%27)/AttachmentSessions(%27AAMkAGI5YTY0ZTM4LTViYTktNGM0NC04MTk3LWFjZjBlMTg1MmU2ZABGAAAAAABnZOgv6hYTS4T2ovW9N_JpBwAdlr4xzuUoQoGxNCP3KeEiAAAAAAEBAABufLUX4I7ZQam6nocOyooYAADecfOxAAA=%27)?authtoken=...TOKEN VALUE exists here!....

Is this the correct opaque authenticated URL for the PUT ? The token used in the POST for creating the upload session is different from the token in the uploadUrl property, so thats probably the mentioned pre-authenticated Url , right ?

Also the base Url seemed a bit weird to us: https://outlook.office365.com/api/gv1.0/ shouldn't it be https://outlook.office365.com/api/v1.0/ ?

[ramsessanchez]

Hi, I believe I have found the issue that was causing the failure for large file upload. It seems the content-length header was being added, however, the content-length property on the RequestBody object was not being set and leading to failures while uploading. I have put a PR for this so hopefully we are able to have this patched early next week / upon next minor version bump, 6.3.0. Apologies for the hassle and thanks for using the SDK.

[ramsessanchez]

https://github.com/microsoft/kiota-java/pull/1088

[dvag-joerg-winter]

Thanks for the update. For testing this, I tried building/installing (locally) the kiota version (1.0.3-SNAPSHOT including your PR)

My failing upload now has this different message:

btw, it's a 15MB upload

[dvag-joerg-winter]

@ramsessanchez Hi, still not sure if this is on our side (infrastructure or parameters in a broader sense) or a fault of the SDK. The attached Kotlin code above should reproduce this issue. Would it help, if I converted it to a Java Unittest ?

[dvag-joerg-winter]

@Ndiritu Hi, we tested with kiota-java 1.0.6 and still get this failure:

2024-03-05T08:28:37.655+01:00 ERROR 31816 --- [           main] c.d.l.s.mailservice.MailServiceImpl      : Upload of big-image.jpg unsuccessful: generalException

com.microsoft.kiota.ApiException: generalException

    at com.microsoft.kiota.ApiExceptionBuilder.withMessage(ApiExceptionBuilder.java:45)
    at com.microsoft.graph.core.requests.upload.UploadResponseHandler.handleResponse(UploadResponseHandler.java:62)
    at com.microsoft.graph.core.requests.upload.UploadSliceRequestBuilder.put(UploadSliceRequestBuilder.java:69)
    at com.microsoft.graph.core.tasks.LargeFileUploadTask.uploadSlice(LargeFileUploadTask.java:207)
    at com.microsoft.graph.core.tasks.LargeFileUploadTask.upload(LargeFileUploadTask.java:131)

source looks like this

    private fun uploadAttachment(
        graphClientUser: UserItemRequestBuilder,
        messageId: String,
        inputStream: InputStream,
        filename: String
    ): UploadResult<FileAttachment>? {
        val largeAttachment = AttachmentItem().apply {
            attachmentType = AttachmentType.File
            name = filename
            size = inputStream.available().toLong()
            contentType = "image/jpeg" // TODO ?
        }
        val uploadSessionPostRequestBody = CreateUploadSessionPostRequestBody().apply {
            attachmentItem = largeAttachment

        }
        val uploadSession = graphClientUser
            .messages()
            .byMessageId(messageId)
            .attachments()
            .createUploadSession()
            .post(uploadSessionPostRequestBody)

        val callback = IProgressCallback { current, max ->
            log.info("Uploaded $filename $current bytes of $max total bytes")
        }

        uploadSession?.let {
            val largeFileUploadTask = LargeFileUploadTask(
                graphClient.requestAdapter,
                it,
                inputStream,
                largeAttachment.size!!,
                // 320 * 1024,
                FileAttachment::createFromDiscriminatorValue
            )
            try {
                val uploadResult = largeFileUploadTask.upload(2, callback)
                log.info("Upload of $filename successful: ${uploadResult?.isUploadSuccessful}")
                return uploadResult
            } catch (e: Exception) {
                log.error("Upload of $filename unsuccessful: ${e.message}")
                throw e
            }
        }
        throw RuntimeException("no session")
    }

[ramsessanchez]

Hi, after testing with 1.0.6 I can confirm that the upload is successful with the following code:

        FileInputStream file = new FileInputStream("file-path");
        long size = file.available();

        AttachmentItem item = new AttachmentItem();
        item.setAttachmentType(AttachmentType.File);
        item.setName("Text.txt");
        item.setContentType("text/plain");
        item.setSize(size);

        com.microsoft.graph.users.item.messages.item.attachments.createuploadsession.CreateUploadSessionPostRequestBody requestBody = new com.microsoft.graph.users.item.messages.item.attachments.createuploadsession.CreateUploadSessionPostRequestBody();
        requestBody.setAttachmentItem(item);
        UploadSession session = graphClient.me().messages().byMessageId("id").attachments().createUploadSession().post(requestBody);

        LargeFileUploadTask<AttachmentItem> largeFileUploadTask = new LargeFileUploadTask(graphClient.getRequestAdapter(), session, file, size, AttachmentItem::createFromDiscriminatorValue);

however, the SDK is returning an exception as the handling of the response expects a response with a body/content-type/content-length, however this API does not return that on a successful response and so despite the response code being 201 we are throwing an exception due to improper response handling. Thank you for taking the time to share your code, we will look to fix this as soon as possible.

[dvag-joerg-winter]

Thx @ramsessanchez for the update. Just checked the large upload with kiota 1.0.6, but the file is still not present in the received mail message. You are probably saying that technically the upload is Ok now, but still the file is not handled completely (attached to the message), because of the mentioned exception ?