Unable to upload a file

[RegisStGelais]

I'm able to get the list of all files in a OneDrive folder and also download a file from a OneDrive Folder to my android device. But I am unable to upload a file to OneDrive. It silently crash when I call the post() function. I use MSAL to connect to OneDrive. My scopes are: "User.Read", "Files.ReadWrite.All" I use microsoft-graph:2.0.0

Here is a simplified code with minimal error handling inspired from the sample app found here:

Note: I can debug step all the way to buildRequest() but it crashes when I try to step thru .post() The requestUrl seams ok: https://graph.microsoft.com/v1.0/me/drive/root:/TestFolder/Test File.sco:/microsoft.graph.createUploadSession (Crashes at the line that is commented // * Silently craches here ***** )

The runtime exception cause is: com.microsoft.graph.core.ClientException: Error during http request

   private void oneDrive_UploadFile(final IAuthenticationResult authenticationResult, final String strLocalFilePath)
    {
        // Create a callback used by the upload provider
        IProgressCallback<DriveItem> callback = new IProgressCallback<DriveItem>()
        {
            @Override
            // Called after each slice of the file is uploaded
            public void progress(final long current, final long max)
            {
                System.out.println(
                        String.format("Uploaded %d bytes of %d total bytes", current, max)
                );
            }

            @Override
            public void success(final DriveItem result)
            {
                System.out.println(
                        String.format("Uploaded file with ID: %s", result.id)
                );
                executeOneDriveTask(authenticationResult);
            }

            public void failure(final ClientException ex)
            {
                System.out.println(
                        String.format("Error uploading file: %s", ex.getMessage())
                );
            }
        };

        // Get an input stream for the file
        File file = new File(strLocalFilePath);
        if (file.exists())
        {
            final String accessToken = authenticationResult.getAccessToken();

            IGraphServiceClient graphClient =
                    GraphServiceClient
                            .builder()
                            .authenticationProvider(new IAuthenticationProvider()
                            {
                                @Override
                                public void authenticateRequest(IHttpRequest request)
                                {
                                    Log.d(TAG, "Authenticating request," + request.getRequestUrl());
                                    request.addHeader("Authorization", "Bearer " + accessToken);
                                }
                            })
                            .buildClient();

            InputStream fileStream = null;
            try
            {
                fileStream = new FileInputStream(file);
            }
            catch (FileNotFoundException e)
            {
                e.printStackTrace();
            }
            long streamSize = file.length();

            // Create an upload session
            UploadSession uploadSession = graphClient
                    .me()
                    .drive()
                    .root()
                    // itemPath like "/Folder/file.txt"
                    // does not need to be a path to an existing item
                    .itemWithPath(AppPrefs.getInstance().getOneDriveFolder() + '/' + file.getName())
                    .createUploadSession(new DriveItemUploadableProperties())
                    .buildRequest()
                    .post();   // ************* Silently craches here   *****************  

            ChunkedUploadProvider<DriveItem> chunkedUploadProvider = new ChunkedUploadProvider<DriveItem>(
                    uploadSession,
                    graphClient,
                    fileStream,
                    streamSize,
                    DriveItem.class);

            // Config parameter is an array of integers
            // customConfig[0] indicates the max slice size
            // Max slice size must be a multiple of 320 KiB
            int[] customConfig = {2 * 320 * 1024};

            // Do the upload
            try
            {
                chunkedUploadProvider.upload(callback, customConfig);
            }
            catch (IOException e)
            {
                e.printStackTrace();
            }
        }
    }

Any clue ?

Thanks

[RegisStGelais]

I was finally able to find the problem. I actually had 2 issues. The filename have spaces in it. I though that itemWithPath would convert into valid Url characters. Once I figured that I then found that the call was on the main thread.

[baywet]

Thanks for the update. Similar to #259 and it which was be fixed by #449 and released in 2.1.0 (I can see you're using 2.0.0)

[baywet]

Can I assume the issue is resolved and that we can close it?

[RegisStGelais]

Actually I still have en issue with paths containing a space. Let say my path is Test/Test File.sco I tried to convert to UTF-8 using URLEncoder.encode(value, StandardCharsets.UTF_8.toString()); but the spaces are converted to a +. The file get uploaded to onedrive but the filename still have the +. (Test+File.sco) Normaly spaces in Url need to be coded as %20 so I tried URLEncoder.encode(value.replace(" ", "%20"), StandardCharsets.UTF_8.toString()); With that the file on onedrive is named Test%20File.sco

Any hint?

[baywet]

are you still using 2.0.0? if so please update to 2.1.0

[RegisStGelais]

If you recall my duplicate error issue, I can't use 2.1.0 I need to wait for 2.1.1

[RegisStGelais]

case #457

[baywet]

No you don't have to wait, you can force the version of the duplicate dependency or exclude it. See https://github.com/microsoftgraph/msgraph-sdk-java/issues/450#issuecomment-692980036 and https://github.com/microsoftgraph/msgraph-sdk-java/issues/457#issuecomment-690293737

[RegisStGelais]

Ok, I upgraded to 2.1.0 and now I can't upload anymore. I get HTTP 431 error:

2020-09-16 10:09:30.627 31563-32121/com.laubrass.android.umtplus D/ContentValues: Authenticating request,https://api.onedrive.com/rup/c1fba35378bf924c/eyJSZXNvdXJjZUlEIjoiQzFGQkEzNTM3OEJGOTI0QyEyMzQ4NjciLCJSZWxhdGlvbnNoaXBOYW1lIjoiQ2hhbmdlJTIwRmxhdCUyMFRpcmUuc2NvIn0/4mpjoN8lDTCBJ5GZLkLt7I0qUhkcrsIJn8-E1r8us_wMvUnRnp8i_O05mSB7Z6gXp6m4fiCLSG6ScPqhrLksMBpBlxg12xZMpdQoHQzaIM29g/eyJuYW1lIjoiQ2hhbmdlJTIwRmxhdCUyMFRpcmUuc2NvIiwiQG5hbWUuY29uZmxpY3RCZWhhdmlvciI6InJlcGxhY2UifQ/4wpaNQ5dRWbEC61-_nPoSZ9D9UE2WwT4dKEbVa9bbubAmaLAtJYfAb_AG6phEJ94hUhwN6popABI8z6FgbGxxztLu_cX4vQlIOtHimU34GyoUZkcWKTlnP4OKcegD4QmKuUqrcLC2Z_1eU_RqPfzYknL2HIZM3peCNnMa4YOW4BJCVMKELPkeVWTvwKYX53GG49CiEX0gcSb-4HGWNSwmJ53RFQEeMtHInhHmj56RzG71Z_9rRhbn-K2KgWAggDr9FsXAImOA2V5u0PdPAHft9ZaBYjS916GXrdF0tQRMnQDttXIuOjOwzLM7X5kxv10soMiq02YlCXnZbefHWAnlgli2MKED8ADeuK8csGimLcwOyXpRFDPxNfX9lffLvCieqyOEnePpycGwxsJHMaw_9_ojQgv1LQMlDGhd4jHU8AOOwOlhpsEF9Ke4J2gXnuMZn7MD1LX8XC5QOg1L5nmWhDWGr03jlQwxxmoEICyIcXd4Dm4VVWfjP-6ZoaHlLnllvwbIdKxiXbFgwndvQErJS_TmO1vHjDRiELxomyvcfFQS-wj0l_dRHZ4v1HY1M6lGSzXlmyTL00hn4mN5B3qWopw
2020-09-16 10:09:30.860 31563-32121/com.laubrass.android.umtplus D/NetworkManagementSocketTagger: tagSocket(102) with statsTag=0xffffffff, statsUid=-1
2020-09-16 10:09:34.516 31563-32121/com.laubrass.android.umtplus E/global: CoreHttpProvider[sendRequestInternal] - 431Error during http request
2020-09-16 10:09:34.518 31563-32121/com.laubrass.android.umtplus E/global: Throwable detail: com.microsoft.graph.core.ClientException: Error during http request

    --------- beginning of crash
2020-09-16 10:09:34.554 31563-32121/com.laubrass.android.umtplus E/AndroidRuntime: FATAL EXCEPTION: Thread-13
    Process: com.laubrass.android.umtplus, PID: 31563
    com.microsoft.graph.core.ClientException: Request failed with error, retry if necessary.
        at com.microsoft.graph.requests.extensions.ChunkedUploadRequest.upload(ChunkedUploadRequest.java:116)
        at com.microsoft.graph.concurrency.ChunkedUploadProvider.upload(ChunkedUploadProvider.java:186)
        at com.laubrass.android.umtplus.presentation.Main$18.run(Main.java:1648)
        at java.lang.Thread.run(Thread.java:764)
     Caused by: com.microsoft.graph.core.ClientException: Error during http request
        at com.microsoft.graph.http.CoreHttpProvider.sendRequestInternal(CoreHttpProvider.java:429)
        at com.microsoft.graph.http.CoreHttpProvider.send(CoreHttpProvider.java:204)
        at com.microsoft.graph.requests.extensions.ChunkedUploadRequest.upload(ChunkedUploadRequest.java:114)
        at com.microsoft.graph.concurrency.ChunkedUploadProvider.upload(ChunkedUploadProvider.java:186) 
        at com.laubrass.android.umtplus.presentation.Main$18.run(Main.java:1648) 
        at java.lang.Thread.run(Thread.java:764) 
     Caused by: com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was BEGIN_ARRAY at line 1 column 1109 path $.thumbnails
        at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:226)
        at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:131)
        at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:222)
        at com.google.gson.Gson.fromJson(Gson.java:932)
        at com.google.gson.Gson.fromJson(Gson.java:897)
        at com.google.gson.Gson.fromJson(Gson.java:846)
        at com.google.gson.Gson.fromJson(Gson.java:817)
        at com.microsoft.graph.serializer.DefaultSerializer.deserializeObject(DefaultSerializer.java:82)
        at com.microsoft.graph.serializer.DefaultSerializer.deserializeObject(DefaultSerializer.java:76)
        at com.microsoft.graph.concurrency.ChunkedUploadResponseHandler.generateResult(ChunkedUploadResponseHandler.java:170)
        at com.microsoft.graph.concurrency.ChunkedUploadResponseHandler.generateResult(ChunkedUploadResponseHandler.java:47)
        at com.microsoft.graph.http.CoreHttpProvider.sendRequestInternal(CoreHttpProvider.java:369)
        at com.microsoft.graph.http.CoreHttpProvider.send(CoreHttpProvider.java:204) 
        at com.microsoft.graph.requests.extensions.ChunkedUploadRequest.upload(ChunkedUploadRequest.java:114) 
        at com.microsoft.graph.concurrency.ChunkedUploadProvider.upload(ChunkedUploadProvider.java:186) 
        at com.laubrass.android.umtplus.presentation.Main$18.run(Main.java:1648) 
        at java.lang.Thread.run(Thread.java:764) 
     Caused by: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was BEGIN_ARRAY at line 1 column 1109 path $.thumbnails
        at com.google.gson.stream.JsonReader.beginObject(JsonReader.java:386)
        at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:215)
        at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:131) 
        at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:222) 
        at com.google.gson.Gson.fromJson(Gson.java:932) 
        at com.google.gson.Gson.fromJson(Gson.java:897) 
        at com.google.gson.Gson.fromJson(Gson.java:846) 
        at com.google.gson.Gson.fromJson(Gson.java:817) 
        at com.microsoft.graph.serializer.DefaultSerializer.deserializeObject(DefaultSerializer.java:82) 
        at com.microsoft.graph.serializer.DefaultSerializer.deserializeObject(DefaultSerializer.java:76) 
        at com.microsoft.graph.concurrency.ChunkedUploadResponseHandler.generateResult(ChunkedUploadResponseHandler.java:170) 
        at com.microsoft.graph.concurrency.ChunkedUploadResponseHandler.generateResult(ChunkedUploadResponseHandler.java:47) 
        at com.microsoft.graph.http.CoreHttpProvider.sendRequestInternal(CoreHttpProvider.java:369) 
        at com.microsoft.graph.http.CoreHttpProvider.send(CoreHttpProvider.java:204) 
        at com.microsoft.graph.requests.extensions.ChunkedUploadRequest.upload(ChunkedUploadRequest.java:114) 
        at com.microsoft.graph.concurrency.ChunkedUploadProvider.upload(ChunkedUploadProvider.java:186) 
        at com.laubrass.android.umtplus.presentation.Main$18.run(Main.java:1648) 
        at java.lang.Thread.run(Thread.java:764) 

[RegisStGelais]

The issue might be in core. How do I force gradle to use the new core ?

[baywet]

add implementation 'com.microsoft.graph:microsoft-graph-core:1.0.3' and add another exclusion group under the non core one exclude group: 'com.microsoft.graph', module: 'microsoft-graph-core'

[RegisStGelais]

Done, Same HTTP 431 error. My upload code was working with 2.0.0 but not with 2.1.0 and core 1.0.3

[baywet]

can you provide a sample repo with the reproduction please? I've tried reproducing the issue by tweaking this unit test and include multiple spaces in the filename and it worked without any issue.

[RegisStGelais]

The issue I still have with 2.1.0 is not related to the spaces in the file name. I fixed the space issue by replacing them with %20 I also get the same issue when uploading to an hard coded filename without space.

The issue I have with 2.1.0 is the HTTP 431 error

[baywet]

thanks for the precision, still I can't repro the issue on my side, please share a repro repo

[RegisStGelais]

The first execption I see when stepping in the code is com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was BEGIN_ARRAY at line 1 column 1103 path $.thumbnails in CoreHttpProvider.java -> sendRequestInternal called by ChunkedUploadRequest.jaja, Called by the ChuckedUploadProvider.java,

[RegisStGelais]

I also have dropbox in my app and it implement 'com.google.code.gson:gson:2.8.6' Could it be a version conflict ?

[RegisStGelais]

Also. I tried 2.0.0 with 1.0.3 Core. implementation 'com.microsoft.graph:microsoft-graph-core:1.0.3' implementation('com.microsoft.graph:microsoft-graph:2.0.0') { exclude group: 'com.microsoft.graph', module: 'microsoft-graph-core' } Upload works. Issue is probably not in core 1.0.3 but in graph 2.1.0

[baywet]

I also have dropbox in my app and it implement 'com.google.code.gson:gson:2.8.6' Could it be a version conflict ?

no as we're using the same version.

If you provide me with a repro I'll be able to investigate. Until then, it's going to be really hard to do anything more than guessing.

[RegisStGelais]

This code is in a big Android Studio project. I could extract some parts and put them in a small project but it will still be an Android Studio project. Is it ok for you ?

[baywet]

yes that should work, thanks

[RegisStGelais]

I did some more debuging and the issue is when ChunkedUploadResult::generateResult tries to deserialize the rawJson response into a DriveItem at the end of the upload. (see Line 170 of ChunckedUploadResponseHandler.java)

The rawJson value is: {"createdBy":{"application":{"displayName":"UmtPlus","id":"4458250c"},"user":{"id":"c1fba35378bf924c"}},"createdDateTime":"2020-09-16T14:53:53.61Z","cTag":"aYzpDMUZCQTM1Mzc4QkY5MjRDITIzNDkyNS4yNTc","eTag":"aQzFGQkEzNTM3OEJGOTI0QyEyMzQ5MjUuMTE","id":"C1FBA35378BF924C!234925","lastModifiedBy":{"application":{"displayName":"UmtPlus","id":"4458250c"},"user":{"id":"c1fba35378bf924c"}},"lastModifiedDateTime":"2020-09-16T17:42:17.847Z","name":"Change Flat Tire.sco","parentReference":{"driveId":"c1fba35378bf924c","driveType":"personal","id":"C1FBA35378BF924C!234867","name":"UmtPlus","path":"/drive/root:/UmtPlus"},"size":59228,"webUrl":"https://1drv.ms/u/s!AEySv3hTo_vBjqst","items":[],"file":{"hashes":{"quickXorHash":"RjqF6zG7yzMKxLlRmXkKr0tK7oQ=","sha1Hash":"A7A1DB7C7355A372E6097C5BD7DF6CF702AFA897","sha256Hash":"97EF73D523368EE939D084F87DE22E28BD9236CC55D6A67EE69183FFC456CA08"},"mimeType":"application/octet-stream"},"fileSystemInfo":{"createdDateTime":"2020-09-16T14:53:53.61Z","lastModifiedDateTime":"2020-09-16T17:42:17.846Z"},"reactions":{"commentCount":0},"tags":[],"lenses":[],"thumbnails":[{"id":"0","large":{"height":800,"url":"https://oxo45g.bl.files.1drv.com/y4pi3j1XhJr0-LmucbMAY7erAc5yeeX8yXaxUqk7p5O1mYVUMnRmzIeFC8LgpZLXCNFkFfVzt_PlChpIBL2VwTp9bdXVToVWsHRKC5MmEiO4Zv3eR9_JCc2ih4jstMbx6AusvkIpCW7FEpWWSeyFQEJR0jbaNNZSs_n6Ryrio2xYl9LhINf19-xYBxVCR4kV188?width=800&height=800&cropmode=none","width":800},"medium":{"height":176,"url":"https://oxo45g.bl.files.1drv.com/y4pi3j1XhJr0-LmucbMAY7erAc5yeeX8yXaxUqk7p5O1mYVUMnRmzIeFC8LgpZLXCNFkFfVzt_PlChpIBL2VwTp9bdXVToVWsHRKC5MmEiO4Zv3eR9_JCc2ih4jstMbx6AusvkIpCW7FEpWWSeyFQEJR0jbaNNZSs_n6Ryrio2xYl9LhINf19-xYBxVCR4kV188?width=176&height=176&cropmode=none","width":176},"small":{"height":96,"url":"https://oxo45g.bl.files.1drv.com/y4pi3j1XhJr0-LmucbMAY7erAc5yeeX8yXaxUqk7p5O1mYVUMnRmzIeFC8LgpZLXCNFkFfVzt_PlChpIBL2VwTp9bdXVToVWsHRKC5MmEiO4Zv3eR9_JCc2ih4jstMbx6AusvkIpCW7FEpWWSeyFQEJR0jbaNNZSs_n6Ryrio2xYl9LhINf19-xYBxVCR4kV188?width=96&height=96&cropmode=none","width":96}}]}

And it fails to deserialize it.

I'm pretty sure that if you try to deserialize that string you will get the same fail.

Note: I use https://jsonformatter.org/json-parser to visualize the Json string and it looks well formed.

[RegisStGelais]

Also: If I break just before the deserialization and remove the thumbnails array part of the json string, the deserialisation succeeds.

[RegisStGelais]

Funny, I can't see a thumbnails property in the driveItem specs

Correction: There is a thumbnails Relationship. its a collection of thumbnailSet. Looks like version 2.1.0 is not able de deserialize it.

[RegisStGelais]

Vincent,

I striped down my project for you to look at the upload issue.

You can get it from this OneDrive link The zip file is password protected. Contact me in private to get it. (you will find my email in my github profile)

There are instruction in the ReadMe.docx file located at the root of the project.

Thank you for looking at this.

Regards

[baywet]

Hey @RegisStGelais Thanks! I was able to identify and fix this bug. It's actually a compound of bugs:

• collections properties were not deserialized when returned from the service
• collections were not being serialized (sent) properly to the service which I fixed for emails with #202
• the serializer was specific to AttachmentCollection, which means the default json serializer was trying to deserialize things by itself mapping the wrong classes.

both issues A and C are now fixed by #483. Merging the PR will close this issue.

[RegisStGelais]

Hi @baywet, I'm glad to hear that you where able to find and fix the issue. I'm waiting for that fix to release the new version of my app. Do I need to wait for version 2.2 or is it possible for me to pull a patched version ?

Thank you for your support.

Regards

[baywet]

We don't publish snapshots at the moment. But you could always pull the repo to build it locally and make a local reference if you want to try things out.

[RegisStGelais]

Do you have an estimated release date for V2.2 ?

[RegisStGelais]

@baywet I just tested the new 2.2 release. implementation('com.microsoft.graph:microsoft-graph:2.2.0') The file is uploaded but the success callback is never called.

Should I open a new issue ?

[baywet]

can you double check your implementation please? Because there's a unit test that specifically tests for that and passes as far as I can remember. https://github.com/microsoftgraph/msgraph-sdk-java/blob/dev/src/test/java/com/microsoft/graph/functional/OneDriveTests.java#L43

[RegisStGelais]

dependencies {
    implementation fileTree(dir: 'libs', include: ['*.jar'])

    implementation 'com.dropbox.core:dropbox-core-sdk:3.1.5'
    implementation 'androidx.appcompat:appcompat:1.1.0'
    implementation 'com.google.code.gson:gson:2.8.6'

    api 'com.squareup.okhttp3:okhttp:4.4.0'

    implementation 'com.microsoft.identity.client:msal:1.6.0'
    implementation 'com.android.volley:volley:1.1.1'

    implementation('com.microsoft.graph:microsoft-graph:2.2.0')

//    implementation 'com.microsoft.graph:microsoft-graph-core:1.0.3' // temporaire pour graph 2.1.0
//   implementation('com.microsoft.graph:microsoft-graph:2.1.0') {
//        exclude group: 'com.microsoft.graph', module: 'microsoft-graph-core'
//    }

    testImplementation 'junit:junit:4.13'
    androidTestImplementation 'androidx.test.ext:junit:1.1.2'
    androidTestImplementation 'androidx.test.espresso:espresso-core:3.3.0'
}

[RegisStGelais]

My progress callback gets called 3 times but success is never called.

[RegisStGelais]

The condition at line 188 of ChunkedUploadProvider.java is never satisfied so the success callback is never called.

          if (result.uploadCompleted()) {
                callback.progress(this.streamSize, this.streamSize);
                callback.success((UploadType) result.getItem());
                break;
            }