OKHttpCLient is setting incorrect content-type on uploads

[bollerdominik]

            B2FileContentSource source = B2FileContentSource.builder(new File("C:\\Users\\test\\Pictures\\test.jpg")).build();
            B2UploadFileRequest request = B2UploadFileRequest
                    .builder(BUCKET_ID, b2FilePath, "image/jpeg", source)
                    .build();
            client.uploadSmallFile(request);

I am trying to upload an image to a public bucket and view it in the browser. The image is uploaded correctly but it is being served as application/json and the browser will not display correctly.

If I upload the same image to the bucket using the b2 website it correctly displays it as image/jpeg.

How can I set the correct content type when uploading an image with the java sdk? I have also tried B2_AUTO as the content type

[certainmagic]

Hi Dominik --

That's a pretty freaky bug report! Thanks for filing it!

Unfortunately, I wasn't able to immediately reproduce it locally. Here's my code (based on yours):

        final File srcFile = new File("/tmp/my.jpg");

        B2FileContentSource source = B2FileContentSource.builder(srcFile).build();
        B2UploadFileRequest request = B2UploadFileRequest
                .builder(bucket.getBucketId(), "file.jpg", "image/jpeg", source)
                .build();
        client.uploadSmallFile(request);

        out.println("--------");
        for (B2FileVersion version : client.fileNames(bucket.getBucketId())) {
            out.println(version);
            out.println(client.getDownloadByNameUrl(bucket.getBucketName(), version.getFileName()));
            out.println("--------");
        }

Here's the output I got (which shows the content-type is right in the metadata):

B2FileVersion{fileId='4_z50ae39fcda24081a76010118_f128d6748ee29deea_d20200201_m173524_c100_v0009990_t0002', contentLength=18000, contentType='image/jpeg', contentSha1='aeaf00fd4a9d2ff902cf115c9443f78de8cce465', contentMd5='5cc15aa6e7fe96ef184730d92828c0f5', action='upload', uploadTimestamp=1580578524000, fileInfo=[1], fileName='file.jpg'}
http://f100.backblazeb2.xyz:8180/file/sdkTest-0e9ca48a6118/file.jpg

When I download the file by that url, I'm getting reasonable looking headers, including the right content-type:

curl -i http://f100.backblazeb2.xyz:8180/file/sdkTest-0e9ca48a6118/file.jpg

HTTP/1.1 200 
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
x-frame-options: SAMEORIGIN
Cache-Control: max-age=0, no-cache, no-store
x-bz-file-name: file.jpg
x-bz-file-id: 4_z50ae39fcda24081a76010118_f128d6748ee29deea_d20200201_m173524_c100_v0009990_t0002
x-bz-content-sha1: aeaf00fd4a9d2ff902cf115c9443f78de8cce465
X-Bz-Upload-Timestamp: 1580578524000
Accept-Ranges: bytes
x-bz-info-src_last_modified_millis: 1580577356000
Content-Type: image/jpeg
Content-Length: 18000
Date: Sat, 01 Feb 2020 17:35:35 GMT
....

This was against my local development version of the server. Can you try running with the extra println()s I added and run the curl, too, please.

ps. I have to run out for a while, so I won't be able to reply immediately.

[certainmagic]

Hi --

I've run my program against the production b2 service as well and it seems to be behaving as expected.

I did notice that before I remembered to switch my bucket from private to public, my curl returned a HTTP 401 response whose body's content-type was 'application/json;charset=utf-8'. It sounds like you're running against a public bucket so that's probably not what you were hitting.

thanks, ab

[bollerdominik]

Hi Thanks for getting back at me.

I just tried it again on a new bucket with various test images. I always get contentType='application/json' from B2FileVersion :(

Again my code snipped

    public void upload() {
        try {
            B2FileContentSource source = B2FileContentSource.build(new File("C:\\Users\\test\\Pictures\\test.jpg"));
            storeFile("test.jpg", source);
        } catch (IOException | B2Exception e) {
            log.error("Could not upload file", e);
            throw new ResponseStatusException(HttpStatus.INTERNAL_SERVER_ERROR, "Could not upload file");
        }
    }

    public void storeFile(String b2FilePath, B2ContentSource source) throws IOException, B2Exception {
        B2UploadFileRequest request = B2UploadFileRequest
                .builder(BUCKET_ID, b2FilePath, "image/jpeg" , source)
                .build();

        client.uploadSmallFile(request);

        for (B2FileVersion version : client.fileNames(BUCKET_ID)) {
            System.out.println(version);
            System.out.println(client.getDownloadByNameUrl("testa123", version.getFileName()));
            System.out.println("--------");
        }

    }

B2FileVersion{fileId='4_z0deaf1d0e05402b37e040d16_f1045ccc660a7cc37_d20200203_m165050_c002_v0001132_t0007', contentLength=165150, contentType='application/json', contentSha1='f65a7a34b9cf4ace3f953c911e287144f101d26b', action='upload', uploadTimestamp=1580748650000, fileInfo=[0], fileName='1.jpg'}
https://f002.backblazeb2.com/file/testa123/1.jpg

[certainmagic]

Hi Dominik --

This version looks a little different than the earlier version where you were specifying "image/jpeg" explicitly. i would expect both to work, but i'm wondering if you're still having the same trouble when you explicitly provide the content-type. i will try again to reproduce it.

what version of the SDK are you using? i'm running against the most recent code on the master branch.

thanks, ab

On Mon, Feb 3, 2020 at 4:47 PM Dominik Boller notifications@github.com wrote:

Hi Thanks for getting back at me.

I just tried it again on a new bucket with various test images. I always get contentType='application/json' from B2FileVersion :(

Again my code snipped

public void upload() {
    try {
        B2FileContentSource source = B2FileContentSource.build(new File("C:\\Users\\test\\Pictures\\test.jpg"));
        storeFile("test.jpg", source);
    } catch (IOException | B2Exception e) {
        log.error("Could not upload file", e);
        throw new ResponseStatusException(HttpStatus.INTERNAL_SERVER_ERROR, "Could not upload file");
    }
}

public void storeFile(String b2FilePath, B2ContentSource source) throws IOException, B2Exception {
    B2UploadFileRequest request = B2UploadFileRequest
            .builder(BUCKET_ID, b2FilePath, B2ContentTypes.B2_AUTO, source)
            .build();

    client.uploadSmallFile(request);

    for (B2FileVersion version : client.fileNames(BUCKET_ID)) {
        System.out.println(version);
        System.out.println(client.getDownloadByNameUrl("testa123", version.getFileName()));
        System.out.println("--------");
    }

}

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Backblaze/b2-sdk-java/issues/104?email_source=notifications&email_token=ABHJFCWPJZHVFFZCVDODUGTRBBDCLA5CNFSM4KOSC5C2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKURTVI#issuecomment-581507541, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHJFCTUPXOB6AKBGN5KPITRBBDCLANCNFSM4KOSC5CQ .

[certainmagic]

Here's my sample program using B2_AUTO. It's basically the code you provided with some code around it to provide the client and BUCKET_ID. it's getting its credentials from $B2_APPLICATION_KEY_ID and $B2_APPLICATION_KEY and the bucketId from $BUCKET_ID.

Issue104.java.txt (i had to make its name end in '.txt' to upload it as a single file)

[certainmagic]

If you can tell me what you're running against, i can try my program against that.

On Mon, Feb 3, 2020 at 5:38 PM certainmagic notifications@github.com wrote:

Here's my sample program using B2_AUTO. It's basically the code you provided with some code around it to provide the client and BUCKET_ID. it's getting its credentials from $B2_APPLICATION_KEY_ID and $B2_APPLICATION_KEY and the bucketId from $BUCKET_ID.

Issue104.java.txt https://github.com/Backblaze/b2-sdk-java/files/4149212/Issue104.java.txt (i had to make its name end in '.txt' to upload it as a single file)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Backblaze/b2-sdk-java/issues/104?email_source=notifications&email_token=ABHJFCRDPQBIVWR6FUYDAT3RBBJBLA5CNFSM4KOSC5C2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKUXDXA#issuecomment-581530076, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHJFCUEH32XRZ4GDPV2YH3RBBJBLANCNFSM4KOSC5CQ .

[bollerdominik]

Thanks I will check your example for a bit but for now the code looks pretty much the same.

I have tried with B2_AUTO and "image/jpeg"

Now that you mention version.. I using the maven dependency which is at 3.1.0. I just noticed the maven version is almost one year old. I assume that's not the latest ?

[certainmagic]

It looks like that's the most recent version we've published, despite having made some number of changes.

Let me grab v3.1 and check the behavior. This level of core functionality really shouldn't have changed...ever.

ttfn, ab

On Mon, Feb 3, 2020 at 6:06 PM Dominik Boller notifications@github.com wrote:

Thanks I will check your example for a bit but for now the code looks pretty much the same.

I have tried with B2_AUTO and "image/jpeg"

Now that you mention version.. I using the maven dependency which is at 3.1.0. I just noticed the maven version is almost one year old. I assume that's not the latest ?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Backblaze/b2-sdk-java/issues/104?email_source=notifications&email_token=ABHJFCTR2VWNHJSV3IHYEBLRBBMLNA5CNFSM4KOSC5C2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKU2ARI#issuecomment-581541957, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHJFCUXSOXV6UVLTZVHU2LRBBMLNANCNFSM4KOSC5CQ .

[bollerdominik]

I just checked your sample. The only thing I changed is the client setup

    private Issue104() {
        this.client = B2StorageClientFactory.createDefaultFactory()
                .create("a", "a", "h");
    }

(replaced the keys with "a")

Same behavior!

B2FileVersion{fileId='4_z0deaf1d0e05402b37e040d16_f1112fcdca2355cea_d20200203_m183119_c002_v0001133_t0044', contentLength=673771, contentType='application/json', contentSha1='e4323598b591209e949597ef94c60edb9a28bb15', action='upload', uploadTimestamp=1580754679000, fileInfo=[1], fileName='test.jpg'}

The only difference from you (besides the version maybe) is that I am on windows. (But afaik I noticed the issue on our server which is linux)

[certainmagic]

Running against the v3.1.0 jars, it's still giving me image/jpeg with B2_AUTO and with "image/jpeg".

On Mon, Feb 3, 2020 at 6:17 PM certainmagic notifications@github.com wrote:

It looks like that's the most recent version we've published, despite having made some number of changes.

Let me grab v3.1 and check the behavior. This level of core functionality really shouldn't have changed...ever.

ttfn, ab

On Mon, Feb 3, 2020 at 6:06 PM Dominik Boller notifications@github.com wrote:

Thanks I will check your example for a bit but for now the code looks pretty much the same.

I have tried with B2_AUTO and "image/jpeg"

Now that you mention version.. I using the maven dependency which is at 3.1.0. I just noticed the maven version is almost one year old. I assume that's not the latest ?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://github.com/Backblaze/b2-sdk-java/issues/104?email_source=notifications&email_token=ABHJFCTR2VWNHJSV3IHYEBLRBBMLNA5CNFSM4KOSC5C2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKU2ARI#issuecomment-581541957 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/ABHJFCUXSOXV6UVLTZVHU2LRBBMLNANCNFSM4KOSC5CQ

.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Backblaze/b2-sdk-java/issues/104?email_source=notifications&email_token=ABHJFCW2OJ5327RX3KSY3OTRBBNVBA5CNFSM4KOSC5C2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKU3FZY#issuecomment-581546727, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHJFCTNZAPSLP5HWCRNM2TRBBNVBANCNFSM4KOSC5CQ .

[certainmagic]

Hey Dominik --

Can you step through this program in a debugger? In particular, I'm curious what headers are being set in B2StorageClientWebifierImpl.uploadFile().

thanks, ab

On Mon, Feb 3, 2020 at 6:42 PM Adam Feder ab@backblaze.com wrote:

Running against the v3.1.0 jars, it's still giving me image/jpeg with B2_AUTO and with "image/jpeg".

On Mon, Feb 3, 2020 at 6:17 PM certainmagic notifications@github.com wrote:

It looks like that's the most recent version we've published, despite having made some number of changes.

Let me grab v3.1 and check the behavior. This level of core functionality really shouldn't have changed...ever.

ttfn, ab

On Mon, Feb 3, 2020 at 6:06 PM Dominik Boller notifications@github.com wrote:

Thanks I will check your example for a bit but for now the code looks pretty much the same.

I have tried with B2_AUTO and "image/jpeg"

Now that you mention version.. I using the maven dependency which is at 3.1.0. I just noticed the maven version is almost one year old. I assume that's not the latest ?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://github.com/Backblaze/b2-sdk-java/issues/104?email_source=notifications&email_token=ABHJFCTR2VWNHJSV3IHYEBLRBBMLNA5CNFSM4KOSC5C2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKU2ARI#issuecomment-581541957 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/ABHJFCUXSOXV6UVLTZVHU2LRBBMLNANCNFSM4KOSC5CQ

.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Backblaze/b2-sdk-java/issues/104?email_source=notifications&email_token=ABHJFCW2OJ5327RX3KSY3OTRBBNVBA5CNFSM4KOSC5C2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKU3FZY#issuecomment-581546727, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHJFCTNZAPSLP5HWCRNM2TRBBNVBANCNFSM4KOSC5CQ .

[bollerdominik]

{Authorization=removed, Content-Type=b2/x-auto, X-Bz-Content-Sha1=hex_digits_at_end, X-Bz-File-Name=test.jpg}

Just tested in on linux. Same behavior

[certainmagic]

I'm not sure if my earlier questions got lost in our back and forth, so i'll repeat them here:

Can you step through this program in a debugger?

In particular, I'm curious what headers are being set in B2StorageClientWebifierImpl.uploadFile().

thanks, ab

On Mon, Feb 3, 2020 at 6:49 PM Dominik Boller notifications@github.com wrote:

{Authorization=removed, Content-Type=b2/x-auto, X-Bz-Content-Sha1=hex_digits_at_end, X-Bz-File-Name=test.jpg}

Just tested in on linux. Same behaviour

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Backblaze/b2-sdk-java/issues/104?email_source=notifications&email_token=ABHJFCVY7OXWYAD44ZH4KP3RBBRMHA5CNFSM4KOSC5C2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKU6OLQ#issuecomment-581560110, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHJFCXHRP7LH27J4AAGZIDRBBRMHANCNFSM4KOSC5CQ .

[bollerdominik]

Those are the headers I saw

{Authorization=removed, Content-Type=b2/x-auto, X-Bz-Content-Sha1=hex_digits_at_end, X-Bz-File-Name=test.jpg}

[certainmagic]

Hi Dominik --

Those headers look fine to me.

Please email helpme@backblaze.com and let them know that you'd like to be connected to 'ab'. Please include the email address you use for your backblaze account and the name or id of the bucket you're uploading to and say that you don't mind if i fetch files from your bucket. I'd like to poke around in our database and see if I can narrow down where things are going wrong.

thanks, ab

On Tue, Feb 4, 2020 at 5:50 AM Dominik Boller notifications@github.com wrote:

Those are the headers I saw

{Authorization=removed, Content-Type=b2/x-auto, X-Bz-Content-Sha1=hex_digits_at_end, X-Bz-File-Name=test.jpg}

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Backblaze/b2-sdk-java/issues/104?email_source=notifications&email_token=ABHJFCQQ3TGHWUR3H6UMSKTRBD6ZRA5CNFSM4KOSC5C2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKWNYYA#issuecomment-581753952, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHJFCQ2PSADLY5JCBZUP63RBD6ZRANCNFSM4KOSC5CQ .

[certainmagic]

Hi Dominik --

Thanks for contacting support. I've sent you two jars that are much more recent to try with; you could also build them yourself from the master branch here. I don't expect the behavior to be any different, but let's rule it out.

I've looked and your files are being marked as "application/json" at the time of the upload, which rules out problems on the download side. I've set a few debugging hooks in our servers to try to catch more hints the next time you do an upload. Please do a few more uploads and let me know when you did them. If there's more info you want to send without posting it here, just reply to the support ticket you opened.

thanks, ab

[bollerdominik]

Hi I just did two uploads again.

• First one: With the latest maven 3.1.0 => upload is contentType='application/json'
• Second one with the jars you send me: upload is contentType='image/jpeg' !!!

So the jars seem to do the trick. However I had to add

     <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpclient</artifactId>
            <version>4.5.11</version>
        </dependency>

to my dependency for it to work and the sample program Issue104.java never seems to stop running as the new http client is doing some endless polling after upload

17:55:20.716 [Thread-0] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Closing expired connections
17:55:20.716 [Thread-0] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Closing connections idle longer than 30 SECONDS

[certainmagic]

Hi Dominik ---

What http layer were you using before if not HttpClient? Was it the OKHttp implementation?

thanks, ab

[bollerdominik]

yes!

<dependency>
    <groupId>com.backblaze.b2</groupId>
    <artifactId>b2-sdk-okhttp</artifactId>
    <version>3.1.0</version>
</dependency>

[certainmagic]

I think I see a bug in B2OkHttpClientImpl. I think it is always sending application/json, even when uploading user content. :(

Wanna try v3.1 with the HttpClient version to confirm the issue is in the OK HTTP layer?

[bollerdominik]

Jup that did it!

I do not mind using the other dependency. Is there any difference?

Btw can I suggest to please update the maven repository again? I would love to work with latest version of the b2 sdk. Thanks

[certainmagic]

The HttpClient version is the version we use here at backblaze. It is has the most mileage on it. The OkHttpClient is a newer implementation for use with Android. We were using it for our Android client, but, due to the limitations that are described in the README, we have moved away from it to an implementation based on the the native java URLConnection. The URLConnection-based implementation is in a PR and not yet merged to master.

I believe that work makes the OkHttpClient obsolete ... and i'd rather remove it than fix bugs in it. I've reopened the thread internally about getting that PR landed in master.

Yes. It has been a while since we posted a new release. I'd like to be able to get the new Android support in and then make a new release.

[certainmagic]

Thanks for filing this bug and thanks for your patience and help in tracking the issue down!

I'm going to update the title of the bug and leave it open for now.

[keithele]

The OKHttpClient which was deprecated previously has been removed in PR: https://github.com/Backblaze/b2-sdk-java/pull/114

Closing this issue since the OKHttpClient is no longer included.