Redundant HTTP header : application/x-www-form-urlencoded

[Arman92]

I'm using tus-java-client in android and here is the code snippet:

` TusUpload upload = new TusUpload(new File(filePath));

                    TusUploader uploader = client.resumeOrCreateUpload(upload);
                    long totalBytes = upload.getSize();
                    long uploadedBytes = uploader.getOffset();

                    // Upload file in 10KB chunks
                    uploader.setChunkSize(10 * 1024);

                    MsgUpdloadStatus updloadStatus = new MsgUpdloadStatus(message,
                            InstantMessage.UploadStatusType.UPLOADING);

                    while (!shouldStop && uploader.uploadChunk() > 0) {
                        uploadedBytes = uploader.getOffset();

                        updloadStatus.setProgress((int) (uploadedBytes / totalBytes));
                        eventBus.send(updloadStatus);
                    }

                    // finish method here can be used for pausing the ongoing upload
                    uploader.finish();

                    if (uploadedBytes >= totalBytes) {
                        updloadStatus.setProgress(100);
                        updloadStatus.uploadStatus = InstantMessage.UploadStatusType.UPLOADED;
                        eventBus.send(updloadStatus);
                    }
                }

            } catch (InterruptedException | ProtocolException | IOException e) {
                e.printStackTrace();
            }`

And the server side is using the prebuilt Go binaries, in server for the first Http header I'm getting [tusd] event="ResponseOutgoing" status="400" method="POST" path="" error="missing or invalid Content-Type header

I searched in sent headers and surprisingly saw this one: application/x-www-form-urlencoded

And the server is not expecting such header, I found no place in java client that is adding this header to HttpHeaders.

[Acconut]

Thank you for reporting. This is definitely not intended; looking into this now.

[Acconut]

I was able to track the issue down: The HttpUrlConnection implementation used in Android (which is not always the same as for every JDK) sets the Content-Type: application/x-www-form-urlencoded header if no other is specified (see https://github.com/frohoff/jdk8u-dev-jdk/blob/da0da73ab82ed714dc5be94acd2f0d00fbdfe2e9/src/share/classes/sun/net/www/protocol/http/HttpURLConnection.java#L611-L612). Previously, this would not conflict with tusd. However, a recent change made the server either require the Content-Type header to be absent or to equal application/offset+octet-stream (see https://github.com/tus/tusd/blob/9a5646ad6c63b99edcd5e080d0b36dc9599b303c/unrouted_handler.go#L196-L199). There are two options to patch this issue: Firstly, make sure the wrong Content-Type header does not get added by Android/Java or, secondly, make tusd ignore unexpected values for the header instead of erroring out. I am still not sure which one is the better option but both would work as I already tried them. I will think a bit about this situation and then give you more details.

Anyway, if you use the prebuilt binaries for 0.5.2 (https://github.com/tus/tusd/releases/tag/0.5.2) you should not encounter these issues since it does not contain the server change I talked about.

[joshuadiezmo]

after i allow the invalid content type header, i got another error.

Prototype not found.

[Acconut]

Prototype not found.

I can't get much from this message. Do you kind explaining where you actually get this error?

[joshuadiezmo]

after i bypass the error Content-Type: application/x-www-form-urlencoded.

the android app failed the upload says Prototype not found.

[Acconut]

Unfortunately, I am still having issues understanding your situation. I would like to help you but it's mostly impossible if you're not more specific.

after i bypass the error Content-Type: application/x-www-form-urlencoded.

How do you bypass the incorrect content type? tusd v0.6.0 has been released and simply ignored unknown content types. Therefore no such behavior should occur if you are using the latest version.

the android app failed the upload says Prototype not found.

I have never heard of this error message. Please provide either and screenshot, a stack trace or better - even both. Thank you.

[joshuadiezmo]

Sorry. its Protocol not found.

[joshuadiezmo]

I already fix my problem.

The problem is when I upload, the node server did not return the url protocol.

so I modify the server.

[kvz]

@bhstahl Perhaps // is good for browsers, but I guess other types of clients might not be able to handle it (yet)

[joshuadiezmo]

can I contribute to node server and push my fixes?

[Acconut]

@kvz Yes, we had this issue in the tus-js-client also. It's quite annoying but since the HTTP specification allows returning relative URLs in the Location, the clients must be able to handle these cases.

@ReverseFlash28 Thank you for this clarification. I am not sure whether this is actually considered a bug inside tus-node-server as they did this change on purpose: https://github.com/tus/tus-node-server/issues/48

Besides that, I will add a patch which allows tus-java-client to deal with relative URLs, so this issue should not occur again.

[bhstahl]

(Apologies for the delay here, I was traveling)

@kvz I agree, and its important to think about non-browser usage. Will open an issue to change that up

[Acconut]

@bhstahl Thank you for considering this choice.

Closing this issue as the original and the later problem have been fixed.