.upload with FileDataPart is slow?

[mobiletoly]

This is for version 2.0.1 Don't know why, but this code (serviceUrl is localhost in my case):

val response = Fuel.upload(serviceUrl/"jars"/"upload")
    .add(FileDataPart(file = fileToUpload, name = "jarfile", contentType = "application/java-archive"))
    .awaitObject(uploadJarDeserializer)

is very slow on large files (over 100 MB). When I switch to

val response = Fuel.upload(serviceUrl/"jars"/"upload")
    .add(BlobDataPart(inputStream = fileToUpload.inputStream(), name = "jarfile", filename = fileToUpload.name, contentType = "application/java-archive"))
    .awaitObject(uploadJarDeserializer)

it is very fast, as expected.

So by replacing FileDataPart to BlobDataPart - I have improved a performance in order of magnitude. Am I doing something wrong with FileDataPart?

[SleeplessByte]

If you check here you can see that this is essentially what it's doing. So in theory, this should not have changed anything.

[SleeplessByte]

@iNoles any ideas?

[iNoles]

If this in Android, he may have to hit slow path operations on Android. He should have to profile his applications to find out where is slow path occurs.

[mobiletoly]

this is desktop app (TornadoFX-based app), running on my Mac, so no Android. Using BlobDataPart is very fast, using Apache HttpClient is very fast, it is only when I use FileDataPart I start having issues.

[SleeplessByte]

@mobiletoly yep, we believe you, we just don't understand why (yet).

Codewise, it seems that it should be the same 😅

[iNoles]

@mobiletoly Can you profile it?

[mobiletoly]

I'm traveling at the moment, so a little bit hard for me to do. So from what I can say just looking in a source code is that FileDataPart and BlobDataPart are different because in case of BlobDataPart the contentLength is null by default, while FileDataPart sets contentLength to the size of the file. If I pass contentLength parameter to BlobDataPart - it becomes very slow as well. There is obviously difference in how this parameter handled in HttpClient class:

private fun setBodyIfDoOutput(connection: HttpURLConnection, request: Request) {
        ...
        val contentLength = body.length
        if (contentLength != null && contentLength != -1L) {
            // The content has a known length, so no need to chunk
            connection.setFixedLengthStreamingMode(contentLength.toLong())
        } else {
            // The content doesn't have a known length, so turn it into chunked
            connection.setChunkedStreamingMode(4096)
        }
        ...
}

I'm wondering if setFixedLengthStreamingMode is a cause of the problem (and if this is a Java specific problem).

[SleeplessByte]

Ugh. @mobiletoly, yes you are right. setFixedLengthStreamingMode is supposed to be faster than setChunkedStreamingMode. However, this is enough information for us to do some debugging here.

Thank you for the time ❤️

[cat-ninja]

Hi! Any updates on this? Facing the same issue as author