ssseasonnn / RxDownload

A multi-threaded download tool written with RxJava and Kotlin
Apache License 2.0
4.14k stars 612 forks source link

It always timeout when I download AWS CDN resources with RxDownload4 #355

Closed yue001 closed 1 year ago

yue001 commented 1 year ago

It always timeout when I download AWS CDN resources with RxDownload4, and as I retry the task manually, the next rang download will timeout again, following is the log:

D/OkHttp: <-- HTTP FAILED: java.net.SocketTimeoutException: timeout
D/RxDownload: {Range=bytes=20971520-26214399}
ssseasonnn commented 1 year ago

maybe you can change okhttp timeout setting, increase the timeout seconds, and try again

yue001 commented 1 year ago

I tried that, but it didn't work. Now I overwrite the request and rebuild a new OkHttpClient when the get() called, then it works well, but when I stoped the task while downloading ,and restart the task, it'll timeout, I suspect that connectFlowable.connect() is bad after manager.stop(), what do you think?

yue001 commented 1 year ago

I disabled the keep-alive of the connection by adding the header "Connection: close", and it's done well. It looks like the AWS CDN platform does not support connection reuse, I have raised the issue with them

yue001 commented 1 year ago

After cooperating with AWS technicians to investigate, the cause of the problem has been confirmed. The AWS CDN platform uses the http2.0 protocol. The first request of the download task (Range: bytes=0-) does not read the data, and the connection of each subsequent range request is reused of the first request. but the data (bytes=0-) in the previous link has not been read, and the flow control information has not been updated, resulting in request congestion, similar to the OKHttp issue https://github.com/square/okhttp/issues /6749

I modify the first detection request at the application layer to use the HEAD request, that is, not requesting file data, which can solve the problem:

if("bytes=0-".equals(headers["Range"])){
    return request()<HeadExporeApi>()
        .get(url, headers, params)
        .map {
            if(it. raw(). isSuccessful)
                Response.success(EmpytResponseBody, it.raw())
            else
                Response. error(EmpytResponseBody, it. raw())
         }
    }else{
        return api.get(url, headers, params)
}

The operation of sending the first detection request by HEAD method, maybe can be placed in RxDownload ; @ssseasonnn