tonyofrancis / Fetch

The best file downloader library for Android
https://www.meta.stackoverflow.com/tags/fetch2
Apache License 2.0
1.66k stars 343 forks source link

Google Drive files Size -1 #396

Closed maulik9898 closed 5 years ago

maulik9898 commented 5 years ago

When I add google drive file for downloading fetch failed to get the size and also fetch download the file as SEQUENTIAL Download but google drive files support multipart downloading.

Direct DL Link: https://doc-14-60-docs.googleusercontent.com/docs/securesc/ha0ro937gcuc7l7deffksulhg5h7mbp1/0no78ht120jh68pn1u8r6c72utbablam/1561356000000/12024208594662412395/*/1bj3kEsBDdaVolTFanb3km8XlKHvDsfgC?e=download

Original File link: https://drive.google.com/a/sot.pdpu.ac.in/uc?export=download&confirm=s1bX&id=1bj3kEsBDdaVolTFanb3km8XlKHvDsfgC

tonyofrancis commented 5 years ago

@maulik9898 when ever the server returns -1 for the content length the sequential downloader is used.

maulik9898 commented 5 years ago

oh, I see Google doesn't send Content-Length header while downloading the file.

I got a workaround for this,

if I send a request like this

GET /docs/securesc/ha0ro937gcuc7l7deffksulhg5h7mbp1/ju6b0ng23crff9cmlm340n86962qg3su/1561384800000/12024208594662412395/*/1bj3kEsBDdaVolTFanb3km8XlKHvDsfgC?e=download h2
Host: doc-14-60-docs.googleusercontent.com
range: bytes=0-255
user-agent: okhttp/4.0.0-RC1

I get response header as

x-guploader-uploadid: AEnB2Up9hjdYnbkGUXB5u2QEVaFQ-d_zQoNV6_qOVKFM98GTm2si0QuYOi5CiPJDrddU4Ji8tFtInROdaMy27aVCl6boKq1C_DmYBooqz0gIRzKLub1iNtU
access-control-allow-origin: *
access-control-allow-credentials: false
access-control-allow-headers: Accept, Accept-Language, Authorization, Cache-Control, Content-Disposition, Content-Encoding, Content-Language, Content-Length, Content-MD5, Content-Range, Content-Type, Date, GData-Version, google-cloud-resource-prefix, Host, If-Match, If-Modified-Since, If-None-Match, If-Unmodified-Since, Origin, OriginToken, Pragma, Range, Slug, Transfer-Encoding, Want-Digest, x-chrome-connected, X-ClientDetails, X-Client-Version, X-Firebase-Locale, X-GData-Client, X-GData-Key, X-Goog-AdX-Buyer-Impersonation, X-Goog-Api-Client, X-Goog-AuthUser, X-Goog-PageId, X-Goog-Encode-Response-If-Executable, X-Goog-Correlation-Id, X-Goog-Request-Info, X-Goog-Request-Reason, X-Goog-Experiments, x-goog-iam-authority-selector, x-goog-iam-authorization-token, X-Goog-Spatula, X-Goog-Upload-Command, X-Goog-Upload-Content-Disposition, X-Goog-Upload-Content-Length, X-Goog-Upload-Content-Type, X-Goog-Upload-File-Name, X-Goog-Upload-Header-Content-Length, X-Goog-Upload-Offset, X-Goog-Upload-Protocol, X-Goog-Visitor-Id, X-Goog-FieldMask, X-HTTP-Method-Override, X-JavaScript-User-Agent, X-Pan-Versionid, X-Proxied-User-IP, X-Origin, X-Referer, X-Requested-With, X-Upload-Content-Length, X-Upload-Content-Type, X-Use-HTTP-Status-Code-Override, X-Ios-Bundle-Identifier, X-Android-Package, X-Ariane-Xsrf-Token, X-YouTube-VVT, X-YouTube-Page-CL, X-YouTube-Page-Timestamp, X-Goog-Meeting-Botguardid, X-Goog-Meeting-Debugid, X-Goog-Meeting-Token, X-Client-Data, X-Sfdc-Authorization, MIME-Version, Content-Transfer-Encoding, X-Earth-Engine-Computation-Profile
access-control-allow-methods: GET,OPTIONS
content-range: bytes 0-255/430141753
content-type: application/x-zip-compressed
content-disposition: attachment;filename="_igetintopc.com_Proteus_8.7_SP3_Professional_3.zip";filename*=UTF-8''_igetintopc.com_Proteus_8.7_SP3_Professional_3.zip
date: Mon, 24 Jun 2019 15:10:02 GMT
expires: Mon, 24 Jun 2019 15:10:02 GMT
cache-control: private, max-age=0
content-length: 256
server: UploadServer
alt-svc: quic=":443"; ma=2592000; v="46,44,43,39"

and using content-range: bytes 0-255/430141753 to find the size of the file which will be 430141753 bytes

using the above solution I can show the user the size but not able to get fetch to download the file in PARALLEL mode

I found above solution from observing Request send by ADM app to find the size of the file because ADM was able to get the size of the file and also able to download in parallel

I also refer this link https://medium.com/@suvanika/multipart-download-and-byte-ranges-b5d6b7577e95 to get more info about multipart and range header

tonyofrancis commented 5 years ago

@maulik9898 I will have a look at this.

tonyofrancis commented 5 years ago

@maulik9898 Thanks for reporting your finds. I was able to integrate getting the content length from the Content-Range header if the Content-Length header is not available. This fix will go into the next release. I am pushing for today or tomorrow.

tonyofrancis commented 5 years ago

@maulik9898 new version of Fetch 3.0.9 and androidx 3.1.3 released with this fix.

maulik9898 commented 5 years ago

it's still not working. Size shows as -1. I was able to pinpoint the problem


fun getContentLengthFromHeader(headers: Map<String, List<String>>, defaultValue: Long): Long {
    var contentLength = defaultValue
    if (headers.containsKey("Content-Length")) {
        val size = headers["Content-Length"]?.firstOrNull()?.toLongOrNull()
        if (size != null && size > 0) {
            contentLength = size
            return contentLength
        }
    }
    if (headers.containsKey("Content-Range")) {
        val value = headers["Content-Range"]?.firstOrNull()
        if (value != null) {
            val index = value.lastIndexOf("/")
            if (index != -1 && ((index + 1) < value.length)) {
                val sizeText = value.substring(index + 1)
                val size = sizeText.toLongOrNull()
                if (size != null && size > 0) {
                    contentLength = size
                    return contentLength
                }
            }
        }
    }
    return contentLength
}

the reason is above code is not case insensitive because google send header in small caps content-range: bytes 0-255/430141753

tonyofrancis commented 5 years ago

Google never follow protocol do they. Ill fix soon

tonyofrancis commented 5 years ago

@maulik9898 This issue is now fixed in new Fetch version 3.0.10/Androidx 3.1.4

maulik9898 commented 5 years ago

Thx for the update

maulik9898 commented 5 years ago

@tonyofrancis works perfectly