search

nextcloud / android

📱 Nextcloud Android app
https://play.google.com/store/apps/details?id=com.nextcloud.client
GNU General Public License v2.0
4.31k stars 1.78k forks source link

Downloads fail when Transfer-Encoding = chunked #113

Closed josh4trunks closed 5 years ago

josh4trunks commented 8 years ago

Actual behaviour

Nextcloud app version: 1.1.0

This bug is also present is the ownCloud android app. https://github.com/owncloud/android/issues/1128 The fix seems to be discussed breifly there, though I don't see any commits referenced.

tobiasKaminsky commented 8 years ago

Can you help me and and explain where and what to set in the server environment to reproduce this behaviour?

josh4trunks commented 8 years ago

In my case chunking happens because I have Varnish in front of my nginx webserver compressing text files with gzip. I don't know how to reproduce this directly but maybe people in the owncloud bug know. possibly by enabling gzip chunked is used?

tobiasKaminsky commented 8 years ago

I thought it is a supported server option... With this I doubt that we can support this.

josh4trunks commented 8 years ago

I believe there is a way to have chunked encoding without gzip enabled. It is part of the HTTP/1.1 specification. But we can just wait till this is fixed in the ownCloud android client and apply the same changes.

johnwmail commented 7 years ago

Can you help me and and explain where and what to set in the server environment to reproduce this behaviour?

Hi, OpenBSD's httpd is enable/force chunked encoding by default.

johnwmail commented 7 years ago

I still have this problem, how to solve it. Any workaround?

josh4trunks commented 7 years ago

In my environment, where this problem was caused by some type of incompatibility with Varnish, I was able to get this working by setting beresp.do_stream = false whenever I enable gzip for content-type = text/application.

Maybe you can check what Varnish is doing wrong/weird with do_stream

johnwmail commented 7 years ago

So, still no fix on client side?

mario commented 7 years ago

Does this happens only on 3/4g or also on wifi?

mario commented 7 years ago

Also, can you inspect the headers and tell me what does it says for content-size?

johnwmail commented 7 years ago

Yes, both 3/4g and wifi also have this problem.

I don't know how to inspect the header, this is https, tcpdump not work, I will try to Google it later

Thanks

johnwmail commented 7 years ago

Hello, any idea how to fix it?

tobiasKaminsky commented 7 years ago

@josh4trunks can you send us also credentials for a test account to android at nextcloud dot com? Then I can have a look, but as oC also does not have a solution, I am unsure if we can do it...

johnwmail commented 7 years ago

@tobiasKaminsky I would also setup test account (with Transfer-Encoding = chunked) for this. But I do not want to post login/password here, would you send email to me?

johnw.mail at gmail dot com

Thanks.

josh4trunks commented 7 years ago

@tobiasKaminsky I already fixed this in my production environment. See https://github.com/nextcloud/android/issues/113#issuecomment-288131898

Did you still want an account on my working Nextcloud install?

tobiasKaminsky commented 7 years ago

Ahh, I overread that you have solved it and that is (more or less) a server/environment side problem. So I will close this.

mario commented 7 years ago

Please re-open. This has to be solved client side as well.

On Tue, 16 May 2017 at 08:51, Tobias Kaminsky notifications@github.com wrote:

Ahh, I overread that you have solved it and that is (more or less) a server/environment side problem. So I will close this.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/nextcloud/android/issues/113#issuecomment-301691838, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAWskILYtOYij5qArpmULXnoDiYM18xks5r6UeDgaJpZM4JIfc7 .

johnwmail commented 7 years ago

@mario agree, this should be solved client side as well. I tried talk to OpenBSD people,

http://marc.info/?l=openbsd-misc&m=149422158413997&w=2

They quoted RFC7230

"A recipient MUST be able to parse and decode the chunked transfer coding."

I would like to setup nextcloud AC with "chunked transfer coding" enabled.

Email me, if people who want/need the AC to debug/testing

johnw dot mail at gmail dot com

Thanks.

johnwmail commented 7 years ago

https://tools.ietf.org/html/rfc7230#section-4.1

tobiasKaminsky commented 7 years ago

edit: to warm here...searching for httpclient "Chunked Transfer Encoding" gives enough :-)

https://stackoverflow.com/questions/6124233/how-to-handle-read-the-response-with-transfer-encodingchunked

tdm4 commented 7 years ago

Having the same problem here. Using nginx instead of httpd for now (on OpenBSD) with chunked_transfer_encoding off; Not ideal, but it works. Android client breaks RFC7230 unfortunately. :(

johnwmail commented 7 years ago

Hi @tdm4

I written a dirty hack to disable chunked_transfer_encoding for OpenBSD httpd, Maybe you will interesting

http://marc.info/?l=openbsd-misc&m=149422389314402&w=2

rpcameron commented 7 years ago

Is there any activity on this, or are the Nextcloud devs going to wait on Owncloud to fix this since the download function/code seems to be in their Android library code?

Until this issue is resolved, I cannot use either client. Yes, I see the patch for OpenBSD httpd, but I'm not going to modify my system software to fix a bug that exists in other third-party software because the third-party software is broken and doesn't properly implement their HTTP client.

mario commented 7 years ago

Actually our library differs from theirs in quite a lot of things...

But yea, hopefully soon...

On Thu, 24 Aug 2017 at 15:50, rpcameron notifications@github.com wrote:

Is there any activity on this, or are the Nextcloud devs going to wait on Owncloud to fix this since the download function/code seems to be in their Android library code?

Until this issue is resolved, I cannot use either client. Yes, I see the patch for OpenBSD httpd, but I'm not going to modify my system software to fix a bug that exists in other third-party software because the third-party software is broken and doesn't properly implement their HTTP client.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nextcloud/android/issues/113#issuecomment-324640891, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAWslBpYsARE3dUhKRYBoZx79BaHYHuks5sbX-mgaJpZM4JIfc7 .

rpcameron commented 7 years ago

@mario Ah, sorry. When I was trying to sort through the code I must have followed a wrong reference when tracking source files and includes. Does this mean that Nextcloud will soon finally support this, so I no longer have field support problems from users using Android devices? Or will they still need to be redirected to some other app to access their files on their phones ...

johnwmail commented 7 years ago

@rpcameron Is it owncloud, already fixed this bug?

rpcameron commented 7 years ago

@johnwmail No, it is still open there, too (1128, since 31 Aug 2015). It has been acknowledged, constantly postponed, and now is "backlogged".

edermi commented 7 years ago

Same issue here. Would love to access the nextcloud hosted on my OpenBSD with the Android App

rpcameron commented 7 years ago

Is there any movement on this bug? I recently saw that version 2 RCs are available so I decided to give them a shot, but no luck. It almost seems there's no interest from the developers to fix this.

(And yes, I know the old adage about fixing it yourself if you want it done. I have investigated it to try and find the relevant code, but I haven't programmed in Java in over a decade, and trying to find the relevant functions left me in a knot bouncing from class to class across the different repos.)

tobiasKaminsky commented 7 years ago

I tried it, but it is not that easy to fix. I can have a look again once I have a little more time :-/

rpcameron commented 7 years ago

So, 2.0 was just released, but this bug that's been present since the beginning has no movement.

This app is still useless for accessing files on Android if chunked downloads are not supported as per the HTTP spec.

xstasi commented 7 years ago

Hi there, poking in to say that I too am afflicted by this issue.

+1

tobiasKaminsky commented 7 years ago

Can someone provide an test account on a server where this is happening? Please send it to android at nextcloud dot com with a reference to this issue.

rpcameron commented 7 years ago

@tobiasKaminsky I just sent an email with login details for a test account on an OpenBSD 6.2 server running its httpd, which only serves with "transfer encoding = chunked". Maybe this means he Android client will finally support the HTTP spec ...

tobiasKaminsky commented 7 years ago

@rpcameron thank you. I do my very best :-)

tobiasKaminsky commented 7 years ago

@xstasi thank you also for the credentials 👍

tobiasKaminsky commented 7 years ago

I guess I have a fix for this: https://github.com/nextcloud/android-library/pull/98

We will soon have our dev version back again. This builds a nearly nightly snapshot from master. So once the fix is merged, you can give it a try and report back :-)

rpcameron commented 6 years ago

Just ran a quick test with the latest Dev version (20171204), and it appears to be downloaded files (at long last). Thanks to the devs for finally addressing this issue ... Hopefully a stable release will soon make its way out.

AndyScherzinger commented 6 years ago

https://github.com/nextcloud/android-library/pull/98 did fix it and has been integrated with master and with the new dev builds now been shipped for the dev version. The next stable version doesn't have a date yet but is also progressing quite well: https://github.com/nextcloud/android/milestones/Nextcloud%20App%202.1.0

johnwmail commented 6 years ago

Where can we download the dev app?

AndyScherzinger commented 6 years ago

@johnwmail it is distributed via f-droid: https://f-droid.org/packages/com.nextcloud.android.beta/ (so best is to install the f-droid client and Search for "Nextcloud Dev").

tobiasKaminsky commented 6 years ago

Please note that fdroid currently does not have the latest version, it being built this day and will be available soon.

Or take a look here: https://github.com/nextcloud/android/blob/master/CONTRIBUTING.md#dev-release

levaidaniel commented 5 years ago

Hi there!

Is it normal that version 3.5.1 (March, 18, 2019) still doesn't download anything from OpenBSD httpd(8)? Is this mentioned fix supposed to be in this version?

johnwmail commented 5 years ago

Hi, they fixed a year.

tobiasKaminsky commented 5 years ago

Is it normal that version 3.5.1 (March, 18, 2019) still doesn't download anything from OpenBSD httpd(8)? Is this mentioned fix supposed to be in this version?

Can you create us a test account, test if the problem occurs also there and if so send the credentials to tobias at nextcloud dot com with a reference to this issue?

levaidaniel commented 5 years ago

Sure thing, I sent the mail containing the info.

levaidaniel commented 5 years ago

In the meantime, here are the logs from the application and Chrome, respectively. Maybe these shed some light into the problem:

Android app Chrome

levaidaniel commented 5 years ago

Interesting thing is using the same app on Android 7.1.1 yields more or less positive results over using the same v3.5.1 app on Android 9. The former only fails sometimes, usually after waiting for a while after a successful download -- it says the file couldn't be synced so using the downloaded version (or something along these lines). But other than that, on older Android the success rate is definitely higher. On Android 9 I was never ever able to download anything (even though using the same application version: 3.5.1).

tobiasKaminsky commented 5 years ago

Sure thing, I sent the mail containing the info.

I logged into testaccount and download Nextcloud Flyer.pdf and the AEG manual. Both work fine with emulator and android 9.0.

levaidaniel commented 5 years ago

Thanks for trying it out!

Is there anything in the logs I pasted here that could point to an error that might cause this?