Apt update fails - Githubissues

glitchphan commented 8 years ago

Not sure if I'm the only one with this problem.

$ apt update Get:1 http://termux.net stable InRelease [965 B] Err:1 http://termux.net stable InRelease Clearsigned file isn't valid, got 'NOSPLIT' (does the network require authentication?) Fetched 965 B in 4s (196 B/s) Reading package lists... Done W: Failed to fetch http://termux.net/dists/stable/InRelease Clearsigned file isn't valid, got 'NOSPLIT' (does the network require authentication?) E: Some index files failed to download. They have been ignored, or old ones used instead.

glitchphan commented 8 years ago

I know I'm online, etc

$ telnet termux.net 80

HTTP/1.1 400 Bad Request Date: Fri, 08 Apr 2016 02:38:43 GMT Content-Type: text/html Content-Length: 177 Connection: close Server: -nginx CF-RAY: -

<html> <head><title>400 Bad Request</title></head> <body bgcolor="white"> <center><h1>400 Bad Request</h1></center> <hr><center>cloudflare-nginx</center> </body> </html> Connection closed by foreign host

fornwall commented 8 years ago

Do the problem remain? Can you visit http://termux.net/dists/stable/InRelease in the browser?

glitchphan commented 8 years ago

Still having the problem. I see a pgp signed message and signature when I open in browser.

glitchphan commented 8 years ago

When I grab that URL from within Termux, though, I get this.

$ wget -O - http://termux.net/dists/stable/InRelease Connecting to termux.net (104.18.37.234:80) }S�r�F��+�'=T��!��ͮ��͌@��+;ɋv��{�ӌ��('�p�KFq;��i��W�pڍeA��3{P{˳�;ۢ\��0��ٚ�e#T�]��}�f��J��m[�b}<� ��s^܂�ŏ�Bn��a~�#�0��H@��Y+��B!3K�:g�� 3A, �!�!e�"m�e�1 �� <�{Au�>��l�ҖAh0��JJ7 a��T0"5jC��G 1�.��kg��^�8Dc��(?Ou ��Kn-�R��c8��B�AB�g!!�2��1�2l-ό0�hH�q-Ugē3l���t��v9�^�BbN�@3�6�x�8��!̴� $�s�]PA�7�� b� /]a��"S�g� ��YG<��~%�:��1�̬�ƬsR��

�OU+J��dPKb��2��Z!�i�=�?5�4��

'#��2o�S�$�n�]�h6��_N�qV�il�Q�lEQ�L�/-6̣(�ǉ�Θ�7iq��n}��͞fy�D�Gp{^�ɶS?�@��-�x?�n�MQn�qUL�Q�'˗ٴD��I�7��d��/yj��9�o�6n_#Z�e�T��~1��%;fd��ɚ�#�r�#�4�e�ќ5mpݭQm��gcVU��+k'��Xی��éw��6��Yk�zܔ�n�s$�C֍� j��n�Mo��m:�L�ן�x7�Ѭ��\��i��Q?Y/��'ͼV�~�9-��#��- 100% |*****| 968 0:00:00 ETA $

fornwall commented 8 years ago

@glitchphan The wget output is strange. Could you try both on a separate network and see if you get the same problem? Also, could you save the file (omit the -O option) and put up the file somewhere so I can exame it?

glitchphan commented 8 years ago

https://drive.google.com/file/d/0B-A2cTCaFGMNWV9SWDNKUHJ0OW8/view?usp=sharing

I definitely get a different file on a different computer/network. Still need to test the phone on another network as well as another device tethered to the phone.

fornwall commented 8 years ago

Interesting! The file you receive is actually InRelease in gzipped format (seen by using zcat InRelease). I'll check if it could be from my servers end, please check from your end by varying the network if possible!

Could you try to access the following two urls with wget and see if they return the gzipped data as well?

1: https://termux.net/dists/stable/InRelease 2: http://apt.termux.com/dists/stable/InRelease

glitchphan commented 8 years ago

The https link seems to be cleartext file but the second one gives a gzipped file again.

I finally got a chance to do an apt update on another network over WiFi and it worked fine. That's how I was able to get curl to try https :). Very strange that the network matters. Maybe my data is going through some proxy, even though I don't have one set in APN settings.

fornwall commented 8 years ago

@glitchphan Which network is the one returning the gzipped resource?

It would be helpful if you could run this and paste the output to see the http headers:

curl -H "Accept-Encoding: gzip" -D- -O http://termux.net/dists/stable/InRelease

fornwall commented 8 years ago

Also run once without the Accept-Encoding header:

curl -D- -O http://termux.net/dists/stable/InRelease

glitchphan commented 8 years ago

I'm on AT&T MVNO for my phone service, and on that network when I'm getting the gzipped stuff. I did both of those commands and got slightly different sized gzipped files.

$ curl -D- -O http://termux.net/dists/stable/InRelease % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --: 0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0HTTP/1.1 200 OK Date: Sun, 24 Apr 2016 16:10:08 GMT Content-Type: text/plain Set-Cookie: __cfduid=d5b0db4b545641fa7fd7983f9bf5804921461514208; expires=Mon, 24-Apr-17 16:10:08 GMT; path=/; domain=.termux.net; HttpOnly Last-Modified: Sun, 24 Apr 2016 12:48:15 GMT ETag: "4034ed-5c0-5313a793e0cf5" Accept-Ranges: bytes Server: cloudflare-nginx CF-RAY: 298ad2dacb2e1edd-SJC Content-Length: 969 Content-Encoding: gzip

0 969 0 0 0 0 0 0 --:100 969 100 969 0 0 408 0 0:00:02 0:00:02 --:--:-- 426 $ mv InRelease InRelease.without $ curl -H "Accept-Encoding: gzip" -D- -O http://termux.net/dists/stable/InRelease % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0HTTP/1.1 200 OK Date: Sun, 24 Apr 2016 16:11:06 GMT Content-Type: text/plain Transfer-Encoding: chunked Set-Cookie: __cfduid=d70d6fb9c5e520185d55900a290256aaa1461514266; expires=Mon, 24-Apr-17 16:11:06 GMT; path=/; domain=.termux.net; HttpOnly Last-Modified: Sun, 24 Apr 2016 12:48:15 GMT ETag: W/"4034ed-5c0-5313a793e0cf5" Server: cloudflare-nginx CF-RAY: 298ad443e4e3071f-SJC Content-Encoding: gzip

0 0 0 0 0 0 0 0 --:100 988 0 988 0 0 1001 0 --:--:-- --:--:-- --:--:-- 1148 $ diff InRelease InRelease.without Files InRelease and InRelease.without differ $ zcat InRelease > one $ zcat InRelease.without > two $ diff one two $

fornwall commented 8 years ago

Thanks! Finally, can you try appending a parameter t=1 and see if that removes the Content-Encoding: gzip header in the response? Like in:

curl -D- -O http://termux.net/dists/stable/InRelease?t=1

glitchphan commented 8 years ago

Nope, still there.

$ curl -D- -O http://termux.net/dists/stable/InRelease?t=1 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --: 0 0 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:-- 0HTTP/1.1 200 OK Date: Mon, 25 Apr 2016 01:36:43 GMT Content-Type: text/plain Set-Cookie: __cfduid=d756e468c7d7d313c024c91e9969119e21461548203; expires=Tue, 25-Apr-17 01:36:43 GMT; path=/; domain=.termux.net; HttpOnly Last-Modified: Sun, 24 Apr 2016 17:22:06 GMT ETag: "402624-5c0-5313e4c9b945c" Accept-Ranges: bytes Server: cloudflare-nginx CF-RAY: 298e10cf011941cf-SJC Content-Length: 970 Content-Encoding: gzip

100 970 100 970 0 0 381 0 0:00:02 0:00:02 --:--:-- 1750 $

fornwall commented 8 years ago

Ok, could you try again with the same curl command as the last? I added the Vary: Accept-Encoding header from my servers end.

glitchphan commented 8 years ago

Same thing with all regular http tries.

Content-Encoding: gzip

fornwall commented 8 years ago

Ok, let's try again if we can avoid Content-Encoding: gzip :). Could you try again with a new url?

curl -D- -O http://termux.net/dists/stable/InRelease?t=1

glitchphan commented 8 years ago

Success! :)

$ curl -D- -O http://termux.net/dists/stable/InRelease?t=1 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --: 0 0 0 0 0 0 0 0 --: 0 0 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:-- 0HTTP/1.1 200 OK Date: Fri, 29 Apr 2016 07:09:21 GMT Content-Type: application/octet-stream Content-Length: 1472 Set-Cookie: __cfduid=ddd47b30deca797e4e15a1102263105971461913761; expires=Sat, 29-Apr-17 07:09:21 GMT; path=/; domain=.termux.net; HttpOnly Last-Modified: Thu, 28 Apr 2016 14:06:11 GMT ETag: "402879-5c0-5318c074f9729" Accept-Ranges: bytes Server: cloudflare-nginx CF-RAY: 29b0ed926d730d97-SJC

91 1472 91 1348 0 0 522 0 0:100 1472 100 1472 0 0 553 0 0:00:02 0:00:02 --:--:-- 580 $ file InRelease\?t\=1 InRelease?t=1: ASCII text

fornwall commented 8 years ago

Great :)! Does apt update work on the device now?

glitchphan commented 8 years ago

It does! Thank you!

termux / termux-app

Apt update fails #78