Not able to download spring through any UI

[AppSynergy]

N.B. Since I upgraded to Ubuntu 14.04, I've had very little luck getting SWL to behave itself. I had some time today, and tried to identify and resolve as many of issues as I could. I deleted and rebuild SWL before identifying these issues. These issues may well be related.

If I create or join a multiplayer battle, SWL attempts to download an engine (which is always 96.0, even if the room is running e.g. 91.0). This download never succeeds:

Problem downloading: [Error] ../../tools/pr-downloader/src/main.cpp:180:main(): No engine version found for 96.0

If I attempt a manual download, e.g. entering 91.0 and selecting Engine, this has the same result.

Problem downloading: [Error] ../../tools/pr-downloader/src/main.cpp:180:main(): No engine version found for 91.0

https://github.com/spring/pr-downloader/blob/master/src/main.cpp#l180

If I really-manually download and use that, that seems to work.

[db81]

This is a pr-downloader bug that has plagued us for a while and I'm not sure how to reproduce it. The few times this happened to me it started working after a short while. I suspect this has to do with the server it polls for updates being unavailable randomly. If you can reproduce this reliably, that's certainly interesting.

It downloads 96.0 for 91.0 rooms because the unitsync in 91.0 was segfaulting too much to bear. So now it uses 96.0 for syncing in 91.0 rooms. abma hates this hack, but at least swl doesn't crash when the map changes to Iced Coffee.

[AppSynergy]

I can reproduce this 100% at the moment, so let me know if I can provide any env info etc. to help understand it.

[AppSynergy]

Some console output when I load up, go straight to download and attempt 91.0:

adam@MotherOfInvention:~$ Games/Weblobby/weblobby
[INFO]    springHome is "/home/adam/.spring"
[INFO]    Connecting to uberserver on lobby.springrts.com:8200
[INFO]    Running command (Downloading Engine 91.0):
/home/adam/.spring/weblobby/pr-downloader/pr-downloader --filesystem-writepath /home/adam/.spring/weblobby --download-engine 91.0
[WARNING] http://weblobby.springrts.com/qt/lwidgets/DownloadManager.js:174 Problem downloading: [Error] ../../tools/pr-downloader/src/main.cpp:180:main(): No engine version found for 91.0
[INFO]    Command finished: Downloading Engine 91.0
[INFO]    Deleted spring settings: "/home/adam/.spring/weblobby/engine/91.0/springsettings.cfg"
[INFO]    Loading unitsync at "/home/adam/.spring/weblobby/engine/91.0/libunitsync.so"
[INFO]    Deleted spring settings: "/home/adam/.spring/weblobby/engine/91.0/springsettings.cfg"

Where I've removed the mutiple occurances of the following line, which appears to get spammed every couple of seconds regardless of what you're doing:

 libpng warning: Ignoring extra png_read_update_info() call; row buffer not reallocated

(which I spotted sourced here?)

[db81]

Try running /home/adam/.spring/weblobby/pr-downloader/pr-downloader --filesystem-writepath /home/adam/.spring/weblobby --download-engine 91.0 directly, it should show more info.

[AppSynergy]

adam@MotherOfInvention:~$ /home/adam/.spring/weblobby/pr-downloader/pr-downloader --filesystem-writepath /home/adam/.spring/weblobby --download-engine 91.0
pr-downloader 0.7-146-g620bd64
[Info] Using filesystem-writepath: /home/adam/.spring/weblobby
[Error] ../../tools/pr-downloader/src/pr-downloader.cpp:83:search(): engine dl
[Error] ../../tools/pr-downloader/src/main.cpp:180:main(): No engine version found for 91.0
[Info] Download complete!

NB: download is not complete. ^0^

[AppSynergy]

https://github.com/spring/pr-downloader/blob/master/src/pr-downloader.cpp#l81

//engine downloads only work with http -- what else would I be using?

So my IDownload::category != IDownload::CAT_ENGINE_LINUX64, I guess?

There is fixme on 40...

[db81]

It gives [Error] ../../tools/pr-downloader/src/pr-downloader.cpp:83:search(): engine dl to me too and then proceeds to download everything just fine (after printing a bunch of other errors).

[db81]

If you have/willing to install a complier and necessary deps you could run cmake -DCMAKE_BUILD_TYPE=Debug to compile it with debugging output enabled and see if it says [Debug] /home/user/dev/pr-downloader/src/Downloader/Download.cpp:122:addMirror(): http://springrts.com/dl/buildbot/default/master/91.0/linux32/spring_91.0_minimal-portable-linux32-static.7z at all...

[AppSynergy]

Install the project...
-- Install configuration: "Debug"
-- Installing: /usr/local/bin/pr-downloader

adam@MotherOfInvention:~/Dev/pr-downloader$ pr-downloader --filesystem-writepath /home/adam/.spring/weblobby --download-engine 91.0
pr-downloader 0.7-160-ge347557
[Info] Using filesystem-writepath: /home/adam/.spring/weblobby
[Error] /home/adam/Dev/pr-downloader/src/pr-downloader.cpp:83:search(): engine dl
[Debug] /home/adam/Dev/pr-downloader/src/Downloader/Http/HttpDownloader.cpp:39:search(): 91.0
[Debug] /home/adam/Dev/pr-downloader/src/main.cpp:92:download(): Couldn't find 91.0
[Error] /home/adam/Dev/pr-downloader/src/main.cpp:180:main(): No engine version found for 91.0
[Debug] /home/adam/Dev/pr-downloader/src/pr-downloader.cpp:219:DownloadStart(): Nothing to do, did you forget to call DownloadAdd()?
[Info] Download complete!

[abma]

works for me. file seems to exist, too: http://api.springfiles.com/?springname=91.0&category=engine_linux64

not sure if category is correct, maybe engine_linux32, which does exist, too.

is a firewall/proxy used? is /etc/hosts broken/does it contain weird entries?

[AppSynergy]

I've got a x86_64 chip, so assuming it should be engine_linux64, yep.

No firewalls/proxies (of my doing, can only think to blame ISP) and /etc/hosts should only contain localhost line (from memory). Will check at home later.

[AppSynergy]

/etc/hosts:

127.0.0.1   localhost
127.0.1.1   MotherOfInvention

# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

which doesn't seem offensive or unusual.

Proxy sanity check: http://amibehindaproxy.com/

Your IP address:    80.65.246.182
Proxyserver:    No proxyserver detected.
You do not have a proxy server or filtered internet activated.

N.B. I'm only still chasing this issue because I'm a puzzle addict. Are we having fun? ;)

[db81]

You can wireshark it and see exactly how the search fails. Or run it through a debugger.

[abma]

@AppSynergy

i've added some verbose messages to current pr-downloader. if you compile it again, it maybe shows some useful info...

https://github.com/spring/pr-downloader/commit/89ea4904169a7f84cbd05621bcca8d3097e049db

[db81]

@AppSynergy, any interest in investigating it further or did you run out of puzzle addiction fuel? :P

[db81]

Not a swl bug, as evidenced by the issue manifesting when pr-downloader is run manually. I'm still interested in it though.

[AppSynergy]

Sorry for late feedback, work, illness etc.

Re: "Not a swl bug, as evidenced by" -- seems true.

Re: @abma verbose messages:

HttpDownloader.cpp:107:search(): Returned xml isn't an array!

LOG_DEBUG("%s", typeid(result).name()  );     --> N6XmlRpc11XmlRpcValueE
LOG_DEBUG("%s", result.getType()  );             --> (null)

No idea what to make of this, literally the first C++ exposure I've had in a decade.

[abma]

can you give the command line when you get this result?

there is NO access from 80.65.246.182 at http://api.springfiles.com/ ... your internet is "broken"!

can you access http://api.springfiles.com/ in your browser? do you have some local firewall/virus scanner which blocks access?

please provide all info/log output and not only snippets.

[AppSynergy]

Whaddya mean, NO access from 80.65.246.182?

N.B. I don't have a static IP, so that would have been the address my ISP had assigned on the day I checked amibehindaproxy. FYI today's external IP is: 87.254.77.9

Yes, I can access http://api.springfiles.com.

==> "Input is case sensitive! Use * (multiple chars) or ? (single char) as wildcard. 10 Results are returned at max." etc....

No virus scanners. Only firewall I can think of ufw, which when disabled, doesn't impact result.

adam@MotherOfInvention:~$ pr-downloader --filesystem-writepath /home/adam/.spring/weblobby --download-engine 91.0
pr-downloader 0.7-162-g1b60b53
[Info] Using filesystem-writepath: /home/adam/.spring/weblobby
[Error] /home/adam/Dev/pr-downloader/src/pr-downloader.cpp:83:search(): engine dl
[Debug] /home/adam/Dev/pr-downloader/src/Downloader/Http/HttpDownloader.cpp:67:search(): 91.0
[Debug] /home/adam/Dev/pr-downloader/src/Downloader/Http/HttpDownloader.cpp:70:search(): springfiles.search
[Info] XmlRpcCurlClient new client: host api.springfiles.com, port 80.
[Debug] /home/adam/Dev/pr-downloader/src/Downloader/Http/HttpDownloader.cpp:76:search(): N9IDownload8categoryE
[Info] XmlRpcCurlClient::execute: method springfiles.search.
[Debug] /home/adam/Dev/pr-downloader/src/Downloader/Http/HttpDownloader.cpp:105:search(): N6XmlRpc11XmlRpcValueE
[Debug] /home/adam/Dev/pr-downloader/src/Downloader/Http/HttpDownloader.cpp:106:search(): (null)
[Error] /home/adam/Dev/pr-downloader/src/Downloader/Http/HttpDownloader.cpp:107:search(): Returned xml isn't an array!
[Debug] /home/adam/Dev/pr-downloader/src/main.cpp:92:download(): Couldn't find 91.0
[Error] /home/adam/Dev/pr-downloader/src/main.cpp:180:main(): No engine version found for 91.0
[Debug] /home/adam/Dev/pr-downloader/src/pr-downloader.cpp:219:DownloadStart(): Nothing to do, did you forget to call DownloadAdd()?
[Info] Download complete!

where there's a couple of additional debug lines I added myself (trying to figure out what some of those objects actually are), see:

https://github.com/AppSynergy/pr-downloader/tree/swlIssue22

PS: props for still playing now this issue is closed/invalid.

[db81]

I think it's time to trace network access directly. tcpdump -A host api.springfiles.com or even strace -o trace.log -f -e trace=network -s 999999 pr-downloader --download-engine 91.0 if tcpdump doesn't give you anything.

[abma]

imo better: ngrep host api.springfiles.com

[abma]

no access from 87.254.77.9 as well in logs of api.springfiles.com. not a single valid http-request seems to reach the server.

[abma]

very likely the received data is just empty

[AppSynergy]

trace.log below.

Running ngrep host api.springfiles.com whilst this happens, not a bean.

Sanity check: ngrep -d wlan0 port 80 sees all the things.

Suspicious looking bit (imho):

sin_port=htons(53), sin_addr=inet_addr("127.0.1.1")

Is it trying to resolve DNS using local? Wtf?

I've learnt more during this issue that I ever did at college.

10104 socket(PF_LOCAL, SOCK_STREAM, 0)  = 4
10104 connect(4, {sa_family=AF_LOCAL, sun_path="/run/user/1000/keyring-fPwxmD/pkcs11"}, 110) = 0
10104 socket(PF_INET6, SOCK_DGRAM, IPPROTO_IP) = 5
10106 socket(PF_LOCAL, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 5
10106 connect(5, {sa_family=AF_LOCAL, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
10106 socket(PF_LOCAL, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 5
10106 connect(5, {sa_family=AF_LOCAL, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
10106 socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 5
10106 connect(5, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("127.0.1.1")}, 16) = 0
10106 sendmmsg(5, {{{msg_name(0)=NULL, msg_iov(1)=[{"\214\34\1\0\0\1\0\0\0\0\0\0\3api\vspringfiles\3com\0\0\1\0\1", 37}], msg_controllen=0, msg_flags=0}, 37}, {{msg_name(0)=NULL, msg_iov(1)=[{"+a\1\0\0\1\0\0\0\0\0\0\3api\vspringfiles\3com\0\0\34\0\1", 37}], msg_controllen=0, msg_flags=0}, 37}}, 2, MSG_NOSIGNAL) = 2
10106 recvfrom(5, "\214\34\201\200\0\1\0\1\0\2\0\3\3api\vspringfiles\3com\0\0\1\0\1\300\f\0\1\0\1\0\0\6\232\0\4%\273\\\256\300\20\0\2\0\1\0\2\233\212\0\26\3ns1\rhelderhosting\2nl\0\300\20\0\2\0\1\0\2\233\212\0\6\3ns2\300E\300A\0\1\0\1\0\0\6\232\0\4mj\240\16\300c\0\1\0\1\0\0\6\232\0\4\262?\0365\300c\0\34\0\1\0\0\6\232\0\20*\1\4\370\1\0204)\0\0\0\0\0\0\0\2", 2048, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("127.0.1.1")}, [16]) = 165
10106 sendto(5, "\214\34\1\0\0\1\0\0\0\0\0\0\3api\vspringfiles\3com\0\0\1\0\1", 37, MSG_NOSIGNAL, NULL, 0) = 37
10106 recvfrom(5, "\214\34\201\200\0\1\0\1\0\2\0\3\3api\vspringfiles\3com\0\0\1\0\1\300\f\0\1\0\1\0\0\6\225\0\4%\273\\\256\300\20\0\2\0\1\0\2\233\205\0\26\3ns1\rhelderhosting\2nl\0\300\20\0\2\0\1\0\2\233\205\0\6\3ns2\300E\300A\0\1\0\1\0\0\6\225\0\4mj\240\16\300c\0\1\0\1\0\0\6\225\0\4\262?\0365\300c\0\34\0\1\0\0\6\225\0\20*\1\4\370\1\0204)\0\0\0\0\0\0\0\2", 2048, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("127.0.1.1")}, [16]) = 165
10106 sendto(5, "+a\1\0\0\1\0\0\0\0\0\0\3api\vspringfiles\3com\0\0\34\0\1", 37, MSG_NOSIGNAL, NULL, 0) = 37
10106 socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 5
10106 connect(5, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("127.0.1.1")}, 16) = 0
10106 sendto(5, "\214\34\1\0\0\1\0\0\0\0\0\0\3api\vspringfiles\3com\0\0\1\0\1", 37, MSG_NOSIGNAL, NULL, 0) = 37
10106 recvfrom(5, "\214\34\201\200\0\1\0\1\0\2\0\3\3api\vspringfiles\3com\0\0\1\0\1\300\f\0\1\0\1\0\0\6\220\0\4%\273\\\256\300\20\0\2\0\1\0\2\233\200\0\26\3ns1\rhelderhosting\2nl\0\300\20\0\2\0\1\0\2\233\200\0\6\3ns2\300E\300A\0\1\0\1\0\0\6\220\0\4mj\240\16\300c\0\1\0\1\0\0\6\220\0\4\262?\0365\300c\0\34\0\1\0\0\6\220\0\20*\1\4\370\1\0204)\0\0\0\0\0\0\0\2", 2048, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("127.0.1.1")}, [16]) = 165
10106 socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 5
10106 connect(5, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("127.0.1.1")}, 16) = 0
10106 sendto(5, "+a\1\0\0\1\0\0\0\0\0\0\3api\vspringfiles\3com\0\0\34\0\1", 37, MSG_NOSIGNAL, NULL, 0) = 37
10106 +++ exited with 0 +++
10104 +++ exited with 1 +++

[abma]

can you change /etc/hosts 127.0.1.1 MotherOfInvention to 127.0.0.1 MotherOfInvention ?

for me this looks like dns is failing at your computer.

or maybe just remove this line (shouldn't be needed at all)

you could also stop nscd, maybe this program fails. (not sure if it is used/installed, but it looks like your computer uses 127.0.1.1 as dns server)

[AppSynergy]

That line is likely a typo, was probably meant to be 127.0.0.1 the whole time, but makes no difference to pr-downloader whatsoever.... (and why it's there? I think it's to do with Laravel automatic environment detection, another story...)

Anyway, this might be a complete coincidence.

Why would my computer think that 127.0.1.1 is a DNS server?

"..in Ubuntu 12.04 Desktop there is, by default, a local nameserver running, namely, a dnsmasq process controlled by NetworkManager which listens on 127.0.0.1. In Ubuntu 12.10 the listen address has been changed to 127.0.1.1."

http://askubuntu.com/questions/201603/how-do-i-edit-my-resolv-conf-file-to-fix-wrong-dns-problem

_(clusterf_ks of disagreement abound on that thread..)

Might I proffer: pr-downloader and dnsmasq on Ubuntu just don't get on?

[abma]

no, dnsmasq isn't needed.

whats the output of cat /etc/resolv.conf ?

also you could try tcpdump -vvv -s 0 -l -n port 53 -i any

[AppSynergy]

/etc/resolv.conf, as expected:

# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 127.0.1.1

[abma]

no, thats not as excepted. this shouldn't be 127.0.1.1.

which process listens at port 53?

netstat -tanpu |grep :53 (i guess dnsmasq)

whats the output of "dig @127.0.1.1 api.springfiles.com" return?

[AppSynergy]

TCP listens.

tcp        0      0 127.0.1.1:53            0.0.0.0:*               LISTEN 

Also a bunch of udps, but not LISTEN.

[abma]

thats just 1/3 of my questions :-)

[AppSynergy]

This is becoming an epic expedition, I'll try and keep up

$ dig @127.0.1.1 api.springfiles.com

; <<>> DiG 9.9.5-3-Ubuntu <<>> @127.0.1.1 api.springfiles.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 19500
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;api.springfiles.com.       IN  A

;; ANSWER SECTION:
api.springfiles.com.    1076    IN  A   37.187.92.174

;; AUTHORITY SECTION:
springfiles.com.    154830  IN  NS  ns1.helderhosting.nl.
springfiles.com.    154830  IN  NS  ns2.helderhosting.nl.

;; Query time: 135 msec
;; SERVER: 127.0.1.1#53(127.0.1.1)
;; WHEN: Tue Nov 11 19:58:25 GMT 2014
;; MSG SIZE  rcvd: 116

[AppSynergy]

In case this is somehow relevant, I just tried reinstalling resolvconf. I note this has changed my /etc/resolv.conf file into the symlink it (apparently) always should have been...

https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes#Networking

Nothing else appears different (pr-downloader success, results of above incantations, etc..)

[abma]

can you please update pr-downloader please?

added some verbose info when it fails.

https://github.com/spring/pr-downloader

[AppSynergy]

I hit the log on 77, but neither of the other two.

The err? Drum roll...

[Info] XmlRpcCurlClient::execute: curl error: Timeout was reached

[AppSynergy]

Maybe this?

"a SIGALRM is raised during name resolution which libcurl thinks is the timeout alarm."

http://ravidhavlesha.wordpress.com/2012/01/08/curl-timeout-problem-and-solution/

I'm going to get dinner now.

[abma]

can you provide the full log please?

i'm not sure if its dns timeout or http connect timeout. very likely its dns but this makes no sense at all :-|

how long does it take until the error happens?

output of "time pr-downloader --download-engine 98.0" ?!

[AppSynergy]

No worries, though nothing else has changed...

$ strace -o trace.log -f -e trace=network -s 999999 pr-downloader --filesystem-writepath /home/adam/.spring/weblobby --download-engine 91.0
pr-downloader 0.7-162-g4f679a1
[Info] Using filesystem-writepath: /home/adam/.spring/weblobby
[Error] /home/adam/Dev/pr-downloader/src/pr-downloader.cpp:83:search(): engine dl
[Debug] /home/adam/Dev/pr-downloader/src/Downloader/Http/HttpDownloader.cpp:65:search(): 91.0
[Info] XmlRpcCurlClient new client: host api.springfiles.com, port 80.
[Info] XmlRpcCurlClient::execute: method springfiles.search.
[Info] XmlRpcCurlClient::execute: curl error: Timeout was reached
[Error] /home/adam/Dev/pr-downloader/src/Downloader/Http/HttpDownloader.cpp:101:search(): Returned xml isn't an array!
[Debug] /home/adam/Dev/pr-downloader/src/main.cpp:92:download(): Couldn't find 91.0
[Error] /home/adam/Dev/pr-downloader/src/main.cpp:180:main(): No engine version found for 91.0
[Debug] /home/adam/Dev/pr-downloader/src/pr-downloader.cpp:219:DownloadStart(): Nothing to do, did you forget to call DownloadAdd()?
[Info] Download complete!

$ cat trace.log 
6783  socket(PF_LOCAL, SOCK_STREAM, 0)  = 4
6783  connect(4, {sa_family=AF_LOCAL, sun_path="/run/user/1000/keyring-67haZA/pkcs11"}, 110) = 0
6783  socket(PF_INET6, SOCK_DGRAM, IPPROTO_IP) = 5
6785  socket(PF_LOCAL, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 5
6785  connect(5, {sa_family=AF_LOCAL, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
6785  socket(PF_LOCAL, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 5
6785  connect(5, {sa_family=AF_LOCAL, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
6785  socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 5
6785  connect(5, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("127.0.1.1")}, 16) = 0
6785  sendmmsg(5, {{{msg_name(0)=NULL, msg_iov(1)=[{"49\1\0\0\1\0\0\0\0\0\0\3api\vspringfiles\3com\0\0\1\0\1", 37}], msg_controllen=0, msg_flags=0}, 37}, {{msg_name(0)=NULL, msg_iov(1)=[{"\374\212\1\0\0\1\0\0\0\0\0\0\3api\vspringfiles\3com\0\0\34\0\1", 37}], msg_controllen=0, msg_flags=0}, 37}}, 2, MSG_NOSIGNAL) = 2
6785  recvfrom(5, "49\201\200\0\1\0\1\0\2\0\3\3api\vspringfiles\3com\0\0\1\0\1\300\f\0\1\0\1\0\0\6\213\0\4%\273\\\256\300\20\0\2\0\1\0\2N\221\0\26\3ns1\rhelderhosting\2nl\0\300\20\0\2\0\1\0\2N\221\0\6\3ns2\300E\300A\0\1\0\1\0\0\0\324\0\4mj\240\16\300c\0\1\0\1\0\0\0\324\0\4\262?\0365\300c\0\34\0\1\0\0\0\324\0\20*\1\4\370\1\0204)\0\0\0\0\0\0\0\2", 2048, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("127.0.1.1")}, [16]) = 165
6785  sendto(5, "49\1\0\0\1\0\0\0\0\0\0\3api\vspringfiles\3com\0\0\1\0\1", 37, MSG_NOSIGNAL, NULL, 0) = 37
6785  recvfrom(5, "49\201\200\0\1\0\1\0\2\0\3\3api\vspringfiles\3com\0\0\1\0\1\300\f\0\1\0\1\0\0\6\206\0\4%\273\\\256\300\20\0\2\0\1\0\2N\214\0\26\3ns1\rhelderhosting\2nl\0\300\20\0\2\0\1\0\2N\214\0\6\3ns2\300E\300A\0\1\0\1\0\0\0\317\0\4mj\240\16\300c\0\1\0\1\0\0\0\317\0\4\262?\0365\300c\0\34\0\1\0\0\0\317\0\20*\1\4\370\1\0204)\0\0\0\0\0\0\0\2", 2048, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("127.0.1.1")}, [16]) = 165
6785  sendto(5, "\374\212\1\0\0\1\0\0\0\0\0\0\3api\vspringfiles\3com\0\0\34\0\1", 37, MSG_NOSIGNAL, NULL, 0) = 37
6785  socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 5
6785  connect(5, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("127.0.1.1")}, 16) = 0
6785  sendto(5, "49\1\0\0\1\0\0\0\0\0\0\3api\vspringfiles\3com\0\0\1\0\1", 37, MSG_NOSIGNAL, NULL, 0) = 37
6785  recvfrom(5, "49\201\200\0\1\0\1\0\2\0\3\3api\vspringfiles\3com\0\0\1\0\1\300\f\0\1\0\1\0\0\6\201\0\4%\273\\\256\300\20\0\2\0\1\0\2N\207\0\26\3ns2\rhelderhosting\2nl\0\300\20\0\2\0\1\0\2N\207\0\6\3ns1\300E\300c\0\1\0\1\0\0\0\312\0\4mj\240\16\300A\0\1\0\1\0\0\0\312\0\4\262?\0365\300A\0\34\0\1\0\0\0\312\0\20*\1\4\370\1\0204)\0\0\0\0\0\0\0\2", 2048, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("127.0.1.1")}, [16]) = 165
6785  socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 5
6785  connect(5, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("127.0.1.1")}, 16) = 0
6785  sendto(5, "\374\212\1\0\0\1\0\0\0\0\0\0\3api\vspringfiles\3com\0\0\34\0\1", 37, MSG_NOSIGNAL, NULL, 0) = 37
6785  +++ exited with 0 +++
6783  +++ exited with 1 +++

I'm afk for the night now. Good luck!

[abma]

ok, thx.

1. how long does it take until the error happens?
2. output of "time pr-downloader --download-engine 98.0" ?!

[AppSynergy]

real ~ 15s

which mainly takes place between these two output lines:

[Info] XmlRpcCurlClient new client: host api.springfiles.com, port 80.
[Info] XmlRpcCurlClient::execute: method springfiles.search.

(you know a guy's serious when he starts using interrobangs to ask questions ;)

[abma]

why does this take 15 seconds? are you using a 56k modem? :-| (this should even be faster with a 56k modem...)

[abma]

does opening api.springfiles.com taking so long, too for you?

[AppSynergy]

You know, my first thought was... no, of course not! It's much quicker than that!

But lo and behold:

$ time wget api.springfiles.com
--2014-11-12 21:43:39--  http://api.springfiles.com/
Resolving api.springfiles.com (api.springfiles.com)... 37.187.92.174
Connecting to api.springfiles.com (api.springfiles.com)|37.187.92.174|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘index.html’

    [ <=>                                                             ] 10,321      --.-K/s   in 0.1s    

2014-11-12 21:43:55 (79.7 KB/s) - ‘index.html’ saved [10321]

real    0m15.489s
user    0m0.003s
sys 0m0.005s

Really doesn't feel like that long...

Once was the day when my proud country (and it's national ISP) were famed for their cutting edged-ness.... these are not those days....

[abma]

ouch!

just to compare, my time:

wget api.springfiles.com --2014-11-12 22:58:58-- http://api.springfiles.com/ Resolving api.springfiles.com (api.springfiles.com)... 37.187.92.174 Connecting to api.springfiles.com (api.springfiles.com)|37.187.92.174|:80... connected. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/html] Saving to: 'index.html.4'

[ <=>                                                                                                                                      ] 10,321      --.-K/s   in 0.04s   

2014-11-12 22:58:58 (271 KB/s) - 'index.html.4' saved [10321]

real 0m0.147s user 0m0.000s sys 0m0.008s

ok, now i've an idea whats wrong, lets see :)

[abma]

@AppSynergy

can you try which value here fixes it for you?

https://github.com/spring/pr-downloader/blob/master/src/Downloader/CurlWrapper.cpp#L12

i guess CURLOPT_CONNECTTIMEOUT, 20 should work for you?!

[AppSynergy]

Ah yes... 10 < 15 < 20 is a pretty good theory. Will report back

[AppSynergy]

And we have a winner! Successful download confirmed, with:

curl_easy_setopt(ret, CURLOPT_CONNECTTIMEOUT, 20);

I guess the verdict is that 10 seconds was a little optimistic? (and I'd be tempted to up that value even more, I can't be on the shittiest connection in the entire world, can I?)

Glad to have helped with "a pr-downloader bug that has plagued us for a while and I'm not sure how to reproduce it". It turns out "how to reproduce it" is "to come live in my country".

Ice cream for everyone.

[AppSynergy]

PS: *&%k you, my faceless and evil ISP...

[abma]

ok, thanks a lot for your help, changed it in pr-downloader!

https://github.com/spring/pr-downloader/commit/030cf627f7c361d2105f06047b360e92952a3ef4

[db81]

Sweet! Thanks to both of you for sticking with it until the end.