search

archlinuxfr / yaourt

[unmaintained] A Pacman frontend with more features and AUR support
598 stars 98 forks source link

Curl can't resolve host if ipv6 is available #275

Open 05storm26 opened 8 years ago

05storm26 commented 8 years ago

I have dual stack ipv4-ipv6. In this case when i try to use yaourt it fails to connect to aur repositories. Pacman works fine.

This is what it looks like:

yaourt -Ss firefox-nightly curl error: Couldn't resolve host name

networkctl status enp3s0 ● 2: enp3s0 Link File: /usr/lib/systemd/network/99-default.link Network File: /etc/systemd/network/wired.network Type: ether State: routable (configured) Path: pci-0000:03:00.0 Driver: r8169 Vendor: Realtek Semiconductor Co., Ltd. Model: RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (Motherbo HW Address: 6c:f0:__:__:__:__ (GIGA-BYTE TECHNOLOGY CO.,LTD.) Address: 192.168.2.202 <external native ipv6 address here> <another external native ipv6 address here> fdd4:7815:47cd:0:241d:6166:dbb4:169c fdd4:7815:47cd:0:6ef0:49ff:feb5:85f6 fe80::993f:ea86:ce:bee0 Gateway: 192.168.1.1 (TP-LINK TECHNOLOGIES CO.,LTD.) fe80::c66e:1fff:fe87:cbde (TP-LINK TECHNOLOGIES CO.,LTD.) DNS: 192.168.1.1

If I disable ipv6 in the networkmanager and reconnect yaourt works fine.

f2404 commented 8 years ago

Are you sure your IP v6 connectivity is actually working? I.e. DNS resolving, routing...

05storm26 commented 8 years ago

Well it seems to work:

host google.com google.com has address 216.58.209.174 google.com has IPv6 address 2a00:1450:400d:806::200e google.com mail is handled by 40 alt3.aspmx.l.google.com. google.com mail is handled by 50 alt4.aspmx.l.google.com. google.com mail is handled by 10 aspmx.l.google.com. google.com mail is handled by 20 alt1.aspmx.l.google.com. google.com mail is handled by 30 alt2.aspmx.l.google.com.

Also mtr -Tb -rw -c 5 google.com is successfully reaching google and i see ipv6 addresses.

f2404 commented 8 years ago

Does ping -6 google.com also work? If it does, I will need to look into the code working with curl and investigate if any IPv6-related options could be missing.

UP In fact, we should rather be checking aur.archlinux.org instead of google.com.

f2404 commented 8 years ago

I'm getting the following on my system with no IPv6 connectivity:

$ curl -6 https://aur.archlinux.org
curl: (7) Couldn't connect to server

Can you check if IPv6 address is resolved for you?

$ host aur.archlinux.org 
aur.archlinux.org has address 5.9.250.164
aur.archlinux.org has IPv6 address 2a01:4f8:160:3033::2
aur.archlinux.org mail is handled by 10 mx.archlinux.org.
f2404 commented 8 years ago

@05storm26 ping

05storm26 commented 8 years ago

Hi, sorry for the late answer. The problem appears to be that I'am using tor. And i set this up in networkmanager (as a socks proxy). Since tor doesn't support ipv6 too well yet it is not enabled for me. Maybe curl is trying to respect these proxy settings? I don't think pacman does this. That is probably why pacman works. If I set the proxy to None in the networkmanager than (and open a new terminal window) curl and yaourt works.

My opinion on this matter is that maybe yaourt when using curl should try to use the same network settings as pacman does. If pacman doesn't respect the proxy settings that you can set in networkmanager than imho yaourt should not use these either.

05storm26 commented 8 years ago

Since tor doesn't support ipv6 too well yet it is not enabled for me.

I mean ipv6 is enabled but tor only uses ipv4.

f2404 commented 8 years ago

Can you try if yaourt/curl will work through ipv4 tor proxy? Can you configure it this way? I'm thinking maybe the issue is not in using proxy per se but in using tor as transport.

f2404 commented 8 years ago

Actually, to put pacman and yaourt in the same position, you should configure the first to use proxy: https://wiki.archlinux.org/index.php/proxy_settings

05storm26 commented 8 years ago

Yes but I don't care if they use the proxy or not. I just think that by default both should use the same settings. If one by default doesn't use the proxy set in networkmanager than the another when using curl shouldn't not use those proxy setting either.

f2404 commented 8 years ago

Well I think you should care as your proxy is not working. The thing is that curl has got configured to use proxy in your system; it's just pacman that by default is ignoring system proxy settings which I don't think is a good idea in general. Maybe we could add an "ignore system proxy settings" option to yaourt (need to check if there's such option for libcurl).

05storm26 commented 8 years ago

ipv6 disabled, proxy set to none in networkmanager: 

curl ident.me
<valid external ipv4 address here>

(everything works obviously)

ipv6 disabled, proxy set to use the local socks proxy (tor) in networkmanager:

curl ident.me
195.154.165.246

You can see that that ip address is a tor exit node: https://atlas.torproject.org/#search/195.154.165.246

Traceroute doesn't follow the proxy settings (which couldn't even be possible since traceroute uses icmp packages but tor only allows tcp connections): 

traceroute aur.archlinux.org
traceroute to aur.archlinux.org (5.9.250.164), 30 hops max, 60 byte packets
 1  OpenWrt.lan (192.168.1.1)  0.200 ms  0.300 ms  0.357 ms
 2  10.0.0.1 (10.0.0.1)  1.409 ms  1.429 ms  1.440 ms
 3  10.1.129.65 (10.1.129.65)  1.604 ms  1.459 ms  1.500 ms
 4  xe-1-0-0.cr02.budapest.digicable.hu (94.21.3.6)  8.482 ms  8.523 ms  8.536 ms
 5  cr02.budapesta.rdsnet.ro (213.154.126.65)  9.499 ms  9.359 ms  9.412 ms
 6  * * *
 7  decix2-gw.hetzner.de (80.81.193.164)  23.861 ms  24.966 ms  23.849 ms
 8  core22.hetzner.de (213.239.245.17)  31.104 ms  29.436 ms  31.119 ms
 9  juniper2.rz16.hetzner.de (213.239.245.134)  29.517 ms juniper1.rz16.hetzner.de (213.239.245.94)  54.750 ms  45.919 ms
10  hos-tr2.ex3k3.rz16.hetzner.de (213.239.222.100)  33.611 ms hos-tr5.ex3k3.rz16.hetzner.de (213.239.251.74)  33.718 ms hos-tr4.ex3k3.rz16.hetzner.de (213.239.223.228)  31.709 ms
11  luna.archlinux.org (5.9.250.164)  30.814 ms  30.209 ms  29.548 ms

Also: 

curl aur.archlinux.org
<html>
<head><title>301 Moved Permanently</title></head>
<body bgcolor="white">
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx/1.10.0</center>
</body>
</html>
nslookup aur.archlinux.org 
Server:     192.168.1.1
Address:    192.168.1.1#53

Non-authoritative answer:
Name:   aur.archlinux.org
Address: 5.9.250.164
host aur.archlinux.org
aur.archlinux.org has address 5.9.250.164
aur.archlinux.org has IPv6 address 2a01:4f8:160:3033::2
aur.archlinux.org mail is handled by 10 mx.archlinux.org.

So yes curl does work if there isn't ipv6 but there is a tor proxy configured.

IPv6 enabled no tor proxy:

curl ident.me
<valid external ipv6 address here>
 mtr -Tb -rw -6 -c 5 aur.archlinux.org
Start: Sun Jun 26 19:09:56 2016
HOST: avietnamihaboruvege                                                                                 Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- 2a01-036d-2800-4c3b-0000-0000-0000-0001.pool6.digikabel.hu (2a01:36d:2800:4c3b::                     0.0%     5    0.5   0.5   0.5   0.6   0.0
  2.|-- 2a01-036c-2800-4c3b-d0fc-1b84-c215-ebce.pool6.digikabel.hu (2a01:36c:2800:4c3b:d                     0.0%     5    8.6   9.9   8.5  14.8   2.6
  3.|-- 2a01:368:2801:1000::1                                                                                0.0%     5    1.9   1.7   1.2   2.0   0.0
  4.|-- te-3-3.bb01.szolnok.digicable6.hu (2a01:368::5e15:30d)                                               0.0%     5   11.0   8.0   7.0  11.0   1.6
  5.|-- xe-7-0-0.cr02.budapest.digicable6.hu (2a01:368::5e15:30a)                                            0.0%     5    8.7   8.7   8.6   8.8   0.0
  6.|-- cr02.v6.budapesta.rdsnet.ro (2a02:2f09:20:0:8708:20:845:1)                                           0.0%     5    8.1   8.6   8.1   9.0   0.0
  7.|-- 2a02:2f00:8708:2:2:0:6:0                                                                             0.0%     5   11.4  11.7  11.3  12.0   0.0
  8.|-- br01.v6.frankfurt.rdsnet.ro (2a02:2f00:8708:4:1:0:732:0)                                             0.0%     5   25.5  26.1  25.5  27.0   0.5
  9.|-- ae0-406.fra20.core-backbone.com (2a01:4a0:1338:64::1)                                                0.0%     5   25.9  27.3  25.5  33.5   3.4
 10.|-- 2a01:4a0:1338:3::2                                                                                   0.0%     5   28.9  28.2  27.1  29.1   0.5
 11.|-- core4.hetzner.de (2a01:4f8:0:3::d)                                                                   0.0%     5   33.9  28.7  26.5  33.9   3.0
 12.|-- core21.hetzner.de (2a01:4f8:0:3::9)                                                                  0.0%     5   35.5  32.8  31.7  35.5   1.5
 13.|-- juniper1.rz16.hetzner.de (2a01:4f8:0:3::5a)                                                          0.0%     5   32.3  32.7  30.3  36.6   2.3
 14.|-- hos-tr2.ex3k3.rz16.hetzner.de (2a01:4f8:0:16:2:a:16:3)                                               0.0%     5   35.1  34.6  32.8  35.8   1.1
 15.|-- luna.archlinux.org (2a01:4f8:160:3033::2)                                                            0.0%     5  134.5 108.1  66.7 202.4  60.0

Curl also works:

curl -6 aur.archlinux.org
<html>
<head><title>301 Moved Permanently</title></head>
<body bgcolor="white">
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx/1.10.0</center>
</body>
</html>

IPv6 enabled with tor SOCKS proxy set up in networkmanager:

curl ident.me
curl: (6) Failed to resolve "ident.me" for SOCKS4 connect.
curl -4 ident.me
109.201.133.100

109.201.133.100 is a tor exit: https://atlas.torproject.org/#search/109.201.133.100

curl -6 ident.me
curl: (6) Failed to resolve "ident.me" for SOCKS4 connect.

Resolving still works:

host aur.archlinux.org
aur.archlinux.org has address 5.9.250.164
aur.archlinux.org has IPv6 address 2a01:4f8:160:3033::2
aur.archlinux.org mail is handled by 10 mx.archlinux.org.

Traceroute still works

 mtr -Tb -rw -6 -c 5 aur.archlinux.org
Start: Sun Jun 26 19:20:24 2016
HOST: avietnamihaboruvege                                                                                 Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- 2a01-036d-2800-4c3b-0000-0000-0000-0001.pool6.digikabel.hu (2a01:36d:2800:4c3b::                     0.0%     5    0.5   0.5   0.4   0.6   0.0
  2.|-- 2a01-036c-2800-4c3b-d0fc-1b84-c215-ebce.pool6.digikabel.hu (2a01:36c:2800:4c3b:d                     0.0%     5    8.3   8.8   8.3  10.1   0.5
  3.|-- 2a01:368:2801:1000::1                                                                                0.0%     5    1.8   2.3   1.2   5.3   1.6
  4.|-- te-3-3.bb01.szolnok.digicable6.hu (2a01:368::5e15:30d)                                               0.0%     5    7.7   9.6   7.7  16.9   4.1
  5.|-- xe-7-0-0.cr02.budapest.digicable6.hu (2a01:368::5e15:30a)                                            0.0%     5   10.0   9.0   8.2  10.0   0.5
  6.|-- cr02.v6.budapesta.rdsnet.ro (2a02:2f09:20:0:8708:20:845:1)                                           0.0%     5    8.1   8.5   8.1   8.9   0.0
  7.|-- 2a02:2f00:8708:2:2:0:6:0                                                                             0.0%     5   12.1  12.1  11.2  12.8   0.0
  8.|-- br01.v6.frankfurt.rdsnet.ro (2a02:2f00:8708:4:1:0:732:0)                                             0.0%     5   25.2  26.1  25.2  28.2   1.1
  9.|-- ae0-406.fra20.core-backbone.com (2a01:4a0:1338:64::1)                                                0.0%     5   25.7  27.0  25.3  31.7   2.6
 10.|-- 2a01:4a0:1338:3::2                                                                                   0.0%     5   27.1  27.8  27.1  28.7   0.0
 11.|-- core4.hetzner.de (2a01:4f8:0:3::d)                                                                   0.0%     5   26.1  27.9  26.1  33.1   2.9
 12.|-- core21.hetzner.de (2a01:4f8:0:3::9)                                                                  0.0%     5   32.3  33.2  32.2  34.8   0.9
 13.|-- juniper2.rz16.hetzner.de (2a01:4f8:0:3::86)                                                          0.0%     5   30.7  33.1  30.7  40.4   4.1
 14.|-- 2a01:4f8:0:16:10:a:16:3                                                                              0.0%     5   34.0  33.9  31.4  37.6   2.5
 15.|-- luna.archlinux.org (2a01:4f8:160:3033::2)                                                            0.0%     5  134.3 107.9  66.7 201.6  59.8

Traceroute with ipv6 icmp doens't seen to work flawlessly (I get the same output without tor proxy; that is why I use mtr with -T flag which works by using tcp somehow instead of icmp or whatever the equivivalent is in ipv6 that traceroute is trying to use) but I think it is not really relevant for this:

traceroute -6 aur.archlinux.org
traceroute to aur.archlinux.org (2a01:4f8:160:3033::2), 30 hops max, 80 byte packets
 1  2a01-036d-2800-4c3b-0000-0000-0000-0001.pool6.digikabel.hu (2a01:36d:2800:4c3b::1)  0.458 ms  0.695 ms  0.680 ms
 2  2a01-036c-2800-4c3b-d0fc-1b84-c215-ebce.pool6.digikabel.hu (2a01:36c:2800:4c3b:d0fc:1b84:c215:ebce)  8.535 ms  21.947 ms  15.236 ms
 3  2a01:368:2801:1000::1 (2a01:368:2801:1000::1)  22.166 ms  22.992 ms  22.309 ms
 4  te-3-3.bb01.szolnok.digicable6.hu (2a01:368::5e15:30d)  27.681 ms  21.109 ms  27.584 ms
 5  xe-7-0-0.cr02.budapest.digicable6.hu (2a01:368::5e15:30a)  15.770 ms  28.830 ms  15.697 ms
 6  cr02.v6.budapesta.rdsnet.ro (2a02:2f09:20:0:8708:20:845:1)  28.759 ms  27.253 ms  27.139 ms
 7  2a02:2f00:8708:2:2:0:6:0 (2a02:2f00:8708:2:2:0:6:0)  30.452 ms  24.780 ms  17.202 ms
 8  br01.v6.frankfurt.rdsnet.ro (2a02:2f00:8708:4:1:0:732:0)  30.674 ms  32.849 ms  25.339 ms
 9  ae0-406.fra20.core-backbone.com (2a01:4a0:1338:64::1)  27.244 ms  25.331 ms  25.552 ms
10  2a01:4a0:1338:3::2 (2a01:4a0:1338:3::2)  27.317 ms  28.913 ms  27.970 ms
11  core1.hetzner.de (2a01:4f8:0:3::1)  26.102 ms  26.063 ms  25.985 ms
12  core21.hetzner.de (2a01:4f8:0:3::9)  32.180 ms core22.hetzner.de (2a01:4f8:0:3::b2)  32.290 ms core21.hetzner.de (2a01:4f8:0:3::d9)  32.624 ms
13  juniper2.rz16.hetzner.de (2a01:4f8:0:3::86)  30.681 ms juniper1.rz16.hetzner.de (2a01:4f8:0:3::5a)  30.502 ms juniper2.rz16.hetzner.de (2a01:4f8:0:3::86)  30.485 ms
14  hos-tr4.ex3k3.rz16.hetzner.de (2a01:4f8:0:16:4:a:16:3)  32.573 ms hos-tr2.ex3k3.rz16.hetzner.de (2a01:4f8:0:16:2:a:16:3)  33.459 ms hos-tr1.ex3k3.rz16.hetzner.de (2a01:4f8:0:16:1:a:16:3)  31.731 ms
15  * * *
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * * *
23  * * *
24  * * *
25  * * *
26  * * *
27  * * *
28  * * *
29  * * *
30  * * *
 traceroute -4 aur.archlinux.org
traceroute to aur.archlinux.org (5.9.250.164), 30 hops max, 60 byte packets
 1  OpenWrt.lan (192.168.1.1)  0.177 ms  0.267 ms  0.323 ms
 2  10.0.0.1 (10.0.0.1)  1.538 ms  1.466 ms  1.522 ms
 3  10.1.129.65 (10.1.129.65)  2.417 ms  2.392 ms  2.399 ms
 4  xe-1-0-0.cr02.budapest.digicable.hu (94.21.3.6)  9.840 ms  9.882 ms  9.825 ms
 5  cr02.budapesta.rdsnet.ro (213.154.126.65)  8.927 ms  8.851 ms  9.054 ms
 6  * * *
 7  decix2-gw.hetzner.de (80.81.193.164)  24.835 ms  25.164 ms  25.073 ms
 8  core21.hetzner.de (213.239.245.13)  31.962 ms core22.hetzner.de (213.239.245.17)  29.825 ms  32.391 ms
 9  juniper2.rz16.hetzner.de (213.239.245.134)  28.468 ms juniper1.rz16.hetzner.de (213.239.245.94)  80.402 ms  80.430 ms
10  hos-tr1.ex3k3.rz16.hetzner.de (213.239.222.68)  34.638 ms hos-tr2.ex3k3.rz16.hetzner.de (213.239.222.100)  35.597 ms  34.635 ms
11  luna.archlinux.org (5.9.250.164)  29.902 ms  30.775 ms  31.062 ms

Curl does the same for aur.archlinux.org as for ident.me:

curl -6 aur.archlinux.org
curl: (6) Failed to resolve "aur.archlinux.org" for SOCKS4 connect.

So to summarize I think that the problem is that if a proxy set curl tries to use that with ipv6 if ipv6 is available event if the proxy doesn't support ipv6

pacman works bc I think it doesn't respect proxy settings in networkmanager:

pacman -Syyu
[sudo] password for adam: 
:: Synchronising package databases...
 core                                  119,4 KiB   796K/s 00:00 [##################################] 100%
 extra                                1744,6 KiB   743K/s 00:02 [##################################] 100%
 community                               3,5 MiB  1158K/s 00:03 [##################################] 100%
 multilib                              161,1 KiB  1732K/s 00:00 [##################################] 100%
 infinality-bundle-fonts                96,5 KiB   567K/s 00:00 [##################################] 100%
 infinality-bundle-fonts.sig           287,0   B  0,00B/s 00:00 [##################################] 100%
 infinality-bundle                       4,3 KiB  0,00B/s 00:00 [##################################] 100%
 infinality-bundle.sig                 287,0   B  0,00B/s 00:00 [##################################] 100%
 infinality-bundle-multilib              2,0 KiB  0,00B/s 00:00 [##################################] 100%
 infinality-bundle-multilib.sig        287,0   B  0,00B/s 00:00 [##################################] 100%
 pipelight                             150,5 KiB  1287K/s 00:00 [##################################] 100%
 pipelight.sig                         543,0   B  0,00B/s 00:00 [##################################] 100%
:: Starting full system upgrade...
resolving dependencies...
looking for conflicting packages...

Packages (6) afl-2.16b-1  fuse-2.9.7-1  libinput-1.3.3-1  python2-gmpy2-2.0.8-1  v4l-utils-1.10.1-1
             youtube-dl-2016.06.25-1

Total Download Size:    2,90 MiB
Total Installed Size:  15,23 MiB
Net Upgrade Size:       0,07 MiB

:: Proceed with installation? [Y/n] y
:: Retrieving packages...
 fuse-2.9.7-1-x86_64                   109,6 KiB  1423K/s 00:00 [##################################] 100%
 libinput-1.3.3-1-x86_64               101,2 KiB  1386K/s 00:00 [##################################] 100%
 v4l-utils-1.10.1-1-x86_64             669,5 KiB  2,84M/s 00:00 [##################################] 100%
 afl-2.16b-1-x86_64                    440,6 KiB  4,78M/s 00:00 [##################################] 100%
 python2-gmpy2-2.0.8-1-x86_64          144,5 KiB  14,1M/s 00:00 [##################################] 100%
 youtube-dl-2016.06.25-1-any          1505,3 KiB  8,17M/s 00:00 [##################################] 100%
(6/6) checking keys in keyring                                  [##################################] 100%
(6/6) checking package integrity                                [##################################] 100%
(6/6) loading package files                                     [##################################] 100%
(6/6) checking for file conflicts                               [##################################] 100%
(6/6) checking available disk space                             [##################################] 100%
:: Processing package changes...
(1/6) upgrading afl                                             [##################################] 100%
(2/6) upgrading fuse                                            [##################################] 100%
(3/6) upgrading libinput                                        [##################################] 100%
(4/6) upgrading python2-gmpy2                                   [##################################] 100%
(5/6) upgrading v4l-utils                                       [##################################] 100%
(6/6) upgrading youtube-dl                                      [##################################] 100%
:: Running post-transaction hooks...
(1/4) Updating icon theme caches...
(2/4) Updating manpage index...
(3/4) Updating udev Hardware Database...
(4/4) Updating the desktop file MIME type cache...
05storm26 commented 8 years ago

Maybe we could add an "ignore system proxy settings" option to yaourt (need to check if there's such <option for libcurl).

That sounds good, altough i think that if pacman ignores these proxy settings yaourt by default should do the same. Either way if there is going to be a way to make yaourt call curl in a way to ignore the proxy settings in the same way as pacman does, I am gonna be happy.

f2404 commented 8 years ago

On the second thought - there's already a way to disable proxy settings for specific app: https_proxy= yaourt <command> Would it be more convenient to call yaourt --no-proxy <command> instead? I'm not sure.

f2404 commented 8 years ago

@larchunix Do you think we need such option in package-query/yaourt, or using the environment variables should be fine?

larchunix commented 8 years ago

yaourt has already so many options, I prefer to use the environment variable.

@Skunnyk what about you ?