search

vitobotta / hetzner-k3s

The easiest and fastest way to create and manage Kubernetes clusters in Hetzner Cloud using the lightweight distribution k3s by Rancher.
MIT License
1.8k stars 134 forks source link

Possibly a new problem with ip address validation #451

Closed axgkl closed 2 weeks ago

axgkl commented 2 weeks ago

Hi,

the ip addy detection was changed away from akamai to ipinfo.io but that one seems to also have problems for certain hosts - while akamai works for such hosts :/

Note: I'm creating the cluster from a proxy host inside hetzner, which itself is created before by a github action, which sets up the whole cluster from scratch. Just got the second failure for a specific host, within the last days:

the ip assigned by hetzner for those two runs was the same, 37.27.192.245, while others do work. Logging into the host reveals that akamai works, while ipinfo rejects it.

```bash root@citest-proxy:~# curl root@citest-proxy:~# curl "http://whatismyip.akamai.com"" > ^C root@citest-proxy:~# curl "http://whatismyip.akamai.com" 37.27.192.245root@citest-proxy:~# curl "http://whatismyip.akamai.com" 37.27.192.245root@citest-proxy:~# curl https://ipinfo.io/ip" 403 Forbidden

Error: Forbidden

Your client does not have permission to get URL /ip from this server.

root@citest-proxy:~# curl ifconfig.me 2a01:4f9:c012:6c6::1 # curl icanhazip.com 2a01:4f9:c012:6c6::1 root@citest-proxy:~# curl ipinfo.io/ip 403 Forbidden

Error: Forbidden

Your client does not have permission to get URL /ip from this server.

root@citest-proxy:~# curl ipecho.net/plain 2a01:4f9:c012:6c6::1 ```

I wanted to report early, so that you are aware of it - by nature of the problem, it's practically not testable. I'll will get back to it, if it happens again, for other IPs.

Lastly, I think we could skip that pub ip validation all togehter, when allowed networks is set to 0.0.0.0/0, like in my case, with a pub ip anyway only on that jump host(?)

vitobotta commented 2 weeks ago

Hi, I guess the problem might be that ipinfo rejects requests made form hosts that it detects as servers or something like that. If you can find another service that works in your context and also in China (the reason why I changed it from Akamai) I can change it and make a release.

axgkl commented 2 weeks ago

up to now ipinfo works with any server i created in that location (helsinki) - except that one. so in general i would leave it. just wanted to report that they seem to do maintain a blacklist or sth like that and maybe others run into that problem as well, so u r informed.

in general i'd say that we could skip that check alltogether if

  1. allowed networks is 0.0.0.0/0 AND
  2. you don't need that ip for other purposes later in the install

or at least not hard exit the install, when the lookup fails in such cases (?)

closing for now and would re-open if it happens again for other ips. as we say in here: "einmal is keinmal" (only once is like nothing at all or so) ;-)

abdullahdevrel commented 2 weeks ago

I work for IPinfo. If you can let me know which IP addresses you can't access our service from, we will look into it.

Reach out to me:

vitobotta commented 2 weeks ago

I work for IPinfo. If you can let me know which IP addresses you can't access our service from, we will look into it.

Reach out to me:

Thanks for offering help!

axgkl commented 1 week ago

Hi Abdullah,

I work for IPinfo. If you can let me know which IP addresses you can't access our service from, we will look into it.

Reach out to me:

do you mind discussing it here? note, the ip is not mine, but from within a hetzner owned pool and they allocated it to one server i created there, meanwhile destroyed again.

Here are the infos, for this one akamai lookup worked, ipinfo rejected:

root@citest-proxy:~# curl "http://whatismyip.akamai.com"
37.27.192.245

root@citest-proxy:~# curl https://ipinfo.io/ip

<html><head>
<meta http-equiv="content-type" content="text/html;charset=utf-8">
<title>403 Forbidden</title>
</head>
<body text=#000000 bgcolor=#ffffff>
<h1>Error: Forbidden</h1>
<h2>Your client does not have permission to get URL <code>/ip</code> from this server.</h2>
<h2></h2>
</body></html>
root@citest-proxy:~# curl ifconfig.me
2a01:4f9:c012:6c6::1

in general your service works very well, this is the only ip i ever had that problem with...

Cheers, gunther

abdullahdevrel commented 1 week ago

@axgkl That is weird. This IP address should have access to our service. It could be a bug. I have opened an internal ticket. I will report back once I have more information.

abdullahdevrel commented 1 week ago

Hi @axgkl I think we have a clue on what is going on.

GCP mislocates some Hetzner IP ranges to Iran. Now, we as an IP geolocation service correctly can locate IP addresses of Hetzner. However, our service infrastructure is based on GCP.

So, GCP itself is blocking these ranges as they think they are located in Iran.

Context: https://community.ipinfo.io/docs?topic=303

There is a discussion going on HN on a similar issue with Cloudflare mislocating Hetzner IPs in Iran as well: https://news.ycombinator.com/item?id=41585249

axgkl commented 1 week ago

crazy. i'm a boomer and i once, long ago, thought the internet would bring people together. and now politics like this, built right into major infra...

anyways, thanks for the find, we won't be able to change such things, unfortunately.

abdullahdevrel commented 1 week ago

@axgkl I know what you mean exactly. I am a boomer in spirit as well ;)

Thank you for having a conversation with me. If you have any questions about IP data, you now know me. Reach out to me if you have any issues. I will be happy to investigate the issue. You can find my contact information on my Github profile.