search

pirxthepilot / wtfis

Passive hostname, domain and IP lookup tool for non-robots
MIT License
889 stars 47 forks source link

Feature Request: Supporting IPinfo.io (Geolocation, ASN) and Host.io (For domain information) #65

Open abdullahdevrel opened 1 year ago

abdullahdevrel commented 1 year ago

Requesting integration of:

Both these services are accessible to users easily and comes with full accuracy and generous free tier.

IPinfo.io

API request example:

curl https://ipinfo.io/123.226.51.149?token=$token
{
  "ip": "123.226.51.149",
  "hostname": "p1259149-ipngn2001niigatani.niigata.ocn.ne.jp",
  "city": "Niigata",
  "region": "Niigata",
  "country": "JP",
  "loc": "37.8864,139.0059",
  "org": "AS4713 NTT Communications Corporation",
  "postal": "950-0954",
  "timezone": "Asia/Tokyo"
}

ASN information is available on the org field.

Host.io

curl https://host.io/api/full/github.com?token=$token
{
  "domain": "github.com",
  "web": {
    "domain": "github.com",
    "rank": 30,
    "url": "https://github.com/",
    "ip": "192.30.255.113",
    "date": "2023-08-28T06:18:32.912Z",
    "length": 235688,
    "server": "GitHub.com",
    "encoding": "utf8",
    "copyright": "© 2023 GitHub, Inc.",
    "facebook": "GitHub",
    "twitter": "github",
    "title": "GitHub: Letâs build from here · GitHub",
    "description": "GitHub is where over 100 million developers shape the future of software, together. Contribute to the open source community, manage your Git repositories, review code like a pro, track bugs and features, power your CI/CD and DevOps workflows, and secure code before you commit it.",
    "links": [
      "github.blog",
      "electronjs.org",
      "github.community",
      "githubstatus.com",
      "twitter.com",
      "facebook.com",
      "linkedin.com",
      "youtube.com",
      "twitch.tv",
      "tiktok.com"
    ]
  },
  "dns": {
    "domain": "github.com",
    "a": [
      "192.30.255.112"
    ],
    "mx": [
      "1 aspmx.l.google.com.",
      "10 alt3.aspmx.l.google.com.",
      "10 alt4.aspmx.l.google.com.",
      "5 alt1.aspmx.l.google.com.",
      "5 alt2.aspmx.l.google.com."
    ],
    "ns": [
      "dns1.p08.nsone.net.",
      "dns2.p08.nsone.net.",
      "dns3.p08.nsone.net.",
      "dns4.p08.nsone.net.",
      "ns-1283.awsdns-32.org.",
      "ns-1707.awsdns-21.co.uk.",
      "ns-421.awsdns-52.com.",
      "ns-520.awsdns-01.net."
    ]
  },
  "ipinfo": {
    "192.30.255.113": {
      "city": "Seattle",
      "region": "Washington",
      "country": "US",
      "loc": "47.6062,-122.3321",
      "postal": "98101",
      "timezone": "America/Los_Angeles",
      "asn": {
        "asn": "AS36459",
        "name": "GitHub, Inc.",
        "domain": "github.com",
        "route": "192.30.255.0/24",
        "type": "isp"
      }
    },
    "192.30.255.112": {
      "city": "Seattle",
      "region": "Washington",
      "country": "US",
      "loc": "47.6062,-122.3321",
      "postal": "98101",
      "timezone": "America/Los_Angeles",
      "asn": {
        "asn": "AS36459",
        "name": "GitHub, Inc.",
        "domain": "github.com",
        "route": "192.30.255.0/24",
        "type": "isp"
      }
    }
  },
  "related": {
    "ip": [
      {
        "value": "192.30.255.113",
        "count": 9558
      },
      {
        "value": "192.30.255.112",
        "count": 9645
      }
    ],
    "asn": [
      {
        "value": "AS36459",
        "count": 55240
      }
    ],
    "ns": [
      {
        "value": "nsone.net",
        "count": 3112162
      },
      {
        "value": "awsdns-32.org",
        "count": 48035
      },
      {
        "value": "awsdns-21.co.uk",
        "count": 53472
      },
      {
        "value": "awsdns-52.com",
        "count": 54451
      },
      {
        "value": "awsdns-01.net",
        "count": 54069
      }
    ],
    "mx": [
      {
        "value": "google.com",
        "count": 16432132
      }
    ],
    "backlinks": [
      {
        "value": "github.com",
        "count": 875699
      }
    ],
    "redirects": [
      {
        "value": "github.com",
        "count": 19280
      }
    ]
  }
}

Please let me know what you think. The project currently relies on WHOIS information for geolocation information, which does not provide a reliable picture of location of IP addresses.

pirxthepilot commented 1 year ago

Hi @abdullahdevrel , thanks for the idea! I am also a fan of ipinfo. I was not aware that ipwhois provides less accurate geolocation - out of curiosity, can you provide more details about this, and how ipinfo is different?

abdullahdevrel commented 1 year ago

thanks for the idea! I am also a fan of ipinfo.

I am genuinely happy to hear that!

can you provide more details about this, and how ipinfo is different?

Definitely. I had actually written a community post for this feature request, but I was hesitating a bit because that is a lot to mention in a single ticket. 😅

Check this post out: https://community.ipinfo.io/t/ipinfos-ping-based-geolocation-provides-far-more-reliable-location-information-compared-to-whois-records/3084

The TLDR is that WHOIS records are not a reliable data source for geolocation; consequently, providers who rely on it may incorrect IP geolocation data. We use ping-based geolocation, which is more reliable and consistent.

Please let me know how I can help; I am always happy to answer any questions you might have.

pirxthepilot commented 1 year ago

@abdullahdevrel that's so cool! Thanks for the context. Does ipinfo execute the pings every time someone queries an IP, or does it run discovery on some sort of schedule?

abdullahdevrel commented 1 year ago

@pirxthepilot The geolocation process runs daily. It is not possible to run ping geolocation process as someone queries an IP; it would be too time-consuming.

We essentially perform an internet-wide scan of all IP addresses. We have hundreds of probe servers located around the world. When we ping an IP address from one probe server, we receive one dimension of information based on the Round Trip Time (RTT). As we ping the IP address from other probe servers, the location information becomes more specific and accurate. This process is known as multilateration.

It is not possible to perform multilateration-based geolocation when someone queries an IP address. The process takes time, and as you have know, our API has a fairly fast response time. We run the geolocation process daily, store that information in a database, and have built our API on top of it. Our IP geolocation data is updated daily.

Another article: https://ipinfo.io/blog/probe-network-how-we-make-sure-our-data-is-accurate/

pirxthepilot commented 1 year ago

@abdullahdevrel ok great! Going to label this as a future enhancement. I'll need to do some house cleaning first, so it might take some time :) Thank you for bringing this up!

abdullahdevrel commented 1 year ago

@pirxthepilot Awesome. Thank you for considering IPinfo :)