Update the whois servers with the IANA list

io-developer / php-whois

PHP WHOIS provides parsed and raw whois lookup of domains and ASN routes. PHP 8.0 compatible (5.4+ old versions)

MIT License

440 stars 118 forks source link

Update the whois servers with the IANA list #89

Open bessone opened 4 years ago

bessone commented 4 years ago

Hello, I made a comparison between the list of official whois servers indicated by IANA and those listed in the JSON, there are more than 300 differences.

Some servers have just changed and the old ones no longer exist, others often correspond to the same server (same IP address) but have changed names (for example with hosts with the same extension as tld). Locally I'm already updating the json file, can it make sense to report the changes on the master?

A single PR or separate in different blocks to make them more readable?

io-developer commented 4 years ago

Hello.

Wow, you've done a good job! I guess it should be one PR with two commits:

Just updating IANA servers
Remove/replace junk

What you think?

bessone commented 4 years ago

Sorry for my late reply. I started thinking about doing various PRs, but in fact it would be useless. I create a new branch with the various modifications, grouped into some commits to make them more readable. I am not going to explain everything, you will understand it at the first PR.

Is it good for you?

io-developer commented 4 years ago

Don't worry I late too :)

Yes. Single PR with strict commits it's OK. One action - one commit.

For example:

removing junk/unnecessary servers - first commit
updating some whois hosts to actual - second commit
something else - third and so on..

I guess you're not needing to do unit tests. Because it's very long and hard work

bessone commented 4 years ago

Hei @io-developer i'm still working on this, just i have not so much time to devote to it.

I just saw your new branch, do you want to download the list of tlds from IANA and use the *.whois-servers.net aliases? Will manual configurations also be possible? Because in my past tests not all tlds were sometimes active as aliases on whois-servers.net

io-developer commented 4 years ago

Hi! Actually that idea from Sep 2018 as I remember. And still have no clear vision how to release it.

Some concept:

Project (this lib) have lightweight server list - just very specific tld/servers/rules
Base whois list loading from IANA (path in config I guess to get possibility to override at project level)
So, rules building from project list + IANA => all are happy :)

Problems:

IANA have some base list (url with plain text), but it's not complete
Also IANA (maybe not iana) provides special whois server which accepts zones instead of domain. But impossible to load all known TLD via one request
So solution may be to refactoring TldServer::match* to work with single TLD instead of full list merging/sorting. It breaks backward compatibility and new bugs can appear

I guess so... What you think?

io-developer commented 4 years ago

I just saw your new branch, do you want to download the list of tlds from IANA

It's related to this issue https://github.com/io-developer/php-whois/issues/37

Will manual configurations also be possible?

Yes, of course :)

bessone commented 4 years ago

Ok, understood. Having a dynamic servers list it's a nice idea (and if I say so where almost all of my commits are to update the servers... :D ) but sometimes Iana's data is incorrect, some whois servers exist but are not documented, others are documented but do not work or are removed, having them automatic I don't know if in the end it can become more of a problem than a solution. Your static list + dinyamic fallback is a good way, but maybe the extra work isn't worth it.

And in the meantime I'm flooding you with PR, in these days of coronavirus there is a bit of calm and I'm catching up on backlog for this project, if I can in the next days / weeks I will also resume this issue.

meetjey commented 4 years ago

Hi all !

Thanks for your work 👍

Is there any luck to update this issue ?

Is there a way to help ?

Thx

bessone commented 4 years ago

Hello, most of the work is already done on my local copy, but I stopped because there is no more news from the maintainer, I have some PR waiting for months and I have others to send that now I have stopped in my fork :(

bessone commented 3 years ago

Hey @io-developer, after i while i'm back! I took advantage of this week of calm with clients in the summer holidays to resume this job that I had abandoned.

I did a big JSON update of the whois servers, separating various commits. There is only a few commits too many because I was lagging behind your master, and I had to align everything, but the important file is the JSON of the TLDs with all the updates and insertions.