search

atenreiro / opensquat

The openSquat is an open-source tool for detecting domain look-alikes by searching for newly registered domains that might be impersonating legit domains and brands.
https://opensquat.com
GNU General Public License v3.0
685 stars 130 forks source link

Database issue #59

Closed wargasmtg closed 3 years ago

wargasmtg commented 3 years ago

Hello,

There is something wrong with the latest database set of today 17-5-2021

I pulled the data and noticed that some domains looked very familiar:

digi-dtoeslag-nl.icu -- digi-dtoeslagen-nl.icu digi-toeslagen-nl.icu digid-toeslagen-nl.icu digidtoeslagen-nl.icu digitoeslagen-nl.icu

When I checked these domain names I noticed that they were registered on 06-05-2021 More importantly, I already submitted these indicators to Pulsedive on 10-05-2021 See example https://pulsedive.com/indicator/?iid=24489603 or https://pulsedive.com/indicator/?iid=24489595

From what I sampled, the .com TLD seems to be working fine, but other TLDs have older data, including .NET (same registry operator as .COM)

During this weekend I noticed that the daily amount of fresh domain names was much much smaller as usual, 28k and 44k. But usually, it is constant around 100-200k it sometimes drops between 100k but not often is my experience and I pull data every day.

wargasmtg commented 3 years ago

After sampling some more I notice that .com is also affected and those registrations are also from 06-05-2021 coronavirusca.com coronavirusglobalnews.com coronavirushealthcare.com coronavirushospice.com coronavirusmedic.com coronavirusmedical.com coronavirusmedics.com coronavirusnhs.com coronavirusnurses.com coronavirusrobot.com coronavirusscarf.com coronavirustoll.com coronavirustopmasks.com coronavirustransparency.com

wargasmtg commented 3 years ago

Okay it seems the issue is no longer present and all looks good.

atenreiro commented 3 years ago

Hello @wargasmtg

Thanks again for reporting this. The feeds are mostly reliable but, there have been sporadic instances where the feeds seem a bit erratic or delayed. I have considered to get a more reliable feed provider, unfortunately that will increase the costs for me as I am accommodating all costs from own pocket.

I thought about moving the project to a freemium model, so that, I can fund additional feeds. For sure, the project will always be free on its core as I believe that securing your domains and customers and keeping the internet secure should be available at no cost.

Any thoughts?