Open kristovatlas opened 7 years ago
Note that for anyone who can tolerate waiting a few extra cron invocations, this doesn't matter much since it's stable enough over time to catch domains that get "missed" in a single invocation. Time-until-seen can be decreased by running cron job more often.
I've seen this before as well and just chalked it up to DNS rate limits (we're pretty much loading up a bunch of domains and firing it off which I've actually saturated my test DSL line before). If we introduce any internal rate limits I'm fine with that, we'll have to start modifying the dnstwist code though to handle it.
Something I noticed when suffering from #9 is that the number of domain variants within a short period of time is inconsistent. This is probably some DNS quirk but might merit more research.
Here's output from a single invocation: