projectdiscovery / dnsx

dnsx is a fast and multi-purpose DNS toolkit allow to run multiple DNS queries of your choice with a list of user-supplied resolvers.
https://docs.projectdiscovery.io/tools/dnsx
MIT License
2.14k stars 238 forks source link

Retry with different resolvers #708

Open pdelteil opened 1 month ago

pdelteil commented 1 month ago

Please describe your feature request:

Often times while scanning domains you will get many false positives or REFUSED states. It would be useful to define a retry flag that retries with N resolvers. I'm assuming the current retry flag uses the same resolver every time.

Describe the use case of this feature:

Let's say we are looking for resolving domains (NOERROR status code), due to resolvers being blocked or malfunctioning (or rate limited?) the result might be REFUSED. I would like then to have this conditions matched:

Show a warning if N is greater than the resolvers defined in the -r parameter.

Thank you.

GeorginaReeder commented 1 month ago

Thanks so much for your feature request @pdelteil , we'll take a look into it!

calab33p commented 1 week ago

@pdelteil , I believe the behavior you are looking for is already there via

-retry int number of dns attempts to make (must be at least 1) (default 2)

It seems to work that way from my read onf the retryabledns code. N would be the retry count. Every retry I believe goes to a new resolver from the list you gave it.

CC @GeorginaReeder

pdelteil commented 1 week ago

Not the same.

As far as I know, a retry only occurs when the response from the request is empty. Not when getting a SERVFAIL: https://github.com/projectdiscovery/retryabledns/blob/main/client.go#L187

Saludos,

Philippe Delteil

On Fri, 30 Aug 2024 at 09:01, calab33p @.***> wrote:

@pdelteil https://github.com/pdelteil , I believe the behavior you are looking for is already there via

-retry int number of dns attempts to make (must be at least 1) (default 2)

It seems to work that way from my read onf the retryabledns code. N would be the retry count. Every retry I believe goes to a new resolver from the list you gave it.

CC @GeorginaReeder https://github.com/GeorginaReeder

— Reply to this email directly, view it on GitHub https://github.com/projectdiscovery/dnsx/issues/708#issuecomment-2321358554, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE2OS72SFSEYPL4U2UK5K7TZUB3MVAVCNFSM6AAAAABLPYQX2KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRRGM2TQNJVGQ . You are receiving this because you were mentioned.Message ID: @.***>

calab33p commented 1 week ago

Yeah, it's a little confusing how retryable has both the Do() and QueryMultiple() functions, but I don't believe the code you are referencing will run. Looks like dnsx is using QueryMultiple() instead and that code retries if Rcode isn't NOERROR:

https://github.com/projectdiscovery/retryabledns/blob/main/client.go#L417

Maybe give it a try @pdelteil

calab33p commented 1 week ago

Also note https://github.com/projectdiscovery/retryabledns/blob/main/client.go#L327 where the next resolver is chosen. Personally, given the way the code is written, I believe the optimal number for retries is a multiple of the number of resolvers configured. So, if you have 50 resolvers, then 50 or 100 or ....

pdelteil commented 1 week ago

You're making comments based on your idea of the code but not from testing. I created this issue for the repo owner to do the heavy lifting regarding the functionality. What you said is not what I need from this tool.

On Sat, Aug 31, 2024, 01:07 calab33p @.***> wrote:

Also note https://github.com/projectdiscovery/retryabledns/blob/main/client.go#L327 where the next resolver is chosen. Personally, given the way the code is written, I believe the optimal number for retries is a multiple of the number of resolvers configured. So, if you have 50 resolvers, then 50 or 100 or ....

— Reply to this email directly, view it on GitHub https://github.com/projectdiscovery/dnsx/issues/708#issuecomment-2322789496, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE2OS76FRHWQNCRUKZOX4G3ZUFMSPAVCNFSM6AAAAABLPYQX2KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRSG44DSNBZGY . You are receiving this because you were mentioned.Message ID: @.***>