zmap / zdns

Fast DNS Lookup Library and CLI Tool
Apache License 2.0
891 stars 121 forks source link

Add reliable large-scale scanning integration test #374

Closed phillip-stephens closed 2 months ago

phillip-stephens commented 2 months ago

PR #370 attempted to add a large-scale integration test where we'd scan the top 100 domains and attempt to verify they were actually hosted at that IP address. This turned out to be quite flaky and unreliable and was removed.

This PR is to bring a more reproducible test into ZDNS. It also runs in only 20 seconds, so it's much more efficient than the old test.

From the README.md documenting this test:

Large Scan Integration Test

This script runs a large scan integration test for the A and ALOOKUP modules of zdns.

10,000 domains were set up in the zdns-testing.com namespace, 5k for subdomain0.zdns-testing.com and 5k for subdomain1.subdomain0.zdns-testing.com, with the following structure:

a.subdomain0.zdns-testing.com -> "0.0.0.0", "0.0.0.1"
b.subdomain0.zdns-testing.com -> "0.0.0.2", "0.0.0.3"
...
gjh.subdomain0.zdns-testing.com -> "0.0.39.14", "0.0.39.15"

a.subdomain1.subdomain0.zdns-testing.com -> "0.1.0.0", "0.1.0.1"
b.subdomain1.subdomain0.zdns-testing.com -> "0.1.0.2", "0.1.0.3" 
...
gjh.subdomain1.subdomain0.zdns-testing.com -> "0.1.39.14", "0.1.39.15"
  1. These two subdomains (subdomain1.subdomain0 and subdomain0) have unique nameservers compared to eachother and to zdns-testing.com
  2. NS records were set up to point from zdns-testing.com -> subdomain0.zdns-testing.com and subdomain0.zdns-testing.com -> subdomain1.subdomain0.zdns-testing.com.

This should enable us to test the iterative resolution of zdns in a reproducible and reliable manner.

Changes