search

owasp-amass / amass

In-depth attack surface mapping and asset discovery
https://owasp.org/www-project-amass/
Other
12.02k stars 1.88k forks source link

Bruteforcing breaks amass #332

Closed amalmurali47 closed 4 years ago

amalmurali47 commented 4 years ago

Installed amass from snap.

$ amass -version
v3.3.2

Command:

amass enum -d uber.com -src -public-dns -noresolvrate false -noresolvscore false -brute -json /tmp/uber -log /tmp/uber_log -v
goroutine 60821542 [semacquire]:
sync.runtime_SemacquireMutex(0xc006fedd3c, 0x10f0300, 0x1)
    /usr/local/go/src/runtime/sema.go:71 +0x47
sync.(*Mutex).lockSlow(0xc006fedd38)
    /usr/local/go/src/sync/mutex.go:138 +0xfc
sync.(*Mutex).Lock(...)
    /usr/local/go/src/sync/mutex.go:81
github.com/OWASP/Amass/v3/stringset.(*StringFilter).Duplicate(0xc006fedd30, 0xc37292c660, 0x22, 0x10f0300)
    /Users/caffix/go/src/github.com/OWASP/Amass/stringset/filter.go:24 +0x187
github.com/OWASP/Amass/v3/enum.(*Enumeration).newNameEvent(0xc0000dc420, 0xc372933560)
    /Users/caffix/go/src/github.com/OWASP/Amass/enum/names.go:28 +0x1fd
github.com/OWASP/Amass/v3/enum.(*Enumeration).executeAlts(0xc0000dc420, 0xc00e09ef00)
    /Users/caffix/go/src/github.com/OWASP/Amass/enum/brute.go:148 +0x1e5
created by github.com/OWASP/Amass/v3/enum.(*Enumeration).performAlterations
    /Users/caffix/go/src/github.com/OWASP/Amass/enum/brute.go:116 +0x143

goroutine 68569529 [semacquire]:
sync.runtime_SemacquireMutex(0xc0031ff5e4, 0xc00865fc00, 0x1)
    /usr/local/go/src/runtime/sema.go:71 +0x47
sync.(*Mutex).lockSlow(0xc0031ff5e0)
    /usr/local/go/src/sync/mutex.go:138 +0xfc
sync.(*Mutex).Lock(...)
    /usr/local/go/src/sync/mutex.go:81
github.com/OWASP/Amass/v3/resolvers.(*BaseResolver).IsStopped(0xc0031ff520, 0xc00865fc00)
    /Users/caffix/go/src/github.com/OWASP/Amass/resolvers/resolver.go:194 +0xb1
github.com/OWASP/Amass/v3/resolvers.(*ScoredResolver).IsStopped(0xc003626870, 0x0)
    /Users/caffix/go/src/github.com/OWASP/Amass/resolvers/scored.go:52 +0x34
github.com/OWASP/Amass/v3/resolvers.(*RateMonitoredResolver).IsStopped(0xc00366b200, 0x0)
    /Users/caffix/go/src/github.com/OWASP/Amass/resolvers/ratemon.go:65 +0x34
github.com/OWASP/Amass/v3/resolvers.(*ResolverPool).numUsableResolvers(0xc005313810, 0xc3722b61e0)
    /Users/caffix/go/src/github.com/OWASP/Amass/resolvers/pool.go:461 +0x6f
github.com/OWASP/Amass/v3/resolvers.(*ResolverPool).Resolve(0xc005313810, 0x153eae0, 0xc0070cf590, 0xc0410691a0, 0x1c, 0x10ee058, 0x1, 0x0, 0x1e07670, 0x0, ...)
    /Users/caffix/go/src/github.com/OWASP/Amass/resolvers/pool.go:344 +0x2bc
github.com/OWASP/Amass/v3/services.(*DNSService).processDNSRequest(0xc003748800, 0x153eae0, 0xc0070cf590, 0xc0428cf6e0)
    /Users/caffix/go/src/github.com/OWASP/Amass/services/dnssrv.go:77 +0x3f4
created by github.com/OWASP/Amass/v3/services.(*DNSService).OnDNSRequest
    /Users/caffix/go/src/github.com/OWASP/Amass/services/dnssrv.go:50 +0x98

Not sure why this is happening.

caffix commented 4 years ago

Interesting issue. We will look into it.

In the meantime, try removing the following flags: -noresolvrate false -noresolvscore false

caffix commented 4 years ago

@amalmurali47 So far, I was not been able to reproduce the bug you have reported

fork-while-fork commented 4 years ago

Were there more stack traces @amalmurali47 ? Those two goroutine stack traces just appear to be waiting to grab a mutex which isn't a problem in itself.

sumgr0 commented 4 years ago

I've been experiencing the same with -brute option. When used as -> amass enum -brute -d -o process never ends and continues running endlessly. While the -passive run completes as expected.

amass version 3.4.0 installed via snap.

lappsec commented 4 years ago

I'm having the same issue as sumgr0. When running a simple "amass enum -brute -d -o " it hangs after/during ASN enumeration and never finishes. This is where it hangs:

amass

This is also amass 3.4.0 installed via snap on Ubuntu 18.04.3 LTS.

fork-while-fork commented 4 years ago

@sumgr0 @lappsec It's not clear that the problem you're experiencing is the same as OPs. However, I do have a fix for your problem that should land in the next release.

@amalmurali47 Any other feedback on the issue you're experiencing would be appreciated.

sumgr0 commented 4 years ago

@fork-while-fork on the contrary, going through the OPs problem, it looks different. However, the error is encountered while using the -brute option itself. My bad to report it in the same thread.

Kindly suggest.

fork-while-fork commented 4 years ago

@sumgr0 Follow #335 to track the fix for your issue.

lmeyerov commented 4 years ago

Not sure if the same, but I was picking this back up and got stuck a 1 dns query / sec:

#mode = passive
maximum_dns_queries = 10000

[domains]
****

[resolvers]
resolver = 1.1.1.1 ; Cloudflare
resolver = 8.8.8.8 ; Google
resolver = 64.6.64.6 ; Verisign
resolver = 74.82.42.42 ; Hurricane Electric
resolver = 1.0.0.1 ; Cloudflare Secondary
resolver = 8.8.4.4 ; Google Secondary
resolver = 9.9.9.10 ; Quad9 Secondary
resolver = 64.6.65.6 ; Verisign Secondary
resolver = 77.88.8.1 ; Yandex.DNS Secondary

[shodan]
apikey = ***

Started fast (1K+ qry/s) then => 1/s:

./amass/binary/amass_v3.4.1_linux_amd64/amass \
    enum -config ./amass/config.ini \
    -json ./amass/out.json \
    -src -log err.log 

=>
...
Querying GoogleCT for ***.com subdomains
Average DNS queries performed: 1/sec
Average DNS queries performed: 1/sec

I'll switch to passive - would not be surprised if operator error here, though I thought we had the above cmd working before :) Maybe there's a timeout option or something, and when speed drops to clearly inoperable like this, should emit a "You should likely X" warning?

lmeyerov commented 4 years ago

Cool, will give a spin right now. Thanks @caffix !

lmeyerov commented 4 years ago

Switched to 3.4.2 and still seeing slowdown. Not sure if the tool or just config settings getting us rate limited. @caffix anything that would help?

image

JesseClarkND commented 4 years ago

I'm seeing this issue as well in v3.5.1 It happens after a few (5-10) runs tho of an enum.ini amass enum -config /root/amass/bin/enum.ini -d alios.cn https://pastebin.com/v4SHqKwE

Then even a simple amass enum -d alios.cn won't work

Sits like this for hours image

caffix commented 4 years ago

@JesseClarkND Please upgrade to the latest version (v3.5.4) and try again, since some important improvements have been released. I would wipe out the default output directory ($HOME/.config/amass on Linux) before using the latest version of the tool

JesseClarkND commented 4 years ago

Thanks @caffix everything has been working well since the update!

caffix commented 4 years ago

Great to hear. Thank you!

IvoPereira commented 4 years ago

@caffix I am using v3.10.5 and I am facing this exact same issue.

After wiping out the data like you suggest it works for a bit, until it sudden starts being stuck again and I need to wipe the config again to get it to work.

Any suggestions on what I might be missing?

sidd-pidd64 commented 3 years ago

github.com/OWASP/Amass/v3/resolvers.(ResolverPool).SubdomainToDomain(0xc000215220, 0xc004ee42a0, 0x28, 0xc005a989b4, 0xc) /home/caffix/go/src/github.com/OWASP/Amass/resolvers/pool.go:178 +0x9d github.com/OWASP/Amass/v3/intel.(Collection).investigateAddr(0xc009c40a00, 0xc005a989b4, 0xc) /home/caffix/go/src/github.com/OWASP/Amass/intel/intel.go:192 +0x606 created by github.com/OWASP/Amass/v3/intel.(*Collection).HostedDomains /home/caffix/go/src/github.com/OWASP/Amass/intel/intel.go:136

sidd-pidd64 commented 3 years ago

i was getting issue with amass intel -ans [number] @caffix