search

projectdiscovery / nuclei

Fast and customizable vulnerability scanner based on simple YAML based DSL.
https://docs.projectdiscovery.io/tools/nuclei
MIT License
19.9k stars 2.45k forks source link

Host with/without HTTP scheme for different hosts #667

Closed savushkin-yauheni closed 2 years ago

savushkin-yauheni commented 3 years ago

Hi team!

Nuclei version

2.3.4

Describe the bug


nuclei -target https://op-ma.apple.com  -t technologies/favicon-detection.yaml  -json

{"templateID":"favicon-detection","info":{"name":"favicon-detection","severity":"info","author":"un-fmunozs","reference":"- https://twitter.com/brsn76945860/status/1171233054951501824\n- https://gist.github.com/yehgdotnet/b9dfc618108d2f05845c4d8e28c5fc6a\n- https://medium.com/@Asm0d3us/weaponizing-favicon-ico-for-bugbounties-osint-and-what-not-ace3c214e139\n- https://github.com/devanshbatham/FavFreak\n- https://github.com/sansatart/scrapts/blob/master/shodan-favicon-hashes.csv\n"},"matcher_name":"jetty-404","type":"http","host":"op-ma.apple.com","matched":"https://op-ma.apple.com/favicon.ico","ip":"17.171.49.194","timestamp":"2021-04-10T09:54:40.529034663Z"}

"host":"op-ma.apple.com" - without http scheme, incorrect

nuclei -target https://google.com  -t technologies/favicon-detection.yaml  -json

{"templateID":"favicon-detection","info":{"severity":"info","author":"un-fmunozs","reference":"- https://twitter.com/brsn76945860/status/1171233054951501824\n- https://gist.github.com/yehgdotnet/b9dfc618108d2f05845c4d8e28c5fc6a\n- https://medium.com/@Asm0d3us/weaponizing-favicon-ico-for-bugbounties-osint-and-what-not-ace3c214e139\n- https://github.com/devanshbatham/FavFreak\n- https://github.com/sansatart/scrapts/blob/master/shodan-favicon-hashes.csv\n","name":"favicon-detection"},"matcher_name":"google","type":"http","host":"https://google.com","matched":"https://google.com/favicon.ico","ip":"142.250.186.78","timestamp":"2021-04-10T09:58:13.778559538Z"}

"host":"https://google.com" - with http scheme, correct

it looks strange, but i tried a lot of domains, and seems that it happened only with apple.com domains O_o

but anyway i think it's a bug.

Thanks.

ehsandeep commented 3 years ago

Thank you @savushkin-yauheni for reporting this, I can reproduce this behavior.

gaby commented 3 years ago

I'm also affected by this bug.

Mzack9999 commented 3 years ago

@savushkin-yauheni @gaby Thanks for reporting the issue. I'm trying to reproduce the behavior to implement a fix, but any target is correctly prefixed with http, and the one from the example provided is unresponsive. Were you able to reproduce this with any other target? Also, would it be possible to share one (if it's private, feel free to jump on our discord channel and share via DM)? Thanks!

savushkin-yauheni commented 3 years ago

Hi @Mzack9999 I will find the similar target tomorrow. Thank you!

savushkin-yauheni commented 3 years ago

Hi again @Mzack9999

nuclei -target http://cgi6.training.apple.com  -t technologies/tech-detect.yaml  -json
nuclei -target http://mfi.apple.com  -t technologies/tech-detect.yaml  -json
ehsandeep commented 3 years ago 
nuclei -target http://cgi6.training.apple.com  -t technologies/tech-detect.yaml -silent -json | jq -r .host
training.apple.com
nuclei -target https://www.apple.com  -t technologies/tech-detect.yaml  -json -silent | jq -r .host
https://www.apple.com

@Mzack9999 looks like this is an issue with the host involving redirects.

Mzack9999 commented 3 years ago

cgi6.training.apple.com redirects to training.apple.com, which returns a custom host header within the response that overwrites at https://github.com/projectdiscovery/nuclei/blob/a6b15f948dcb81e02dfe7df5754a38cef904eaa5/v2/pkg/protocols/http/operators.go#L102 the legitimate one set at https://github.com/projectdiscovery/nuclei/blob/a6b15f948dcb81e02dfe7df5754a38cef904eaa5/v2/pkg/protocols/http/operators.go#L90

The issue here is that the current implementation implies a potential overlap between the response header name and existing map keys, leading to their values being overwritten. To keep compatibility, a possible approach would be to use an alternative syntax when this happens, for example naming the response headers as:

header_name => resp_header_name