Crash report - Githubissues

mikedesu commented 1 year ago

katana version: v1.0.2

Current Behavior:

Crashing with:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0xe07e13]

goroutine 7228 [running]:
github.com/projectdiscovery/retryablehttp-go.FromRequest(0x0)
    /home/darkmage/go/pkg/mod/github.com/projectdiscovery/retryablehttp-go@v1.0.17/request.go:174 +0x73
github.com/projectdiscovery/katana/pkg/engine/hybrid.(*Crawler).navigateRequest.func1(0xc002083f40)
    /home/darkmage/go/pkg/mod/github.com/projectdiscovery/katana@v1.0.2/pkg/engine/hybrid/crawl.go:86 +0x679
github.com/projectdiscovery/katana/pkg/engine/hybrid.(*Hijack).Start.func2(0x0?)
    /home/darkmage/go/pkg/mod/github.com/projectdiscovery/katana@v1.0.2/pkg/engine/hybrid/hijack.go:52 +0x2f
reflect.Value.call({0x1107280?, 0xc0022bb278?, 0x100c000546640?}, {0x12b1fe2, 0x4}, {0xc002badf58, 0x1, 0xc002083f40?})
    /usr/local/go/src/reflect/value.go:586 +0xb0b
reflect.Value.Call({0x1107280?, 0xc0022bb278?, 0xc002083f40?}, {0xc002badf58?, 0x711040?, 0xc000546567?})
    /usr/local/go/src/reflect/value.go:370 +0xbc
github.com/go-rod/rod.(*Browser).eachEvent.func1()
    /home/darkmage/go/pkg/mod/github.com/go-rod/rod@v0.113.3/browser.go:401 +0x412
github.com/projectdiscovery/katana/pkg/engine/hybrid.(*Hijack).Start.func3()
    /home/darkmage/go/pkg/mod/github.com/projectdiscovery/katana@v1.0.2/pkg/engine/hybrid/hijack.go:57 +0x25
created by github.com/projectdiscovery/katana/pkg/engine/hybrid.(*Crawler).navigateRequest
    /home/darkmage/go/pkg/mod/github.com/projectdiscovery/katana@v1.0.2/pkg/engine/hybrid/crawl.go:46 +0x4e5

Expected Behavior:

Expected program to continue running and saving links until completion.

Steps To Reproduce:

Taking the following input list as owned_by_lululemon.txt:

cdn.lululemon.com
cdp.lululemon.com
cloud.e.lululemon.com
dev-iexpense.lululemon.com
eu.lululemon.com
failover.lululemon.com
fr.lululemon.com
fr.shop.lululemon.com
globalstaticassets.lululemon.com
images.eu.lululemon.com
info.lululemon.com
likenew.lululemon.com
localhost.e.lululemon.com
localhost.info.lululemon.com
loopback.lululemon.com
lululemon.com
oraclefindev.lululemon.com
oraclefinqa.lululemon.com
oraclefinstg.lululemon.com
pnimages.lululemon.com
qa-iexpense.lululemon.com
qa-plm.lululemon.com
stage-search-gsr.lululemon.com
static.eu.lululemon.com
static.lululemon.com
stg-iexpense.lululemon.com
stg-plm.lululemon.com
view.e.lululemon.com
www.lululemon.com

The command ran that produced the crash:

cat owned_by_lululemon.txt | katana -r ~/resolvers.txt -hl -d 3 -c 2 -p 4 -rl 4 -o katana-d3-0.txt

MetzinAround commented 1 year ago

Thanks for the issue! @ehsandeep I know this is the same error from #380 (panic memory in headless mode) but that was fixed in v1.0.2, which @mikedesu is using. Just wanted to note the similarities here.

ehsandeep commented 1 year ago

@mikedesu I'm unable to reproduce by running the same list/cmd you shared except -r option, any specific reason for using a custom resolver? Is this include the internal domain?

mikedesu commented 1 year ago

@mikedesu I'm unable to reproduce by running the same list/cmd you shared except -r option, any specific reason for using a custom resolver? Is this include the internal domain?

Its for bug bounty hunting. I have a custom resolver list I use just in case by system defaults can't find a host's IP address. It isn't exhaustive, and contains the following items:

1.0.0.1
1.1.1.1
13.239.157.177
134.195.4.2
149.112.112.112
156.154.70.1
159.89.120.99
176.103.130.130
185.228.168.9
185.228.168.168
185.228.169.9
195.46.39.39
195.46.39.40
198.101.242.72
205.171.2.65
205.171.3.65
208.67.220.220
208.67.222.222
216.146.35.35
216.146.36.36
64.6.64.6
64.6.65.6
74.82.42.42 
76.223.122.150
76.76.10.0
76.76.19.19
76.76.2.0
77.88.8.1
77.88.8.7
77.88.8.88
8.20.247.20
8.26.56.26
8.8.4.4
8.8.8.8
84.200.69.80
84.200.70.40
89.233.43.71
9.9.9.9
91.239.100.100
94.140.14.14
94.140.15.15