search

hakluke / hakrawler

Simple, fast web crawler designed for easy, quick discovery of endpoints and assets within a web application
https://hakluke.com
GNU General Public License v3.0
4.42k stars 483 forks source link

issue:panic: runtime error: slice bounds out of range #61

Closed ipk1 closed 4 years ago

ipk1 commented 4 years ago

hi get the following error when i run it

goroutine 237 [running]: bufio.(Writer).Write(0xc000202840, 0xc00173ae00, 0xf1, 0x100, 0xf1, 0x100, 0xc000d207e0) /usr/local/go/src/bufio/bufio.go:625 +0x230 github.com/hakluke/hakrawler/pkg/collector.(Collector).colorPrint(0xc0003217d0, 0xb1b2c0, 0xc0010e83c0, 0xc000d64a50, 0xf0, 0x1) /root/go_projects/src/github.com/hakluke/hakrawler/pkg/collector/collector.go:393 +0x612 github.com/hakluke/hakrawler/pkg/collector.(Collector).recordIfInScope(0xc0003217d0, 0xb1b2c0, 0xc0010e83c0, 0xc000318600, 0x19, 0xc000d64a50, 0xf0, 0xc00031e580, 0x1) /root/go_projects/src/github.com/hakluke/hakrawler/pkg/collector/collector.go:280 +0x19d github.com/hakluke/hakrawler/pkg/collector.(Collector).visitWaybackURLs(0xc0003217d0, 0xc000318600, 0x19, 0xc000321830, 0xc00031e580) /root/go_projects/src/github.com/hakluke/hakrawler/pkg/collector/collector.go:293 +0x13d github.com/hakluke/hakrawler/pkg/collector.(Collector).Crawl.func3(0xc000324150, 0xc0003217d0, 0xc000318600, 0x19, 0xc000321830, 0xc00031e580) /root/go_projects/src/github.com/hakluke/hakrawler/pkg/collector/collector.go:155 +0x7f created by github.com/hakluke/hakrawler/pkg/collector.(Collector).Crawl /root/go_projects/src/github.com/hakluke/hakrawler/pkg/collector/collector.go:153 +0x453 panic: runtime error: slice bounds out of range [:4131] with capacity 4096

goroutine 495 [running]: bufio.(Writer).Flush(0xc000202840, 0xc002720810, 0x23) /usr/local/go/src/bufio/bufio.go:591 +0x1c0 bufio.(Writer).Write(0xc000202840, 0xc002720810, 0x26, 0x30, 0x26, 0x30, 0xc000d84850) /usr/local/go/src/bufio/bufio.go:627 +0xfa github.com/hakluke/hakrawler/pkg/collector.(Collector).colorPrint(0xc0006514d0, 0xb1b2c0, 0xc001068b80, 0xc001950240, 0x25, 0x1) /root/go_projects/src/github.com/hakluke/hakrawler/pkg/collector/collector.go:393 +0x612 github.com/hakluke/hakrawler/pkg/collector.(Collector).recordIfInScope(0xc0006514d0, 0xb1b2c0, 0xc001068b80, 0xc00040eaa0, 0x1b, 0xc001950240, 0x25, 0xc000296dc0, 0x40e4c8) /root/go_projects/src/github.com/hakluke/hakrawler/pkg/collector/collector.go:280 +0x19d github.com/hakluke/hakrawler/pkg/collector.(Collector).visitHTMLFunc.func1(0xc0009e8f00) /root/go_projects/src/github.com/hakluke/hakrawler/pkg/collector/collector.go:188 +0x235 github.com/gocolly/colly.(Collector).handleOnHTML.func1(0x9, 0xc001190000) /root/go_projects/src/github.com/gocolly/colly/colly.go:1074 +0x8c github.com/PuerkitoBio/goquery.(Selection).Each(0xc0006fd080, 0xc000243cd0, 0x7) /root/go_projects/src/github.com/PuerkitoBio/goquery/iteration.go:10 +0x53 github.com/gocolly/colly.(Collector).handleOnHTML(0xc000294700, 0xc001fe0000, 0x0, 0x0) /root/go_projects/src/github.com/gocolly/colly/colly.go:1064 +0x21b github.com/gocolly/colly.(Collector).fetch(0xc000294700, 0xc000318de0, 0x1b, 0xa370d3, 0x3, 0x1, 0x0, 0x0, 0xc000322a40, 0xc000651620, ...) /root/go_projects/src/github.com/gocolly/colly/colly.go:676 +0x487 github.com/gocolly/colly.(Collector).scrape(0xc000294700, 0xc000318de0, 0x1b, 0xa370d3, 0x3, 0x1, 0x0, 0x0, 0x0, 0xc000651620, ...) /root/go_projects/src/github.com/gocolly/colly/colly.go:577 +0x47e github.com/gocolly/colly.(Collector).Visit(0xc000294700, 0xc00040eaa0, 0x1b, 0x0, 0x0) /root/go_projects/src/github.com/gocolly/colly/colly.go:446 +0x82 github.com/hakluke/hakrawler/pkg/collector.(Collector).Crawl.func4(0xc000324180, 0xc0006514d0, 0xc00040eaa0, 0x1b) /root/go_projects/src/github.com/hakluke/hakrawler/pkg/collector/collector.go:163 +0x6f created by github.com/hakluke/hakrawler/pkg/collector.(*Collector).Crawl /root/go_projects/src/github.com/hakluke/hakrawler/pkg/collector/collector.go:161 +0x393 panic: runtime error: slice bounds out of range [:4131] with capacity 4096

goroutine 837 [running]: bufio.(Writer).Flush(0xc000202840, 0xc0022b70c0, 0x23) /usr/local/go/src/bufio/bufio.go:597 +0x1b0 bufio.(Writer).Write(0xc000202840, 0xc0022b70c0, 0x34, 0x40, 0x34, 0x40, 0xc000d690a0) /usr/local/go/src/bufio/bufio.go:627 +0xfa github.com/hakluke/hakrawler/pkg/collector.(Collector).colorPrint(0xc0008f3b90, 0xb1b2c0, 0xc00128af40, 0xc00152b3c0, 0x33, 0x1) /root/go_projects/src/github.com/hakluke/hakrawler/pkg/collector/collector.go:393 +0x612 github.com/hakluke/hakrawler/pkg/collector.(Collector).recordIfInScope(0xc0008f3b90, 0xb1b2c0, 0xc00128af40, 0xc000291760, 0x17, 0xc00152b3c0, 0x33, 0xc00031f040, 0xc000291101) /root/go_projects/src/github.com/hakluke/hakrawler/pkg/collector/collector.go:280 +0x19d github.com/hakluke/hakrawler/pkg/collector.(Collector).visitWaybackURLs(0xc0008f3b90, 0xc000291760, 0x17, 0xc0008f3bf0, 0xc00031f040) /root/go_projects/src/github.com/hakluke/hakrawler/pkg/collector/collector.go:293 +0x13d github.com/hakluke/hakrawler/pkg/collector.(Collector).Crawl.func3(0xc00041a2b0, 0xc0008f3b90, 0xc000291760, 0x17, 0xc0008f3bf0, 0xc00031f040) /root/go_projects/src/github.com/hakluke/hakrawler/pkg/collector/collector.go:155 +0x7f created by github.com/hakluke/hakrawler/pkg/collector.(*Collector).Crawl /root/go_projects/src/github.com/hakluke/hakrawler/pkg/collector/collector.go:153 +0x453

i am on go1.14.3

hakluke commented 4 years ago

Make sure you are using the latest hakrawler version. Check with hakrawler -v

hakluke commented 4 years ago

Closing due to inactivity