search

crawlab-team / crawlab

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架
https://www.crawlab.cn
BSD 3-Clause "New" or "Revised" License
11.38k stars 1.8k forks source link

container crash (Panic) #1444

Open saleh-hom opened 10 months ago

saleh-hom commented 10 months ago

I am using Crawlab pro as a stack deployed on a docker swarm, all features are working fine. I am also using 1 worker node on a different machine. However every few days the master node panics with the below error messages.

panic: runtime error: index out of range [104] with length 62 goroutine 15940761 [running]: github.com/emirpasic/gods/lists/arraylist.(List).Add(...) /go/pkg/mod/github.com/emirpasic/gods@v1.18.1/lists/arraylist/arraylist.go:47 github.com/crawlab-team/crawlab-pro/core/metric.(Service).getAllNodeMetrics.func1.1({0xc000dcc000?, 0xd4, 0xc0010ec2b0?}) /go/src/app/core/metric/service.go:226 +0x22d github.com/crawlab-team/crawlab-pro/core/metric.(Service).requestRemoteNodeMetrics(0xc000024480, {0x229d490, 0xc001c37860}, {0x22aa028, 0xc000af5930}, 0x1985063?, 0xc001990ee8) /go/src/app/core/metric/service.go:334 +0x4da github.com/crawlab-team/crawlab-pro/core/metric.(Service).getAllNodeMetrics.func1({{0x65, 0xa7, 0xc1, 0xbd, 0xdc, 0x6e, 0x24, 0xde, 0x5b, 0x5a, ...}, ...}) /go/src/app/core/metric/service.go:224 +0xfa created by github.com/crawlab-team/crawlab-pro/core/metric.(*Service).getAllNodeMetrics /go/src/app/core/metric/service.go:223 +0x40c

Steps to reproduce the behavior: I don't think there are set of steps to reproduce the issue, simply leaving the stack running will eventually make the container panic

oleksa-dev commented 8 months ago

@saleh-hom hi, have the same panic issue runtime error: index out of range in pro version. It appears randomly during requests to API performed my Crawlab frontend side.