search

crawlab-team / crawlab

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架
https://www.crawlab.cn
BSD 3-Clause "New" or "Revised" License
11.38k stars 1.8k forks source link

k8s進行0.4.9新版本更新時，出現錯誤 #666

Closed jarvisaoieong closed 4 years ago

jarvisaoieong commented 4 years ago

Bug 描述 我在k8s集群中升級crawlab到0.4.9版本時，報了以下錯誤

* Starting nginx nginx
   ...done.
[GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.

[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
 - using env:   export GIN_MODE=release
 - using code:  gin.SetMode(gin.ReleaseMode)

2020/04/01 11:21:49  info initialized config successfully
2020/04/01 11:21:49  info initialized log config successfully
2020/04/01 11:21:49  info periodically cleaning log is switched off
2020/04/01 11:21:49  info initialized MongoDB successfully
2020/04/01 11:21:49  info initialized Redis successfully
2020/04/01 11:21:49  info initialized schedule successfully
2020/04/01 11:21:49  info initialized user service successfully
2020/04/01 11:21:49  info initialized dependency fetcher successfully
2020/04/01 11:21:49  info initialized task executor successfully
2020/04/01 11:21:49  info register type is :*register.HostnameRegister
2020/04/01 11:21:49  info initialized node service successfully
{subscribe nodes:master 1}
{subscribe nodes:public 1}
2020/04/01 11:21:49 error add func task error: beginning of range (0) below minimum (1): 0
goroutine 1 [running]:
runtime/debug.Stack(0x0, 0x0, 0x0)
    /usr/local/go/src/runtime/debug/stack.go:24 +0x9d
runtime/debug.PrintStack()
    /usr/local/go/src/runtime/debug/stack.go:16 +0x22
crawlab/services.(*GitCronScheduler).AddJob(0xc000010e20, 0xc000386a04, 0xc, 0xc000386a20, 0xd, 0xc000386a40, 0xd, 0xc000386a50, 0xa, 0xc000386a70, ...)
    /go/src/app/services/git.go:122 +0x1af
crawlab/services.(*GitCronScheduler).Update(0xc000010e20, 0x1, 0x1)
    /go/src/app/services/git.go:105 +0x1f9
crawlab/services.(*GitCronScheduler).Start(0xc000010e20, 0xc000010e20, 0x1)
    /go/src/app/services/git.go:66 +0x8a
crawlab/services.InitSpiderService(0x189def0, 0xfe58b4)
    /go/src/app/services/spider.go:543 +0x1dc
main.main()
    /go/src/app/main.go:112 +0x599
2020/04/01 11:21:49 error add job error: beginning of range (0) below minimum (1): 0, job: youtube-reply, cron: 0 0 0 0 * *
goroutine 1 [running]:
runtime/debug.Stack(0x0, 0x0, 0x0)
    /usr/local/go/src/runtime/debug/stack.go:24 +0x9d
runtime/debug.PrintStack()
    /usr/local/go/src/runtime/debug/stack.go:16 +0x22
crawlab/services.(*GitCronScheduler).Update(0xc000010e20, 0x1, 0x1)
    /go/src/app/services/git.go:107 +0x396
crawlab/services.(*GitCronScheduler).Start(0xc000010e20, 0xc000010e20, 0x1)
    /go/src/app/services/git.go:66 +0x8a
crawlab/services.InitSpiderService(0x189def0, 0xfe58b4)
    /go/src/app/services/spider.go:543 +0x1dc
main.main()
    /go/src/app/main.go:112 +0x599
2020/04/01 11:21:49 error update scheduler error: beginning of range (0) below minimum (1): 0
goroutine 1 [running]:
runtime/debug.Stack(0x0, 0x0, 0x0)
    /usr/local/go/src/runtime/debug/stack.go:24 +0x9d
runtime/debug.PrintStack()
    /usr/local/go/src/runtime/debug/stack.go:16 +0x22
crawlab/services.(*GitCronScheduler).Start(0xc000010e20, 0xc000010e20, 0x1)
    /go/src/app/services/git.go:68 +0x14e
crawlab/services.InitSpiderService(0x189def0, 0xfe58b4)
    /go/src/app/services/spider.go:543 +0x1dc
main.main()
    /go/src/app/main.go:112 +0x599
2020/04/01 11:21:49 error init spider service error:beginning of range (0) below minimum (1): 0
goroutine 1 [running]:
runtime/debug.Stack(0x0, 0x0, 0x0)
    /usr/local/go/src/runtime/debug/stack.go:24 +0x9d
runtime/debug.PrintStack()
    /usr/local/go/src/runtime/debug/stack.go:16 +0x22
main.main()
    /go/src/app/main.go:114 +0x62d
panic: beginning of range (0) below minimum (1): 0

goroutine 1 [running]:
main.main()
    /go/src/app/main.go:115 +0x47f2

大概會是什麼原因呢?

复现步骤 该 Bug 复现步骤如下

  1. wget https://raw.githubusercontent.com/crawlab-team/crawlab/master/k8s/crawlab-master.yaml
  2. wget https://raw.githubusercontent.com/crawlab-team/crawlab/master/k8s/crawlab-worker.yaml
  3. 把兩個檔案中的 tikazyq/crawlab:latest 改為 tikazyq/crawlab:0.4.9
  4. 執行 kubectl apply -f crawlab-master.yaml
  5. 出現錯誤

期望结果 能順利升級。

tikazyq commented 4 years ago

临时解决方案步骤:

  1. 备份数据库
  2. 删除数据库里的 schedules
  3. 重启服务
jarvisaoieong commented 4 years ago

刪除數據後可以了，期待新版本能修正這個問題