Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台,支持任何语言和框架
BSD 3-Clause "New" or "Revised" License
11.38k
stars
1.8k
forks
source link
调度不成功,更新docker images后调度任务,一直是待定状态。 #1253
Closed
liziqiang0523 closed 1 year ago
Bug 描述 每次更新docker镜像后,原来的爬虫就不能正常调度了。一直是待定状态。 复现步骤 该 Bug 复现步骤如下
期望结果 任务调度恢复正常。不用再删除爬虫重新添加。
截屏
日志 crawlab_master | /go/pkg/mod/github.com/cenkalti/backoff/v4@v4.1.0/retry.go:28 github.com/cenkalti/backoff/v4.Retry() crawlab_master | /go/pkg/mod/github.com/crawlab-team/crawlab-fs@v0.6.0-beta.20211101.1940.0.20221218100256-a28d12756f73/seaweedfs_manager.go:308 github.com/crawlab-team/crawlab-fs.(SeaweedFsManager).start.func1() crawlab_master | /usr/local/go/src/runtime/asm_amd64.s:1371 runtime.goexit() crawlab_master | [GIN] 2023/01/12 - 10:15:15 | 200 | 10.277706ms | 127.0.0.1 | GET "/tasks?page=1&size=10&conditions=[]&sort=[]&stats=true" crawlab_master | [GIN] 2023/01/12 - 10:15:16 | 500 | 2.378937ms | 127.0.0.1 | GET "/filer/fs/63b525ec2428a9c16aeab08e/.gitignore" crawlab_master | unexpected EOF crawlab_master | /go/pkg/mod/github.com/crawlab-team/go-trace@v0.1.1/trace.go:6 github.com/crawlab-team/go-trace.PrintError() crawlab_master | /go/pkg/mod/github.com/crawlab-team/crawlab-fs@v0.6.0-beta.20211101.1940.0.20221218100256-a28d12756f73/seaweedfs_manager.go:335 github.com/crawlab-team/crawlab-fs.(SeaweedFsManager).error() crawlab_master | /go/pkg/mod/github.com/crawlab-team/crawlab-fs@v0.6.0-beta.20211101.1940.0.20221218100256-a28d12756f73/seaweedfs_manager.go:449 github.com/crawlab-team/crawlab-fs.(SeaweedFsManager).downloadFile() crawlab_master | /go/pkg/mod/github.com/crawlab-team/crawlab-fs@v0.6.0-beta.20211101.1940.0.20221218100256-a28d12756f73/seaweedfs_manager.go:309 github.com/crawlab-team/crawlab-fs.(SeaweedFsManager).start.func1.1() crawlab_master | /go/pkg/mod/github.com/cenkalti/backoff/v4@v4.1.0/retry.go:55 github.com/cenkalti/backoff/v4.RetryNotifyWithTimer() crawlab_master | /go/pkg/mod/github.com/cenkalti/backoff/v4@v4.1.0/retry.go:34 github.com/cenkalti/backoff/v4.RetryNotify() crawlab_master | /go/pkg/mod/github.com/cenkalti/backoff/v4@v4.1.0/retry.go:28 github.com/cenkalti/backoff/v4.Retry() crawlab_master | /go/pkg/mod/github.com/crawlab-team/crawlab-fs@v0.6.0-beta.20211101.1940.0.20221218100256-a28d12756f73/seaweedfs_manager.go:308 github.com/crawlab-team/crawlab-fs.(*SeaweedFsManager).start.func1() crawlab_master | /usr/local/go/src/runtime/asm_amd64.s:1371 runtime.goexit() crawlab_master | [GIN] 2023/01/12 - 10:15:16 | 500 | 2.059957ms | 127.0.0.1 | GET "/filer/fs/63b525ec2428a9c16aeab08e/.gitignore" crawlab_master | unexpected EOF