crawlab-team / crawlab

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台,支持任何语言和框架
https://www.crawlab.cn
BSD 3-Clause "New" or "Revised" License
11.38k stars 1.8k forks source link

worker任务一直卡在待定状态 #1386

Closed geosmart closed 1 year ago

geosmart commented 1 year ago

任务提交后,一直卡在待定状态,安装python依赖也是卡着不动。

worker日志

[root@hz-sjzn-dwh-199-150-136 crawlab]# docker logs -f crawlab_worker 
Using config file: /app/backend/conf/config.yml
2023/09/04 10:11:36  info [GrpcClient] grpc client connected to 10.199.150.14:9666
2023/09/04 10:11:36  info [GrpcClient] grpc client registered client services
2023/09/04 10:11:36  info [GrpcClient] grpc client subscribed to remote server
2023/09/04 10:11:36  info [GrpcClient] start handling stream message...
2023/09/04 10:11:36  info worker[85de05b6-4ac3-11ee-b517-0242ac130002] registered to master. id: 64f534b4f7d1447b25892e61

master日志

/go/pkg/mod/github.com/crawlab-team/go-trace@v0.1.1/trace.go:13 github.com/crawlab-team/go-trace.TraceError()                                                                                                                                                                       
/go/pkg/mod/github.com/crawlab-team/crawlab-core@v0.6.3-0.20230804030437-664646cf1665/grpc/server/plugin_server.go:50 github.com/crawlab-team/crawlab-core/grpc/server.PluginServer.Register()                                                                                      
/go/pkg/mod/github.com/crawlab-team/crawlab-grpc@v0.6.0-beta.20211219.1930.0.20221020032435-afa1c691f73c/plugin_service_grpc.pb.go:159 github.com/crawlab-team/crawlab-grpc._PluginService_Register_Handler.func1()
/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.0.1-0.20190118093823-f849b5445de4/chain.go:31 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1()                                                                                                   
/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.0.1-0.20190118093823-f849b5445de4/auth/auth.go:46 github.com/grpc-ecosystem/go-grpc-middleware/auth.UnaryServerInterceptor.func1()                                                                                      
/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.0.1-0.20190118093823-f849b5445de4/chain.go:34 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1()                                                                                                   
/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.0.1-0.20190118093823-f849b5445de4/recovery/interceptors.go:29 github.com/grpc-ecosystem/go-grpc-middleware/recovery.UnaryServerInterceptor.func1()                                                                      
/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.0.1-0.20190118093823-f849b5445de4/chain.go:39 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1()                                                                                                     
/go/pkg/mod/github.com/crawlab-team/crawlab-grpc@v0.6.0-beta.20211219.1930.0.20221020032435-afa1c691f73c/plugin_service_grpc.pb.go:161 github.com/crawlab-team/crawlab-grpc._PluginService_Register_Handler()                                                                       
/go/pkg/mod/google.golang.org/grpc@v1.42.0/server.go:1282 google.golang.org/grpc.(*Server).processUnaryRPC()                              
/go/pkg/mod/google.golang.org/grpc@v1.42.0/server.go:1616 google.golang.org/grpc.(*Server).handleStream()              
/go/pkg/mod/google.golang.org/grpc@v1.42.0/server.go:921 google.golang.org/grpc.(*Server).serveStreams.func1.2()                                                                                                                                                                    
/usr/local/go/src/runtime/asm_amd64.s:1571 runtime.goexit()                                                                               
rpc error: code = Unknown desc = mongo: no documents in result                                                                            
/go/pkg/mod/github.com/crawlab-team/go-trace@v0.1.1/trace.go:13 github.com/crawlab-team/go-trace.TraceError()                          
/go/pkg/mod/github.com/crawlab-team/crawlab-core@v0.6.3-0.20230804030437-664646cf1665/notification/service.go:132 github.com/crawlab-team/crawlab-core/notification.(*Service)._subscribe()                                                                                         
/go/pkg/mod/github.com/cenkalti/backoff/v4@v4.1.0/retry.go:55 github.com/cenkalti/backoff/v4.RetryNotifyWithTimer()                                                                                                                                                                 
/go/pkg/mod/github.com/cenkalti/backoff/v4@v4.1.0/retry.go:34 github.com/cenkalti/backoff/v4.RetryNotify()                                
/go/pkg/mod/github.com/cenkalti/backoff/v4@v4.1.0/retry.go:28 github.com/cenkalti/backoff/v4.Retry()                                   
/go/pkg/mod/github.com/crawlab-team/crawlab-core@v0.6.3-0.20230804030437-664646cf1665/notification/service.go:105 github.com/crawlab-team/crawlab-core/notification.(*Service).subscribe()                                                                                          
/go/pkg/mod/github.com/crawlab-team/crawlab-core@v0.6.3-0.20230804030437-664646cf1665/notification/service.go:64 github.com/crawlab-team/crawlab-core/notification.(*Service).Start()                                                                                               
/usr/local/go/src/runtime/asm_amd64.s:1571 runtime.goexit()                                                                               
[GIN] 2023/09/04 - 10:13:56 | 200 |    2.845004ms |       127.0.0.1 | GET      "/tasks?page=1&size=10&conditions=[]&sort=[]&stats=true"
2023/09/04 10:14:00  info [SpiderAdminService] start to sync git                                                                          
2023/09/04 10:14:00  info [SpiderAdminService] finished sync git                                                                          
[GIN] 2023/09/04 - 10:14:01 | 200 |    2.536813ms |       127.0.0.1 | GET      "/tasks?page=1&size=10&conditions=[]&sort=[]&stats=true"   
[GIN] 2023/09/04 - 10:14:06 | 200 |    2.933918ms |       127.0.0.1 | GET      "/tasks?page=1&size=10&conditions=[]&sort=[]&stats=true"   
[GIN] 2023/09/04 - 10:14:11 | 200 |    3.119327ms |       127.0.0.1 | GET      "/tasks?page=1&size=10&conditions=[]&sort=[]&stats=true"
[GIN] 2023/09/04 - 10:14:16 | 200 |    3.062651ms |       127.0.0.1 | GET      "/tasks?page=1&size=10&conditions=[]&sort=[]&stats=true"                                                                                                                                             
[GIN] 2023/09/04 - 10:14:21 | 200 |    3.170224ms |       127.0.0.1 | GET      "/tasks?page=1&size=10&conditions=[]&sort=[]&stats=true"                                                                                                                                             
[GIN] 2023/09/04 - 10:14:26 | 200 |    2.876372ms |       127.0.0.1 | GET      "/tasks?page=1&size=10&conditions=[]&sort=[]&stats=true"                                                                                                                                             
[GIN] 2023/09/04 - 10:14:31 | 200 |    3.003889ms |       127.0.0.1 | GET      "/tasks?page=1&size=10&conditions=[]&sort=[]&stats=true"                                                                                                                                             
[GIN] 2023/09/04 - 10:14:36 | 200 |    2.926019ms |       127.0.0.1 | GET      "/tasks?page=1&size=10&conditions=[]&sort=[]&stats=true"                                                                                                                                             
[GIN] 2023/09/04 - 10:14:49 | 200 |    3.217441ms |       127.0.0.1 | GET      "/tasks?page=1&size=10&conditions=[]&sort=[]&stats=true" 
MXF130 commented 1 year ago

解决了吗 我一直被这个问题困扰

geosmart commented 1 year ago

master和worker的镜像版本都升级到最新版本就好了