crawlab-team / crawlab

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台,支持任何语言和框架
https://www.crawlab.cn
BSD 3-Clause "New" or "Revised" License
11.37k stars 1.79k forks source link

backend ./crawlab api 报错 #1046

Closed otll closed 2 years ago

otll commented 2 years ago

@Macbookpro backend % export CRAWLAB_NODE_SERVER=Y @Macbookpro backend % go run main.go master Using config file: /Users/Data/crawlab/backend/conf/config.yml spider error: forbidden /Users/go/pkg/mod/github.com/crawlab-team/go-trace@v0.1.1/trace.go:13 github.com/crawlab-team/go-trace.TraceError() /Users/go/pkg/mod/github.com/crawlab-team/crawlab-core@v0.6.0-beta.20211219.2134/spider/admin/service.go:192 github.com/crawlab-team/crawlab-core/spider/admin.NewSpiderAdminService() /usr/local/go/src/reflect/value.go:543 reflect.Value.call() /usr/local/go/src/reflect/value.go:339 reflect.Value.Call() /Users/go/pkg/mod/go.uber.org/dig@v1.10.0/dig.go:284 go.uber.org/dig.defaultInvoker() /Users/go/pkg/mod/go.uber.org/dig@v1.10.0/dig.go:710 go.uber.org/dig.(node).Call() /Users/go/pkg/mod/go.uber.org/dig@v1.10.0/param.go:245 go.uber.org/dig.paramSingle.Build() /Users/go/pkg/mod/go.uber.org/dig@v1.10.0/param.go:201 go.uber.org/dig.paramList.BuildList() /Users/go/pkg/mod/go.uber.org/dig@v1.10.0/dig.go:432 go.uber.org/dig.(Container).Invoke() /Users/go/pkg/mod/github.com/crawlab-team/crawlab-core@v0.6.0-beta.20211219.2134/controllers/spider.go:1104 github.com/crawlab-team/crawlab-core/controllers.newSpiderContext() /Users/go/pkg/mod/github.com/crawlab-team/crawlab-core@v0.6.0-beta.20211219.2134/controllers/spider.go:38 github.com/crawlab-team/crawlab-core/controllers.getSpiderActions() /Users/go/pkg/mod/github.com/crawlab-team/crawlab-core@v0.6.0-beta.20211219.2134/controllers/spider.go:1125 github.com/crawlab-team/crawlab-core/controllers.newSpiderController() /Users/go/pkg/mod/github.com/crawlab-team/crawlab-core@v0.6.0-beta.20211219.2134/controllers/init.go:16 github.com/crawlab-team/crawlab-core/controllers.InitControllers() /Users/Data/crawlab/backend/apps/utils.go:26 crawlab/apps.initModule() /Users/Data/crawlab/backend/apps/api.go:28 crawlab/apps.(Api).Init() /Users/Data/crawlab/backend/apps/api.go:77 crawlab/apps.NewApi() /Users/Data/crawlab/backend/apps/master.go:60 crawlab/apps.NewMaster() /Users/Data/crawlab/backend/cmd/master.go:23 crawlab/cmd.glob..func3() /Users/go/pkg/mod/github.com/spf13/cobra@v1.1.3/command.go:856 github.com/spf13/cobra.(Command).execute() /Users/go/pkg/mod/github.com/spf13/cobra@v1.1.3/command.go:960 github.com/spf13/cobra.(Command).ExecuteC() /Users/go/pkg/mod/github.com/spf13/cobra@v1.1.3/command.go:897 github.com/spf13/cobra.(Command).Execute() /Users/Data/crawlab/backend/cmd/root.go:27 crawlab/cmd.Execute() /Users/Data/crawlab/backend/main.go:8 main.main() /usr/local/go/src/runtime/proc.go:255 runtime.main() /usr/local/go/src/runtime/asm_arm64.s:1133 runtime.goexit() panic: could not build arguments for function "github.com/crawlab-team/crawlab-core/controllers".newSpiderContext.func1 (/Users/go/pkg/mod/github.com/crawlab-team/crawlab-core@v0.6.0-beta.20211219.2134/controllers/spider.go:1104): failed to build interfaces.SpiderAdminService: received non-nil error from function "github.com/crawlab-team/crawlab-core/spider/admin".NewSpiderAdminService (/Users/go/pkg/mod/github.com/crawlab-team/crawlab-core@v0.6.0-beta.20211219.2134/spider/admin/service.go:161): spider error: forbidden

goroutine 1 [running]: github.com/crawlab-team/crawlab-core/controllers.newSpiderContext() /Users/go/pkg/mod/github.com/crawlab-team/crawlab-core@v0.6.0-beta.20211219.2134/controllers/spider.go:1113 +0x23c github.com/crawlab-team/crawlab-core/controllers.getSpiderActions() /Users/go/pkg/mod/github.com/crawlab-team/crawlab-core@v0.6.0-beta.20211219.2134/controllers/spider.go:38 +0x20 github.com/crawlab-team/crawlab-core/controllers.newSpiderController() /Users/go/pkg/mod/github.com/crawlab-team/crawlab-core@v0.6.0-beta.20211219.2134/controllers/spider.go:1125 +0x20 github.com/crawlab-team/crawlab-core/controllers.InitControllers() /Users/go/pkg/mod/github.com/crawlab-team/crawlab-core@v0.6.0-beta.20211219.2134/controllers/init.go:16 +0xd8 crawlab/apps.initModule({0x104abbdd5, 0xb}, 0x104f1d968) /Users/Data/crawlab/backend/apps/utils.go:26 +0x28 crawlab/apps.(Api).Init(0x1400011a5c0) /Users/Data/crawlab/backend/apps/api.go:28 +0x40 crawlab/apps.NewApi() /Users/Data/crawlab/backend/apps/api.go:77 +0x84 crawlab/apps.NewMaster({0x0, 0x0, 0x0}) /Users/Data/crawlab/backend/apps/master.go:60 +0x34 crawlab/cmd.glob..func3(0x105687d60, {0x1056c9498, 0x0, 0x0}) /Users/Data/crawlab/backend/cmd/master.go:23 +0x28 github.com/spf13/cobra.(Command).execute(0x105687d60, {0x1056c9498, 0x0, 0x0}) /Users/go/pkg/mod/github.com/spf13/cobra@v1.1.3/command.go:856 +0x640 github.com/spf13/cobra.(Command).ExecuteC(0x105687fe0) /Users/go/pkg/mod/github.com/spf13/cobra@v1.1.3/command.go:960 +0x404 github.com/spf13/cobra.(Command).Execute(...) /Users/go/pkg/mod/github.com/spf13/cobra@v1.1.3/command.go:897 crawlab/cmd.Execute(...) /Users/Data/crawlab/backend/cmd/root.go:27 main.main() /Users/Data/crawlab/backend/main.go:8 +0x30 exit status 2

tikazyq commented 2 years ago
export CRAWLAB_NODE_SERVER=Y
go run main.go master
tikazyq commented 2 years ago

duplicated in https://github.com/crawlab-team/crawlab/issues/1039

stevecheen commented 2 years ago

try: linux: export CRAWLAR_NODE_MASTER=Y window $env:CRAWLAR_NODE_MASTER="Y"