search

twitter / scoot

Scoot is a distributed task runner, supporting both a proprietary API and Bazel's Remote Execution.
Apache License 2.0
349 stars 48 forks source link

setup-cloud-scoot fails to run and error is not clearly given #480

Open jiridanek opened 4 years ago

jiridanek commented 4 years ago 
INFO[0000] Killing 1 cmds                                sourceLine="setup/cmds.go:75"
INFO[0005] Still waiting; killing all                    sourceLine="setup/cmds.go:102"
FATA[0005] fork/exec : no such file or directory         sourceLine="setup-cloud-scoot/main.go:53"
exit status 1

Which file does not exist?

[root@host-10-0-133-169 ~]# tar -C /usr/local -xzf  go1.14.linux-amd64.tar.gz 
e[root@host-10-0-133-169 ~]# export PATH+=:/usr/local/go/bin
[root@host-10-0-133-169 ~]# cd scoot/
[root@host-10-0-133-169 scoot]# go run ./binaries/setup-cloud-scoot/main.go --strategy local.local
go: downloading github.com/apache/thrift v0.0.0-20160616201512-0e9fed1e12ed
go: downloading github.com/sirupsen/logrus v1.4.2
go: downloading github.com/golang/protobuf v1.3.2
go: downloading golang.org/x/sys v0.0.0-20190422165155-953cdadca894
go: downloading google.golang.org/grpc v1.14.0
go: downloading google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8
go: downloading golang.org/x/net v0.0.0-20190628185345-da137c7871d7
go: downloading golang.org/x/time v0.0.0-20190308202827-9d24e82272b4
go: downloading github.com/davecgh/go-spew v1.1.1
go: downloading github.com/cenkalti/backoff v2.2.1+incompatible
go: downloading github.com/twitter/groupcache v0.0.0-20191021194929-8215cbf3be0f
go: downloading github.com/sethgrid/pester v0.0.0-20190127155807-68a33a018ad0
go: downloading github.com/nu7hatch/gouuid v0.0.0-20131221200532-179d4d0c4d8d
go: downloading golang.org/x/text v0.3.0
go: downloading github.com/rcrowley/go-metrics v0.0.0-20160613154715-cfa5a85e9f0a
INFO[0000] Starting up a Local ApiServer                 sourceLine="setup/api.go:49"
INFO[0000] RunCmd:  [-http_addr localhost:11100 -grpc_addr localhost:12100 -log_level info]  sourceLine="setup/cmds.go:122"
INFO[0000] Starting[ -http_addr localhost:11100 -grpc_addr localhost:12100 -log_level info]  sourceLine="setup/cmds.go:142"
INFO[0000] Killing 1 cmds                                sourceLine="setup/cmds.go:75"
INFO[0005] Still waiting; killing all                    sourceLine="setup/cmds.go:102"
FATA[0005] fork/exec : no such file or directory         sourceLine="setup-cloud-scoot/main.go:53"
exit status 1
[root@host-10-0-133-169 scoot]# go run ./binaries/setup-cloud-scoot/main.go --strategy local.local --log_level debug
INFO[0000] Starting up a Local ApiServer                 sourceLine="setup/api.go:49"
INFO[0000] RunCmd:  [-http_addr localhost:11100 -grpc_addr localhost:12100 -log_level debug]  sourceLine="setup/cmds.go:122"
INFO[0000] Starting[ -http_addr localhost:11100 -grpc_addr localhost:12100 -log_level debug]  sourceLine="setup/cmds.go:142"
INFO[0000] Killing 1 cmds                                sourceLine="setup/cmds.go:75"
INFO[0005] Still waiting; killing all                    sourceLine="setup/cmds.go:102"
FATA[0005] fork/exec : no such file or directory         sourceLine="setup-cloud-scoot/main.go:53"
exit status 1
[root@host-10-0-133-169 scoot]# go run ./binaries/setup-cloud-scoot/main.go --strategy local.local -log_level debug
INFO[0000] Starting up a Local ApiServer                 sourceLine="setup/api.go:49"
INFO[0000] RunCmd:  [-http_addr localhost:11100 -grpc_addr localhost:12100 -log_level debug]  sourceLine="setup/cmds.go:122"
INFO[0000] Starting[ -http_addr localhost:11100 -grpc_addr localhost:12100 -log_level debug]  sourceLine="setup/cmds.go:142"
INFO[0000] Killing 1 cmds                                sourceLine="setup/cmds.go:75"
INFO[0005] Still waiting; killing all                    sourceLine="setup/cmds.go:102"
FATA[0005] fork/exec : no such file or directory         sourceLine="setup-cloud-scoot/main.go:53"
exit status 1
[root@host-10-0-133-169 scoot]# go run ./binaries/setup-cloud-scoot/main.go --help
Usage of /tmp/go-build935509414/b001/exe/main:
  -api_strategy string
        api strategy to setup server w/bundlestore. (default "local")
  -apiservers int
        number of apiservers to use (default 3)
  -log_level string
        Log everything at this level and above (error|info|debug) (default "info")
  -repo_dir string
        backing repo to use as a basis for handling bundles
  -strategy string
        scheduler/worker strategy to setup (default "local.memory")
  -workers int
        number of workers to use (default 5)
exit status 2

Are the command line options given with double dashes, or single dash. Instructions in readme show double dashes, here in --help output I see single dash.

jiridanek commented 4 years ago

I found the following in strace -f

[pid 30064] setpgid(0, 0)               = 0
[pid 30064] dup2(6, 0)                  = 0
[pid 30064] fcntl(1, F_SETFD, 0)        = 0
[pid 30064] fcntl(2, F_SETFD, 0)        = 0
[pid 30064] execve("", ["", "-http_addr", "localhost:11100", "-grpc_addr", "localhost:12100", "-log_level", "debug"], [/* 34 vars */]) = -1 ENOENT (No such file or directory)
[pid 30058] <... nanosleep resumed> NULL) = 0
[pid 30064] write(8, "\2\0\0\0\0\0\0\0", 8 <unfinished ...>
[pid 30058] clock_gettime(CLOCK_MONOTONIC,  <unfinished ...>
[pid 30064] <... write resumed> )       = 8
jiridanek commented 4 years ago

Aha, got it. Go defaults GOPATH to $HOME/go when the variable is unset, but scoot requires explicit value there,

export GOPATH=/root/go

fixes my problem, scoot now starts up.

edit: but dies eventually with

INFO[0000] Serving GRPC CAS API on: 127.0.0.1:12100      sourceLine="cas/service.go:81"
INFO[0000] Serving http & stats on: localhost:11100      sourceLine="endpoints/endpoints.go:50"
INFO[0000] repo.Repository.Run, [rev-parse --show-toplevel]  sourceLine="repo/repo.go:63"
INFO[0000] repo.Repository.Run complete. Err: <nil>      sourceLine="repo/repo.go:72"
INFO[0000] git.NewRepository: /tmp/scoot-tmp-615000239/gitdb-repo-141061186, top: /tmp/scoot-tmp-615000239/gitdb-repo-141061186  sourceLine="repo/repo.go:132"
INFO[0010] Killing 1 cmds                                sourceLine="setup/cmds.go:75"
INFO[0010] SIGINT: 31727 /root/go/bin/apiserver          sourceLine="setup/cmds.go:91"
INFO[0010] Cmd 31727 (/root/go/bin/apiserver) finished   sourceLine="setup/cmds.go:149"
INFO[0010] All completed                                 sourceLine="setup/cmds.go:100"
FATA[0010] port 11100 is not up after 5s                 sourceLine="setup-cloud-scoot/main.go:53"
exit status 1

but that is probably another issue.

ryancouto commented 4 years ago

thanks for the report! are you able to run the apiserver binary on its own?

jiridanek commented 4 years ago

Yes. I've reported the second issue, the broken port check, in https://github.com/twitter/scoot/issues/481. Turned out I did not have nc installed.