search

faucetsdn / faucetagent

gNMI agent for faucet configuration
4 stars 2 forks source link

no `gnmi_set` found when running 'sudo make test' #1

Closed cglewis closed 5 years ago

cglewis commented 5 years ago 
*** Starting controller
c0 Timeout waiting for FAUCET to start. Log:
Traceback (most recent call last):
  File "/usr/local/bin/faucet", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.5/dist-packages/faucet/__main__.py", line 166, in main
    os.execvp(ryu_args[0], ryu_args)
  File "/usr/lib/python3.5/os.py", line 615, in execvp
    _execvpe(file, args)
  File "/usr/lib/python3.5/os.py", line 660, in _execvpe
    raise last_exc.with_traceback(tb)
  File "/usr/lib/python3.5/os.py", line 650, in _execvpe
    exec_func(fullname, *argrest)
FileNotFoundError: [Errno 2] No such file or directory

*** Starting 2 switches
s1 s2 ...
* Shutting down any agents listening on 10161
* Starting agent
* Waiting for agent to start up
10161/tcp:          
* Sending test configuration to agent
Traceback (most recent call last):
  File "./agenttest.py", line 399, in <module>
    main()
  File "./agenttest.py", line 394, in main
    exit_code = end_to_end_test()
  File "./agenttest.py", line 349, in end_to_end_test
    result = run(cmd, stdout=PIPE, check=True)
  File "/usr/lib/python3.5/subprocess.py", line 693, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.5/subprocess.py", line 947, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.5/subprocess.py", line 1551, in _execute_child
    raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: 'gnmi_set'
makefile:46: recipe for target 'test' failed
make: *** [test] Error 1
lantz commented 5 years ago

Probably GOPATH isn't set correctly in your environment. It should work in travis.

We might want to change how it is set in makefile.

cglewis commented 5 years ago

I did this test on a system I didn't already have golang on, and this is the GOPATH it set for me:

GOPATH=/home/clewis/go PATH=/home/clewis/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin ./agenttest.py

Which is in fact where the code has been downloaded to:

~/go$ tree -d -L 3
.
└── src
    ├── github.com
    │   ├── golang
    │   ├── google
    │   ├── kylelemons
    │   └── openconfig
    ├── golang.org
    │   └── x
    └── google.golang.org
        ├── genproto
        └── grpc

11 directories
~/go$ pwd
/home/clewis/go

Should I be setting something else?

lantz commented 5 years ago

OK, that looks good I think (and I also like the tree command.)

In the current agent version, both makefile and dependencies.sh should use the GOPATH that you have set, or $HOME/go if it isn't set.

The travis test is simply running:

- ./dependencies.sh
- ./test-dependencies.sh

script:
- make
- make codecheck
- sudo -E make test

Note: the version in PR #5 depends on bleeding-edge FAUCET so it adds:

sudo pip3 -q install git+https://github.com/faucetsdn/faucet

AFAIK those are all of the dependencies.

Now that I look at it, maybe the go/gnxi/faucet dependencies should be moved into test-dependencies.sh as technically they are only required to run the end-to-end test. Or maybe everything should just go into dependencies.

Can you confirm whether the above approach works as expected?

cglewis commented 5 years ago

I pulled in the latest and ran ./dependencies.sh and noticed this time it did this:

* Installing gnxi tools
package github.com/google/gnxi/gnmi_capabilities: cannot download, $GOPATH not set. For more details see: go help gopath
can't load package: package github.com/google/gnxi/gnmi_capabilities: cannot find package "github.com/google/gnxi/gnmi_capabilities" in any of:
    /usr/lib/go-1.6/src/github.com/google/gnxi/gnmi_capabilities (from $GOROOT)
    ($GOPATH not set)
package github.com/google/gnxi/gnmi_get: cannot download, $GOPATH not set. For more details see: go help gopath
can't load package: package github.com/google/gnxi/gnmi_get: cannot find package "github.com/google/gnxi/gnmi_get" in any of:
    /usr/lib/go-1.6/src/github.com/google/gnxi/gnmi_get (from $GOROOT)
    ($GOPATH not set)
package github.com/google/gnxi/gnmi_set: cannot download, $GOPATH not set. For more details see: go help gopath
can't load package: package github.com/google/gnxi/gnmi_set: cannot find package "github.com/google/gnxi/gnmi_set" in any of:
    /usr/lib/go-1.6/src/github.com/google/gnxi/gnmi_set (from $GOROOT)
    ($GOPATH not set)
package github.com/google/gnxi/gnmi_target: cannot download, $GOPATH not set. For more details see: go help gopath
can't load package: package github.com/google/gnxi/gnmi_target: cannot find package "github.com/google/gnxi/gnmi_target" in any of:
    /usr/lib/go-1.6/src/github.com/google/gnxi/gnmi_target (from $GOROOT)
    ($GOPATH not set)
* Installing FAUCET as root
* Done

It probably did that last time too, I just missed it. I added export in front of the GOPATH variable in the beginning of the script and it fixed the GOPATH issue, but resulted in this:

* Installing gnxi tools
package context: unrecognized import path "context" (import path does not begin with hostname)
package net/http/httptrace: unrecognized import path "net/http/httptrace" (import path does not begin with hostname)
../../../go/src/google.golang.org/grpc/balancer.go:22:2: cannot find package "context" in any of:
    /usr/lib/go-1.6/src/context (from $GOROOT)
    /home/clewis/go/src/context (from $GOPATH)
../../../go/src/golang.org/x/net/http2/not_go111.go:10:2: cannot find package "net/http/httptrace" in any of:
    /usr/lib/go-1.6/src/net/http/httptrace (from $GOROOT)
    /home/clewis/go/src/net/http/httptrace (from $GOPATH)
package context: unrecognized import path "context" (import path does not begin with hostname)
package net/http/httptrace: unrecognized import path "net/http/httptrace" (import path does not begin with hostname)
../../../go/src/google.golang.org/grpc/balancer.go:22:2: cannot find package "context" in any of:
    /usr/lib/go-1.6/src/context (from $GOROOT)
    /home/clewis/go/src/context (from $GOPATH)
../../../go/src/golang.org/x/net/http2/not_go111.go:10:2: cannot find package "net/http/httptrace" in any of:
    /usr/lib/go-1.6/src/net/http/httptrace (from $GOROOT)
    /home/clewis/go/src/net/http/httptrace (from $GOPATH)
package context: unrecognized import path "context" (import path does not begin with hostname)
package net/http/httptrace: unrecognized import path "net/http/httptrace" (import path does not begin with hostname)
../../../go/src/google.golang.org/grpc/balancer.go:22:2: cannot find package "context" in any of:
    /usr/lib/go-1.6/src/context (from $GOROOT)
    /home/clewis/go/src/context (from $GOPATH)
../../../go/src/golang.org/x/net/http2/not_go111.go:10:2: cannot find package "net/http/httptrace" in any of:
    /usr/lib/go-1.6/src/net/http/httptrace (from $GOROOT)
    /home/clewis/go/src/net/http/httptrace (from $GOPATH)
package context: unrecognized import path "context" (import path does not begin with hostname)
package net/http/httptrace: unrecognized import path "net/http/httptrace" (import path does not begin with hostname)
../../../go/src/google.golang.org/grpc/balancer.go:22:2: cannot find package "context" in any of:
    /usr/lib/go-1.6/src/context (from $GOROOT)
    /home/clewis/go/src/context (from $GOPATH)
../../../go/src/golang.org/x/net/http2/not_go111.go:10:2: cannot find package "net/http/httptrace" in any of:
    /usr/lib/go-1.6/src/net/http/httptrace (from $GOROOT)
    /home/clewis/go/src/net/http/httptrace (from $GOPATH)
* Installing FAUCET as root
* Done

For whatever reason it's angry about that context thing.

lantz commented 5 years ago

Last I checked, dependencies.sh actually sets GOPATH so it is odd that it isn't set.

GOPATH=${GOPATH:=$HOME/go}

You can try running it with bash -x to see why that line is being skipped and also which commands aren't working.

The missing package error could be because you seem to be running golang 1.6, but gnxi requires golang 1.7+.

Also it occurs to me that sudo (without -E) clears its environment by default so you might need

sudo GOPATH=$GOPATH make test

The problems you have run into make me think that maybe we should simply fail if GOPATH isn't set.

cglewis commented 5 years ago

Hmm, interesting, I'm running Ubuntu 16.04 and it installed golang with apt get via the dependencies.sh script, so 1.6 must be the latest packaged release for 16.04.

lantz commented 5 years ago

Ah, maybe my script installed the distro version, which in this case is too old. But then how did it work on travis? I have no idea. I am running Ubuntu 18 and go version reports 

go version go1.10.4 linux/amd64
cglewis commented 5 years ago

I think travis pre-equips their instances with a bunch of stuff like python and golang so they run faster.

lantz commented 5 years ago

Hmm. I am thinking we might want to

lantz commented 5 years ago

I think possibly removing your GOPATH directory and reinstalling might work.

lantz commented 5 years ago

I tested against go 1.6 in Ubuntu 16.04 and ran into the same context error. Then I installed go version 1.12 and the problem no longer occurred So I think checking for go version >= go 1.7 is correct.

cglewis commented 5 years ago

I updated to golang 1.12 and it fixed the gnxi/context issues. There was however a few other things that cropped up.

  1. had to run ./test-dependencies with sudo for two of the pip installs 
    
$ ./test-dependencies.sh 
* Installing testing (make {test,codecheck}) dependencies
Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python3.5/dist-packages/pycodestyle.py'
Consider using the `--user` option or check the permissions.

/tmp/tmp.5q4u4jOzga ~/github/faucetagent Cloning into 'mininet'... remote: Enumerating objects: 28, done. remote: Counting objects: 100% (28/28), done. remote: Compressing objects: 100% (21/21), done. remote: Total 9645 (delta 9), reused 23 (delta 6), pack-reused 9617 Receiving objects: 100% (9645/9645), 3.00 MiB | 5.62 MiB/s, done. Resolving deltas: 100% (6393/6393), done. Checking connectivity... done. Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python3.5/dist-packages/mininet-2.3.0d5.dist-info' Consider using the --user option or check the permissions.


2. while the test got a lot farther this time, it still ultimately failed for me, and I'm not sure why:

$ sudo make test GOPATH=/home/clewis/go PATH=/home/clewis/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin ./agenttest.py

*** Starting 2 switches s1 s2 ...

lantz commented 5 years ago

I created PR #6 to address this issue. The problem you are running into now seems to be that FAUCET isn't starting. I have changed test-dependencies.sh to install the latest and greatest FAUCET so that might help.

lantz commented 5 years ago

This is astonishingly painful - of course it works fine in travis, but it would be nice to work for normal users also. ;-p

cglewis commented 5 years ago

Yeah, sorry for all of the back and forth. I also figured if it was passing in Travis it wouldn't be that hard to try out locally.

lantz commented 5 years ago

One kind of terrible thing is that mininet has to run as root (for now at least) which means that it can get a completely different python environment from the non-root environment.

We'd like to be able to run make codecheck as non-root, but then it may fail if the non-root python environment doesn't have mininet installed in it...

My current approach is to install the test python dependencies as root.

cglewis commented 5 years ago

Tracked down why Faucet wasn't starting. I had set the permissions of the logs files wrong (I had been previously doing Faucet stuff on this box). Once I fixed the permissions on the log file, the tests completed successfully!

lantz commented 5 years ago

Aha, progress at least!!

I'm not sure why that would cause the test to fail actually since it sudo make test runs faucet as root and logs to stdout.

However, if you are happy with the current state of #6, then maybe we can merge it and then close this issue.