search

deis / workflow-e2e

End-to-end tests for Deis Workflow
MIT License
12 stars 18 forks source link

deis tls flaky test #322

Closed vdice closed 7 years ago

vdice commented 8 years ago

(I'm honestly not too familiar with the deis tls command so not sure if this is a flaky test or actual functionality bug. We can reassign if necessary.)

Deis Workflow.deis tls with an existing user who owns an existing app that has already been deployed can enable/disable tls has failed for the last 9 out of 14 e2e runs (as of writing)

Sample job: https://ci.deis.io/job/workflow-test-pr/6464/ Logs: https://ci.deis.io/job/workflow-test-pr/6464/artifact/6464/logs/

Output:

$ deis auth:register http://deis.k8s.local --username=test-247107041 --password=asdf1234 --email=test-247107041@deis.io
14:17:45 Registered test-247107041
14:17:45 Logged in as test-247107041
14:17:45 Configuration file written to /tmp/deis-workflow-home157223086/.deis/test-247107041.json
14:17:45 $ deis apps:create test-283337570 --no-remote
14:17:45 Creating Application... ...done, created test-283337570
14:17:45 If you want to add a git remote for this app later, use `deis git:remote -a test-283337570`
14:17:45 $ deis builds:create --app=test-283337570 deis/example-dockerfile-http
14:17:45 Creating build... ...o...o...o...o...o...o...o...o...o...o...o...o...o...o...o...o...o...o...o...o...o...o...o...o...o...o...o...o...o...o...o...o...o...o...o...o...done
14:17:45 $ deis tls:enable --app=test-283337570
14:17:45 Enabling https-only requests for test-283337570... ...done
14:17:45 Waiting up to 60 seconds for `curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null` to return 301...
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 200
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 200
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 200
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 200
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 200
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 200
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 200
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 200
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 200
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 200
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ deis tls:disable --app=test-283337570
14:17:45 Disabling https-only requests for test-283337570... ...done
14:17:45 Waiting up to 60 seconds for `curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null` to return 200...
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 $ curl -sL -w "%{http_code}\\n" "http://test-283337570.k8s.local" -o /dev/null
14:17:45 301
14:17:45 FAIL: '301
14:17:45 ' does not match expected result of '200'
14:17:45 $ deis apps:destroy --app=test-283337570 --confirm=test-283337570
14:17:45 Destroying test-283337570...
14:17:45 done in 0s
14:17:45 $ deis auth:cancel --username=test-247107041 --password=asdf1234 --yes
14:17:45 Please log in again in order to cancel this account
14:17:45 Logged in as test-247107041
14:17:45 Configuration file written to /tmp/deis-workflow-home157223086/.deis/test-247107041.json
14:17:45 Account cancelled
14:17:45 
14:17:45 
14:17:45 • Failure [162.986 seconds]
14:17:45 deis tls
14:17:45 /go/src/github.com/deis/workflow-e2e/tests/tls_test.go:69
14:17:45   with an existing user
14:17:45   /go/src/github.com/deis/workflow-e2e/tests/tls_test.go:68
14:17:45     who owns an existing app that has already been deployed
14:17:45     /go/src/github.com/deis/workflow-e2e/tests/tls_test.go:67
14:17:45       can enable/disable tls [It]
14:17:45       /go/src/github.com/deis/workflow-e2e/vendor/github.com/onsi/ginkgo/ginkgo_dsl.go:357
14:17:45 
14:17:45       Timed out after 60.000s.
14:17:45       Expected
14:17:45           <bool>: false
14:17:45       to be true
14:17:45 
14:17:45       /go/src/github.com/deis/workflow-e2e/tests/tls_test.go:65
14:17:45 ------------------------------
vdice commented 8 years ago

ping @bacongobbler

bacongobbler commented 8 years ago

let's skip the test for now and make sure we manually QA deis tls:disable to make sure it's actually working for the release, then come back to this for 2.6. Sounds fair?