Closed shyba closed 8 years ago
From @cz8s on January 26, 2016 14:40
possible problems:
From @cz8s on January 26, 2016 15:0
if we run the tests, we need valid credentials. leap already has a way to create temporary users. Thats why it would be good to integrate our tests into leap. See https://github.com/leapcode/leap_platform/blob/develop/tests/helpers/bonafide_helper.rb#L7
From @cz8s on February 8, 2016 10:54
the tests are run by check_mk now. To qa, look at nagios. there should be a a Test called 'Job pixelated' with a history.
From @mnandri on February 11, 2016 16:0
I have tried on staging:
Notify @cz8s
From @cz8s on February 16, 2016 9:24
we have to run check_mk -II
and check_mk -R
before the test shows up in nagios.
@varac: do you know a good way to do this?
From @cz8s on February 16, 2016 9:25
the tests fail because the new dispatcher now send a content-security header which breaks phantomjs
From @cz8s on February 16, 2016 10:50
maybe we should tell the user-agent to not send the csp header if the client sent a header indicating
that it doesn't want to see it? like X-NO-CSP-PLEASE=true
From @cz8s on February 22, 2016 10:46
we are running the tests in our pipeline directly on the server now. that works and its the same way nagios runs them later
From @cz8s on February 23, 2016 9:6
all jobs are registered with nagios on deploy. we need to run a fake job on deploy to make sure the job is registered with nagios. when the cronjob then runs the job, it will already be configured in nagios and the result can be processed.
From @varac on February 25, 2016 15:34
Using the curl/sh onliner from https://github.com/pixelated/puppet-pixelated#testing-pixelated, i get this error:
- [node1.example.org] Error: /Stage[main]/Pixelated::Tests/Exec[dummy_register_job]: Failed to call refresh: /usr/bin/mk-job: line 65: /usr/bin/time: No such file or directory
- [node1.example.org] Error: /Stage[main]/Pixelated::Tests/Exec[dummy_register_job]: /usr/bin/mk-job: line 65: /usr/bin/time: No such file or directory
From @varac on February 25, 2016 15:37
also, bzip2 is missing:
- [node1.example.org] Notice: /Stage[main]/Pixelated::Tests/Exec[install_phantomjs]/returns: tar (grandchild): bzip2: Cannot exec: No such file or directory
- [node1.example.org] Notice: /Stage[main]/Pixelated::Tests/Exec[install_phantomjs]/returns: tar (grandchild): Error is not recoverable: exiting now
- [node1.example.org] Notice: /Stage[main]/Pixelated::Tests/Exec[install_phantomjs]/returns: % Total % Received % Xferd Average Speed Time Time Time Current
- [node1.example.org] Notice: /Stage[main]/Pixelated::Tests/Exec[install_phantomjs]/returns: Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0antomjs]/returns:
0 22.3M 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:-- 0tar: Child died with signal 13
- [node1.example.org] Notice: /Stage[main]/Pixelated::Tests/Exec[install_phantomjs]/returns: tar: Error is not recoverable: exiting now
- [node1.example.org] Notice: /Stage[main]/Pixelated::Tests/Exec[install_phantomjs]/returns: curl: (23) Failed writing body (3511 != 16384)
- [node1.example.org] Error: /usr/bin/curl -L https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-2.1.1-linux-x86_64.tar.bz2 | tar xvj --strip-components=2 -C /usr/local/bin/ phantomjs-2.1.1-linux-x86_64/bin/phantomjs returned 2 instead of one of [0]
From @cz8s on February 25, 2016 15:43
why does this only happen if we install on vagrant?
From @varac on February 25, 2016 17:10
because the vagrant baseimage is stripped down to the bare minimums, and bzip2 is not installed. This is at least the explanation for the second issue (which i already fixed).
From @varac on February 25, 2016 17:21
the first issue about the missing /usr/bin/time
binary is an check-mk upstream bug, it depends on the time
package. I fixed this in leap_platform.
current status: the tests ca not be run be nagios directly because they take too long. we now create a cron job that runs them daily and reports to nagios via mk-job The tests are red at the moment because of bug in sqlcipher. @fbernitt and me are looking into the bug and will see if the tests become green on the way
Finally. Tests are green in go and on staging. The way it works:
to qa: login to nagios on staging and see that the Job pixelated-functional-tests
is green.
If you want to run the tests locally, you have to shut down the firewall on the target system
(e.g. staging): shorewall clear
Then you should be able to run the tests:
puppet-pixelated/files/functional-tests ᐅ TESTHOST=staging.pixelated-project.org behave --tags ~@wip --tags @staging
Be very happy :)
i deployed to dev.pix but the tests fails with No module named enum
root@dev1:~# /usr/local/bin/behave --tags @staging --tags ~@wip --no-capture -k /srv/leap/tests_custom/functional-tests/
Traceback (most recent call last):
File "/usr/local/bin/behave", line 7, in <module>
from behave.__main__ import main
File "/usr/local/lib/python2.7/dist-packages/behave/__init__.py", line 21, in <module>
from behave.matchers import use_step_matcher, step_matcher, register_type
File "/usr/local/lib/python2.7/dist-packages/behave/matchers.py", line 5, in <module>
from parse_type import cfparse
File "/usr/local/lib/python2.7/dist-packages/parse_type/__init__.py", line 3, in <module>
from parse_type.cardinality import Cardinality
File "/usr/local/lib/python2.7/dist-packages/parse_type/cardinality.py", line 8, in <module>
from enum import Enum
ImportError: No module named enum
apt-get install python-enum
fixed it.
From @cz8s on January 26, 2016 14:22
we already have some functional tests that run after every build. But they are not in a very good shape and it would be nice to have tests that can be run regularly to make sure everything still works.
Definition of done
leap test
tries to login with phantomjs and checks if the inbox is visible.Copied from original issue: pixelated/pixelated-platform#155