search

SatelliteQE / robottelo

Robottelo is a test suite that exercises The Foreman.
GNU General Public License v3.0
61 stars 114 forks source link

Automation failure : test_positive_content_override #2148

Closed tkolhar closed 9 years ago

tkolhar commented 9 years ago

this test is failing for snap2 GA TestNames: tests.foreman.cli.test_activationkey.TestActivationKey.test_positive_content_override

this test is failing becoz : self.fail('BZ 1180282 has been closed. This test should be ' AssertionError: BZ 1180282 has been closed. This test should be updated to verify value change via product-content.

the BZ associated with the test is moved to ON_QA:

the test has details: if not bz_bug_is_open(1180282): self.fail('BZ 1180282 has been closed. This test should be ' 'updated to verify value change via product-content.') There need to be updation on above lines in the code for it to work properly

please fix it thanks

JacobCallahan commented 9 years ago

The bug associated with this just failed QA. Waiting until I can verify.

tkolhar commented 9 years ago

results of the test when executed locally using nosetests $ nosetests -c robottelo.properties -m test_positive_content_override /home/tkolhar/git/robottelo/tests/foreman/cli/test_activationkey.py /usr/lib/python2.7/site-packages/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin xunitmp = nose_xunitmp:XunitMP: cannot import name force_unicode RuntimeWarning) @Test: Positive content override ... 2015-05-12 16:36:39 - robottelo - DEBUG - Running test TestActivationKey/test_positive_content_override_1_1 ERROR

ERROR: @Test: Positive content override Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ddt.py", line 146, in wrapper return func(self, _args, _kwargs) File "/home/tkolhar/git/robottelo/tests/foreman/cli/test_activationkey.py", line 1731, in test_positive_content_override product_label = result.stdout[3].split('|')[5].replace(' ', '') IndexError: list index out of range ERROR: @Test: Positive content override Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ddt.py", line 146, in wrapper return func(self, _args, _kwargs) File "/home/tkolhar/git/robottelo/tests/foreman/cli/test_activationkey.py", line 1731, in test_positive_content_override product_label = result.stdout[3].split('|')[5].replace(' ', '') IndexError: list index out of range XML: foreman-results.xml Ran 2 tests in 186.425s

product_label = result.stdout[3].split('|')[5].replace(' ', '') it seems we have wrong index (going out of range)

elyezer commented 9 years ago

@tkolhar I have run the entire module and don't get that error. For more info check https://github.com/SatelliteQE/robottelo/issues/2095#issuecomment-101060332.

elyezer commented 9 years ago

In the latest run this test got the following:

Error Message

string index out of range

robottelo.common.ssh: DEBUG: >>> [qe-sat6-rhel66.usersys.redhat.com] LANG=en_US.UTF-8 hammer -v -u admin -p changeme  activation-key product-content --id="41" --organization-id="216"
robottelo.common.ssh: INFO: Instantiated Paramiko client 0x4169e90
robottelo.common.ssh: INFO: Destroying Paramiko client 0x4169e90
robottelo.common.ssh: INFO: Destroyed Paramiko client 0x4169e90
robottelo.common.ssh: DEBUG: <<< [ERROR 2015-05-14 03:47:26 Exception] Error: Request Timeout
Error: Request Timeout
[ERROR 2015-05-14 03:47:26 Exception] 

RestClient::RequestTimeout (Request Timeout):
    /usr/lib/ruby/gems/1.8/gems/rest-client-1.6.7/lib/restclient/request.rb:184:in `transmit'
    /usr/lib/ruby/gems/1.8/gems/rest-client-1.6.7/lib/restclient/request.rb:64:in `execute'
    /usr/lib/ruby/gems/1.8/gems/rest-client-1.6.7/lib/restclient/request.rb:33:in `execute'
    /usr/lib/ruby/gems/1.8/gems/rest-client-1.6.7/lib/restclient/resource.rb:54:in `get'
    /usr/lib/ruby/gems/1.8/gems/apipie-bindings-0.0.11/lib/apipie_bindings/api.rb:279:in `send'
    /usr/lib/ruby/gems/1.8/gems/apipie-bindings-0.0.11/lib/apipie_bindings/api.rb:279:in `call_client'
    /usr/lib/ruby/gems/1.8/gems/apipie-bindings-0.0.11/lib/apipie_bindings/api.rb:210:in `http_call'
    /usr/lib/ruby/gems/1.8/gems/apipie-bindings-0.0.11/lib/apipie_bindings/api.rb:161:in `call'
    /usr/lib/ruby/gems/1.8/gems/apipie-bindings-0.0.11/lib/apipie_bindings/resource.rb:14:in `call'
    /usr/lib/ruby/gems/1.8/gems/hammer_cli-0.1.4.10/lib/hammer_cli/./apipie/command.rb:43:in `send_request'
    /usr/lib/ruby/gems/1.8/gems/hammer_cli_foreman-0.1.4.11/lib/hammer_cli_foreman/commands.rb:185:in `send_request'
    /usr/lib/ruby/gems/1.8/gems/hammer_cli_foreman-0.1.4.11/lib/hammer_cli_foreman/commands.rb:270:in `send_request'
    /usr/lib/ruby/gems/1.8/gems/hammer_cli_foreman-0.1.4.11/lib/hammer_cli_foreman/commands.rb:316:in `retrieve_and_print'
    /usr/lib/ruby/gems/1.8/gems/hammer_cli_foreman-0.1.4.11/lib/hammer_cli_foreman/commands.rb:290:in `execute'
    /usr/lib/ruby/gems/1.8/gems/clamp-0.6.2/lib/clamp/command.rb:67:in `run'
    /usr/lib/ruby/gems/1.8/gems/hammer_cli-0.1.4.10/lib/hammer_cli/./apipie/../abstract.rb:23:in `run'
    /usr/lib/ruby/gems/1.8/gems/clamp-0.6.2/lib/clamp/subcommand/execution.rb:11:in `execute'
    /usr/lib/ruby/gems/1.8/gems/clamp-0.6.2/lib/clamp/command.rb:67:in `run'
    /usr/lib/ruby/gems/1.8/gems/hammer_cli-0.1.4.10/lib/hammer_cli/./apipie/../abstract.rb:23:in `run'
    /usr/lib/ruby/gems/1.8/gems/clamp-0.6.2/lib/clamp/subcommand/execution.rb:11:in `execute'
    /usr/lib/ruby/gems/1.8/gems/clamp-0.6.2/lib/clamp/command.rb:67:in `run'
    /usr/lib/ruby/gems/1.8/gems/hammer_cli-0.1.4.10/lib/hammer_cli/./apipie/../abstract.rb:23:in `run'
    /usr/lib/ruby/gems/1.8/gems/clamp-0.6.2/lib/clamp/command.rb:125:in `run'
    /usr/lib/ruby/gems/1.8/gems/hammer_cli-0.1.4.10/bin/hammer:108
    /usr/bin/hammer:19:in `load'
    /usr/bin/hammer:19
elyezer commented 9 years ago

@JacobCallahan I'm taking this bug and will close it by skipping the test because BZ https://bugzilla.redhat.com/show_bug.cgi?id=1221778. Once the original BZ gets fixed then we will get another automation failure and a new issue for it.

tkolhar commented 9 years ago

this issue occurred again dated 06/17/2015

tests.foreman.cli.test_activationkey.TestActivationKey.test_positive_content_override_1_1 Error Message 70 != 0

robottelo.common.ssh: DEBUG: >>> [qe-sat6-rhel71.usersys.redhat.com] LANG=en_US.UTF-8 hammer -v -u admin -p changeme activation-key product-content --id="42" --organization-id="219" robottelo.common.ssh: INFO: Instantiated Paramiko client 0x3144cd0 robottelo.common.ssh: INFO: Destroying Paramiko client 0x3144cd0 robottelo.common.ssh: INFO: Destroyed Paramiko client 0x3144cd0 robottelo.common.ssh: DEBUG: <<< stderr [ERROR 2015-06-17 04:48:05 Exception] Error: Request Timeout Error: Request Timeout [ERROR 2015-06-17 04:48:05 Exception]

RestClient::RequestTimeout (Request Timeout): /usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:184:in rescue in transmit' /usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:140:intransmit' /usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:64:in execute' /usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:33:inexecute' /usr/share/gems/gems/rest-client-1.6.7/lib/restclient/resource.rb:51:in get' /usr/share/gems/gems/apipie-bindings-0.0.11/lib/apipie_bindings/api.rb:279:incall_client' /usr/share/gems/gems/apipie-bindings-0.0.11/lib/apipie_bindings/api.rb:210:in http_call' /usr/share/gems/gems/apipie-bindings-0.0.11/lib/apipie_bindings/api.rb:160:incall' /usr/share/gems/gems/apipie-bindings-0.0.11/lib/apipie_bindings/resource.rb:14:in call' /usr/share/gems/gems/hammer_cli-0.1.4.11/lib/hammer_cli/apipie/command.rb:43:insend_request' /usr/share/gems/gems/hammer_cli_foreman-0.1.4.14/lib/hammer_cli_foreman/commands.rb:185:in send_request' /usr/share/gems/gems/hammer_cli_foreman-0.1.4.14/lib/hammer_cli_foreman/commands.rb:271:insend_request' /usr/share/gems/gems/hammer_cli_foreman-0.1.4.14/lib/hammer_cli_foreman/commands.rb:291:in execute' /usr/share/gems/gems/clamp-0.6.2/lib/clamp/command.rb:67:inrun' /usr/share/gems/gems/hammer_cli-0.1.4.11/lib/hammer_cli/abstract.rb:23:in run' /usr/share/gems/gems/clamp-0.6.2/lib/clamp/subcommand/execution.rb:11:inexecute' /usr/share/gems/gems/clamp-0.6.2/lib/clamp/command.rb:67:in run' /usr/share/gems/gems/hammer_cli-0.1.4.11/lib/hammer_cli/abstract.rb:23:inrun' /usr/share/gems/gems/clamp-0.6.2/lib/clamp/subcommand/execution.rb:11:in execute' /usr/share/gems/gems/clamp-0.6.2/lib/clamp/command.rb:67:inrun' /usr/share/gems/gems/hammer_cli-0.1.4.11/lib/hammer_cli/abstract.rb:23:in run' /usr/share/gems/gems/clamp-0.6.2/lib/clamp/command.rb:125:inrun' /usr/share/gems/gems/hammer_cli-0.1.4.11/bin/hammer:108:in <top (required)>' /usr/bin/hammer:23:inload' /usr/bin/hammer:23:in `

'

Stacktrace

File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/home/jenkins/shiningpanda/jobs/107b379a/virtualenvs/d41d8cd9/lib/python2.7/site-packages/ddt.py", line 146, in wrapper return func(self, _args, _kwargs) File "robottelo/common/decorators.py", line 375, in wrapper_func return func(_args, _kwargs) File "/home/jenkins/workspace/satellite6-automation-downstream-rhel71-cli/tests/foreman/cli/test_activationkey.py", line 1736, in test_positive_content_override self.assertEqual(result.return_code, 0) File "/usr/lib64/python2.7/unittest/case.py", line 553, in assertEqual assertion_func(first, second, msg=msg) File "/usr/lib64/python2.7/unittest/case.py", line 546, in _baseAssertEqual raise self.failureException(msg) '70 != 0\n--------------------

tkolhar commented 9 years ago

this issue occurred again dated 06/23/2015 TestNames: tests.foreman.cli.test_activationkey.TestActivationKey.test_positive_content_override_1_0

Error Message 70 != 0

robottelo.common.ssh: DEBUG: >>> [qe-sat6-rhel66.usersys.redhat.com] LANG=en_US.UTF-8 hammer -v -u admin -p changeme activation-key product-content --id="42" --organization-id="241" robottelo.common.ssh: INFO: Instantiated Paramiko client 0x35f6610 robottelo.common.ssh: INFO: Destroying Paramiko client 0x35f6610 robottelo.common.ssh: INFO: Destroyed Paramiko client 0x35f6610 robottelo.common.ssh: DEBUG: <<< stderr [ERROR 2015-06-22 10:17:31 Exception] Error: Request Timeout Error: Request Timeout [ERROR 2015-06-22 10:17:31 Exception]

RestClient::RequestTimeout (Request Timeout): /usr/lib/ruby/gems/1.8/gems/rest-client-1.6.7/lib/restclient/request.rb:184:in transmit' /usr/lib/ruby/gems/1.8/gems/rest-client-1.6.7/lib/restclient/request.rb:64:inexecute' /usr/lib/ruby/gems/1.8/gems/rest-client-1.6.7/lib/restclient/request.rb:33:in execute' /usr/lib/ruby/gems/1.8/gems/rest-client-1.6.7/lib/restclient/resource.rb:54:inget' /usr/lib/ruby/gems/1.8/gems/apipie-bindings-0.0.11/lib/apipie_bindings/api.rb:279:in send' /usr/lib/ruby/gems/1.8/gems/apipie-bindings-0.0.11/lib/apipie_bindings/api.rb:279:incall_client' /usr/lib/ruby/gems/1.8/gems/apipie-bindings-0.0.11/lib/apipie_bindings/api.rb:210:in http_call' /usr/lib/ruby/gems/1.8/gems/apipie-bindings-0.0.11/lib/apipie_bindings/api.rb:161:incall' /usr/lib/ruby/gems/1.8/gems/apipie-bindings-0.0.11/lib/apipie_bindings/resource.rb:14:in call' /usr/lib/ruby/gems/1.8/gems/hammer_cli-0.1.4.11/lib/hammer_cli/./apipie/command.rb:43:insend_request' /usr/lib/ruby/gems/1.8/gems/hammer_cli_foreman-0.1.4.14/lib/hammer_cli_foreman/commands.rb:185:in send_request' /usr/lib/ruby/gems/1.8/gems/hammer_cli_foreman-0.1.4.14/lib/hammer_cli_foreman/commands.rb:271:insend_request' /usr/lib/ruby/gems/1.8/gems/hammer_cli_foreman-0.1.4.14/lib/hammer_cli_foreman/commands.rb:291:in execute' /usr/lib/ruby/gems/1.8/gems/clamp-0.6.2/lib/clamp/command.rb:67:inrun' /usr/lib/ruby/gems/1.8/gems/hammer_cli-0.1.4.11/lib/hammer_cli/./apipie/../abstract.rb:23:in run' /usr/lib/ruby/gems/1.8/gems/clamp-0.6.2/lib/clamp/subcommand/execution.rb:11:inexecute' /usr/lib/ruby/gems/1.8/gems/clamp-0.6.2/lib/clamp/command.rb:67:in run' /usr/lib/ruby/gems/1.8/gems/hammer_cli-0.1.4.11/lib/hammer_cli/./apipie/../abstract.rb:23:inrun' /usr/lib/ruby/gems/1.8/gems/clamp-0.6.2/lib/clamp/subcommand/execution.rb:11:in execute' /usr/lib/ruby/gems/1.8/gems/clamp-0.6.2/lib/clamp/command.rb:67:inrun' /usr/lib/ruby/gems/1.8/gems/hammer_cli-0.1.4.11/lib/hammer_cli/./apipie/../abstract.rb:23:in run' /usr/lib/ruby/gems/1.8/gems/clamp-0.6.2/lib/clamp/command.rb:125:inrun' /usr/lib/ruby/gems/1.8/gems/hammer_cli-0.1.4.11/bin/hammer:108 /usr/bin/hammer:19:in `load' /usr/bin/hammer:19

Stacktrace

File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/home/jenkins/shiningpanda/jobs/9514c559/virtualenvs/d41d8cd9/lib/python2.7/site-packages/ddt.py", line 146, in wrapper return func(self, _args, _kwargs) File "robottelo/common/decorators.py", line 375, in wrapper_func return func(_args, _kwargs) File "/home/jenkins/workspace/satellite6-automation-downstream-rhel66-cli/tests/foreman/cli/test_activationkey.py", line 1736, in test_positive_content_override self.assertEqual(result.return_code, 0) File "/usr/lib64/python2.7/unittest/case.py", line 553, in assertEqual assertion_func(first, second, msg=msg) File "/usr/lib64/python2.7/unittest/case.py", line 546, in _baseAssertEqual raise self.failureException(msg) '70 != 0\n--------------------

tkolhar commented 9 years ago

this test is failing when executed locally using nosetests: $ nosetests -c robottelo.properties -m test_positive_content_override /home/tkolhar/git/robottelo/tests/foreman/cli/test_activationkey.py 2015-07-29 14:54:03 - nailgun.client - DEBUG - Making HTTP GET request to https://qe-sat6-rhel66.usersys.redhat.com/foreman_tasks/api/tasks/db2c0ebc-1cb9-482d-beda-3ef3579007cf with options {'verify': False, 'auth': ('admin', 'changeme'), 'headers': {'content-type': 'application/json'}} and no data. 2015-07-29 14:54:04 - py.warnings - WARNING - /usr/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning. InsecurePlatformWarning

2015-07-29 14:54:06 - nailgun.client - DEBUG - Received HTTP 200 response: {"id":"db2c0ebc-1cb9-482d-beda-3ef3579007cf","label":"Actions::Katello::Provider::ManifestImport","pending":false,"username":"admin","started_at":"2015-07-29T09:24:03Z","ended_at":"2015-07-29T09:24:05Z","state":"stopped","result":"warning","progress":1,"input":{"provider":{"id":744,"name":"Red Hat"},"organization":{"id":474,"name":"6yrKn3","label":"6yrKn3"},"path":"/usr/share/foreman/tmp/import_f9a94a9bdb439623d1e3.zip","force":null,"remote_user":"admin-5c728f59","remote_cp_user":"admin","locale":"en"},"output":{},"humanized":{"action":"Import Manifest","input":[["organization",{"text":"organization '6yrKn3'","link":"/organizations/474/edit"}]],"output":"","errors":["Runtime Error Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1 at org.hibernate.jdbc.Expectations$BasicExpectation.checkBatched:81"]},"cli_example":null} ERROR

ERROR: @Test: Positive content override Traceback (most recent call last): File "/home/tkolhar/git/robottelo/tests/foreman/cli/test_activationkey.py", line 44, in setUp TestActivationKey.org = make_org(cached=True) File "/home/tkolhar/git/robottelo/robottelo/common/decorators.py", line 73, in cacheable_function new_object = func(options) File "/home/tkolhar/git/robottelo/robottelo/cli/factory.py", line 1267, in make_org return create_object(Org, args, options) File "/home/tkolhar/git/robottelo/robottelo/cli/factory.py", line 94, in create_object json.dumps(options, indent=2, sort_keys=True) CLIFactoryError: Failed to create Org with data: { "compute-resource-ids": null, "compute-resources": null, "config-template-ids": null, "config-templates": null, "description": null, "domain-ids": null, "environment-ids": null, "environments": null, "hostgroup-ids": null, "hostgroups": null, "label": null, "media": null, "media-ids": null, "name": "l2vfpV", "realm-ids": null, "realms": null, "smart-proxies": null, "smart-proxy-ids": null, "subnet-ids": null, "subnets": null, "user-ids": null, "users": null } ERROR: @Test: Positive content override Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ddt.py", line 146, in wrapper return func(self, _args, _kwargs) File "/home/tkolhar/git/robottelo/robottelo/common/decorators.py", line 372, in wrapper_func return func(_args, _kwargs) File "/home/tkolhar/git/robottelo/tests/foreman/cli/test_activationkey.py", line 1731, in test_positive_content_override self._make_public_key() File "/home/tkolhar/git/robottelo/tests/foreman/cli/test_activationkey.py", line 89, in _make_public_key ).upload_manifest(TestActivationKey.pub_key['manifest']) File "/usr/lib/python2.7/site-packages/nailgun/entities.py", line 2118, in upload_manifest return _handle_response(response, self._server_config, synchronous) File "/usr/lib/python2.7/site-packages/nailgun/entities.py", line 104, in _handle_response return ForemanTask(server_config, id=response.json()['id']).poll() File "/usr/lib/python2.7/site-packages/nailgun/entities.py", line 1285, in poll timeout File "/usr/lib/python2.7/site-packages/nailgun/entity_mixins.py", line 108, in _poll_task task_info['humanized']['errors'] TaskFailedError: Task db2c0ebc-1cb9-482d-beda-3ef3579007cf completed with result warning. Error message(s): [u'Runtime Error Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1 at org.hibernate.jdbc.Expectations$BasicExpectation.checkBatched:81']

XML: foreman-results.xml Ran 2 tests in 132.939s

FAILED (errors=2)

JacobCallahan commented 9 years ago

@tkolhar this test is passing locally for me and in our automation runs. Looking at your error, it seems it failed during manifest import; not something inherently related to this test. Is this the only test that fails for you?