cdent / gabbi

Declarative HTTP Testing for Python and anything else
http://gabbi.readthedocs.org/
Other
148 stars 34 forks source link

test files with - in the name can lead to failing tests when looking for content-type #278

Open cdent opened 4 years ago

cdent commented 4 years ago

Bear with me, this is hard to explain

Python v 3.6.9

gabbi: 1.49.0

A test file with named device-types.yaml with a test of:

tests:                                                                          
- name: get only 405                                                            
  POST: /device-types                                                           
  status: 405    

errors with the following when run in a unittest-style harness:

    b'Traceback (most recent call last):'
    b'  File "/home/cdent/.uhana/lib/python3.6/site-packages/gabbi/handlers/core.py", line 68, in action'
    b'    response_value = str(response[header])'
    b'  File "/home/cdent/.uhana/lib/python3.6/site-packages/urllib3/_collections.py", line 156, in __getitem__'
    b'    val = self._container[key.lower()]'
    b"KeyError: 'content-type'"
    b''
    b'During handling of the above exception, another exception occurred:'
    b''
    b'Traceback (most recent call last):'
    b'  File "/home/cdent/.uhana/lib/python3.6/site-packages/gabbi/suitemaker.py", line 96, in do_test'
    b'    return test_method(*args, **kwargs)'
    b'  File "/home/cdent/.uhana/lib/python3.6/site-packages/gabbi/case.py", line 95, in wrapper'
    b'    func(self)'
    b'  File "/home/cdent/.uhana/lib/python3.6/site-packages/gabbi/case.py", line 149, in test_request'
    b'    self._run_test()'
    b'  File "/home/cdent/.uhana/lib/python3.6/site-packages/gabbi/case.py", line 556, in _run_test'
    b'    self._assert_response()'
    b'  File "/home/cdent/.uhana/lib/python3.6/site-packages/gabbi/case.py", line 196, in _assert_response'
    b'    handler(self)'
    b'  File "/home/cdent/.uhana/lib/python3.6/site-packages/gabbi/handlers/base.py", line 54, in __call__'
    b'    self.action(test, item, value=value)'
    b'  File "/home/cdent/.uhana/lib/python3.6/site-packages/gabbi/handlers/core.py", line 72, in action'
    b'    header, response.keys()))'
    b"AssertionError: 'content-type' header not present in response: KeysView(HTTPHeaderDict({'Vary': 'Origin', 'Date': 'Tue, 24 Mar 2020 14:17:33 GMT', 'Content-Length': '0', 'status': '405', 'reason': 'Method Not Allowed'}))"
    b''

However, rename the file to foo.yaml and the test works, or run the device-types.yaml file with gabbi-run and the tests work. Presumably something about test naming.

So the short term workaround is to rename the file, but this needs to be fixed because using - in filenames is idiomatic for gabbi.

FND commented 4 years ago

FWIW, I can't easily reproduce this in tests:

  1. I created gabbi/tests/gabbits_live/device-types.yaml

    fixtures:
      - LiveSkipFixture
    
    defaults:
        ssl: True
    
    tests:
    - name: google
      POST: /
      status: 405
  2. tox -epy37 -- test_live is all green

  3. in order to get rid of the Content-Type response header, I adjusted the YAML file to point to a custom WSGI server:

    #!/usr/bin/env python3
    
    from time import sleep
    
    def handler(environ, start_response):
        start_response("200 OK", [("Server", "WSGI Dummy")])
        return (txt.encode("utf-8") for txt in generate_chunks())
    
    def generate_chunks():
        yield "abc"
        sleep(1)
        yield "123"
    
    if __name__ == "__main__":
        from wsgiref.simple_server import make_server
    
        host = "localhost"
        port = 8080
    
        srv = make_server(host, port, handler)
        print("→ http://%s:%s" % (host, port))
        srv.serve_forever()
  4. GOTO 2; same result

AFAICT gabbits_live is run through gabbi/tests/test_live.py, which does appear to use unittest. 🤷‍♂