taverntesting / tavern

A command-line tool and Python library and Pytest plugin for automated testing of RESTful APIs, with a simple, concise and flexible YAML-based syntax
https://taverntesting.github.io/
MIT License
1.02k stars 192 forks source link

Question about AssertionError causing by parametrized value type #936

Closed WinstonLoser closed 2 months ago

WinstonLoser commented 2 months ago

I tried the basic test, test_minimal.tavern.yaml that mentioned in README with parameterize and notice that the values in parametrize are changed into string type which cause AssertionError:

---
marks:
  - parametrize:
      key:
        - id
        - userId
      vals:
        - [1,1]
......
    response:
      status_code: 200
      json:
        id: "{id}"
        userId: "{userId}"
ERROR    tavern.response:response.py:42 Type of returned data was different than expected (expected["userId"] = '1' (type = <class 'tavern._core.formatted_str.FormattedString'>), actual["userId"] = '1' (type = <class 'int'>))   
Traceback (most recent call last):
  File "D:\work\pylearning\tavern\.venv\lib\site-packages\tavern\_core\dict_util.py", line 444, in check_keys_match_recursive
    assert actual_val == expected_val  # noqa
AssertionError: assert {'body': 'qui..., 'userId': 1} == {'body': 'qui...'userId': '1'}
  Omitting 2 identical items, use -vv to show
  Differing items:
  {'userId': 1} != {'userId': '1'}
  {'id': 1} != {'id': '1'}
  Use -v to get more diff

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\work\pylearning\tavern\.venv\lib\site-packages\tavern\_core\dict_util.py", line 444, in check_keys_match_recursive
    assert actual_val == expected_val  # noqa
AssertionError: assert 1 == '1'

So does it only support for string type key? And I find that status_code is not supported parametrize, does it?

michaelboulton commented 2 months ago

If you want to use parametrize with a integer then it will implicitly be converted into a string, if you don't want that you need to explicitly convert it to an integer using a type token (https://tavern.readthedocs.io/en/latest/basics.html#type-conversions) like

    response:
      status_code: 200
      json:
        id: !int "{id}"
        userId: !int "{userId}"

Using parametrize with status codes isn't supported, but you can have a list of expected status codes like: https://github.com/taverntesting/tavern/blob/4396844435a74d0af4c06ca64eb1c15cf02b61ab/tests/integration/test_status_codes.tavern.yaml#L24-L26 If you want to check that an endpoint gives a different status code for different inputs then its best to just split it into 2 separate tests.