Could the frame support data provider function?

[IVANOPT]

Could the frame support data provider, in java, we use data provider annotation to prepare test data.

[svanoort]

@IVANOPT Could you explain a bit more what you mean and how you would use it? Thanks!

[IVANOPT]

I mean, always we run api tests use multiple variable group, most of them are same, for example:

curl -i "127.0.0.1:8080/group?a=1&b=1"
curl -i "127.0.0.1:8080/group?a=1&b=2"
curl -i "127.0.0.1:8080/group?a=1&b=3"

how do we provider different data each time only instead of write the whole test module. In java, we use @DataProvider annotation, you may find more introductions from the link: http://www.mkyong.com/unittest/testng-parameter-testing-example/

In addition, how to check result size greater than 0, which type is list. If there are some operations to validate the list type accurate and how ?

Thanks.

[svanoort]

@IVANOPT PyRestTest supports several ways to achieve what you need here, in order from simplest to most complex:

• Use the "vars" command line argument to supply runtime variables for the tests, and a templated request body/url/headers
• Use environment variables, and the env_variable or env_string generators to store environment variables to a string, then use that string in a template
• Use a custom generator extension to create data from some source and use that data in templates (see: extensions docs )
• There is a proposal in flight to provide an extension point for pluggable request body generation (see https://github.com/svanoort/pyresttest/issues/115 )

The advanced guide provides information on how to use these.

Checking for a list with result size >1 (validators element):

- validators: 
   - compare: {jsonpath_mini: 'my.value', comparator: 'type', expected: 'list' } 
   - extract_test: {jsonpath_mini: 'my.value.0', test: 'exists' }  

The extract_test there is asserting that the 1st element in the list exists, so it'll check for >= 1 element.

I haven't run the above, so there may be minor typos, but it will do what you want.

[IVANOPT]

Thanks, helps me a lot. I have some other questions in usage.

• As I know, we can run a pyresttest file through the following command: pyresttest http://127.0.0.1:9040 example.yaml

if I just want to execute one of the tests of example.yaml, for example, example.yaml includes testA and testB, how to run testA only?

• Another is about variable compare. I defined variable_binds in config, such as the following codes which show config and compare modules, I wanted to compare two lists:

  - config
  - variable_binds: { default_city: ["Atlanta"] }
  
  ......
  
  - compare: {jsonpath_mini: 'data.city', comparator: 'str_eq', expected: {template: $default_city}}
  

  Then I called api to test the result, I got error messages below:

ERROR:Test Failure, failure type: Validator Failed, Reason: Comparison failed, evaluating str_eq([u'Atlanta'], ['Atlanta']) returned False ERROR:Validator/Error details:Extractor: Extractor Type: jsonpath_mini, Query: "data.city", Templated?: False

Obvious that by calling the api call, I got the result with 'u' character...how to compare the above two lists? If the process right?

Thanks for help more.

[svanoort]

if I just want to execute one of the tests of example.yaml, for example, example.yaml includes testA and testB, how to run testA only?

Not currently supported, the solution is to break them into separate files and use 'import' statements in a parent testset if you want to sometimes run them as a unit. It will become a feature if I can make a convenient unittest wrapper for these though (tied to the large xUnit compatible output feature in issues).

I wanted to compare two lists:

Not quite: you're comparing a python object (extracted from JSON) to a raw string (templates all evaluate to strings).

What you probably want to do do is:

- compare: {jsonpath_mini: 'data.city.0', expected: 'Atlanta'}

OR if you want to use the variables, you can use a regex (hacky but it works):

- compare: {jsonpath_mini: 'data.city.0', comparator:'regex', expected: {template: '.*$default_city.*'} }

[svanoort]

@IVANOPT ^ I hope the above answers your question? I've got some thoughts about a better way to work with variables in tests, captured in https://github.com/svanoort/pyresttest/issues/101 - the catch is that it could be a long way off at the moment (given how much time I have to spend on pyresttest).

[IVANOPT]

Solved the problems, really hacky, I'll think more about the solutions to make it more convenient, thanks.

[svanoort]

@IVANOPT Glad it was helpful. I've got plans to improve this, but it'll take some time to implement (and to do it in a way that doesn't break things for existing users).

Since it sounds like your questions have been been answered (and existing enhancement proposals cover your needs), I'm going to go ahead and close out the issue for now. But I would be happy to assist in other issues if you have further questions (or reopen the one if there are further questions on the same subject).

Cheers! Sam