svanoort / pyresttest

Python Rest Testing
Apache License 2.0
1.15k stars 325 forks source link

How to use generators of type 'fixed_sequence'? #264

Open kiptoomm opened 6 years ago

kiptoomm commented 6 years ago

Does anyone have a working example for how to use the 'fixed_sequence' type of generators? In https://github.com/svanoort/pyresttest/blob/master/advanced_guide.md, I can only see an example for the 'number_sequence' type.

I have defined my config like the following:

- config:
    - testset: "Test CRUD Operations"
    - timeout: 100  # Increase timeout from the default 10 seconds
    - headers: {'Content-Type': 'application/vnd.api+json'}
    - variable_binds: {firstname: 'first', lastname: 'last'}
    - generators:
      - 'id': {type: 'number_sequence', start: 2}
      - 'random_name': {type: 'random_text'}
      - 'item_counts': {type: 'fixed_sequence', values: [4, 3, 1, 2, 5]}

And now I'm trying to create the buyer entries like so:

- benchmark: # create new buyer entries
  - generator_binds: {buyer_id: id, items: item_counts}
  - name: "create buyers"
  - url: {template: "/api/buyer"}
  - warmup_runs: 0
  - method: 'POST'
  - headers: {'Content-Type': 'application/vnd.api+json'}
  - body: {template:
            '{
              "data": {
                "type": "buyer",
                "attributes": {
                  "id": "$buyer_id",
                  "first_name": "$firstname$buyer_id",
                  "last_name": "$lastname$buyer_id",
                  "items_bought": "$items"
                }
              }
            }'
          }
  - benchmark_runs: '5' # create 5 buyers
  - metrics:
    - total_time: total
    - total_time: mean

Problem is that my server seems to only receive the first item of the 'items' array, so all values of the 'items_bought' column are set to 4, when it should loop through the entire array and populate each row accordingly. What am I missing?

kiptoomm commented 6 years ago

@svanoort I find that using the generator type 'choice' (outputs a random value from an input list) works OK. I am not sure why 'fixed_sequence' isn't behaving as expected...

EtheriousNatsu commented 6 years ago

@kiptoomm because fixed_sequence, every time choice the same value of index is 0。

kiptoomm commented 6 years ago

@EtheriousNatsu, I'm not sure I completely understand your answer. Do you have a working example of how to use fixed_sequence? Or how should I modify my example so that it works correctly as desired?

nitzanav commented 6 years ago

@kiptoomm I looked into the code, it is jsut a bug, this feature was never used by anyone. https://github.com/svanoort/pyresttest/blob/master/pyresttest/generators.py#L100 instead of:

def factory_fixed_sequence(values):
    """ Return a generator that runs through a list of values in order, looping after end """

    def seq_generator():
        my_list = list(values)
        i = 0
        while(True):
            yield my_list[i]
            if i == len(my_list):
                i = 0
    return seq_generator

It should be:

def factory_fixed_sequence(values):
    """ Return a generator that runs through a list of values in order, looping after end """

    def seq_generator():
        my_list = list(values)
        i = 0
        while(True):
            yield my_list[i]
            i += 1
            if i == len(my_list):
                i = 0
    return seq_generator

The i += 1 is missing

nitzanav commented 6 years ago

If you make progress here tell me, I am looking for another feature, I wish to have fixed_sequence but in order to loop over test suites, not for benchmarking, I need validations...

nitzanav commented 6 years ago

I am willing to pay for development hours for developing a PT of "foreach: sequence" that will repeat a test for each element in the array.

It will need to get array from a json response in test1 and loop on test2 for element in that array. it will need to add the variable name and value to the test name