Endpoint with multiple tags produces duplicate sequences

[nikitastupin]

Hi team,

First of all thanks for the great tool! 😃 Currently I'm configuring and testing it against Apache ServiceComb Service Center. I've noticed that if endpoint have two or more tags then this endpoint will be duplicated by fuzz-lightyear thus producing duplicate sequences! This greatly increases running time if we have a lot of endpoints and big request sequence length.

For example, endpoint addRule has two tags microservices and rules. While generating request sequences this will produce two requests microservices.addRule and rules.addRule. However both requests will point to the same endpoint /v4/{project}/registry/microservices/{serviceId}/rules!

Steps to reproduce

1. Download files.zip which contains two files .pdbrc and v4.yaml.
2. Place .pdbrc in your home directory, change directory to fuzz-lightyear repo and run python3 -m pdb -m fuzz_lightyear file:///full/path/to/v4.yaml

In output you should see something like this:

$ python3 -m pdb -m fuzz_lightyear file:///home/nikita/fuzz-lightyear-service-center/v4.yaml
Breakpoint 1 at /home/nikita/fuzz-lightyear/fuzz_lightyear/main.py:147
Breakpoint 2 at /home/nikita/fuzz-lightyear/fuzz_lightyear/main.py:147
================================================================= fuzzing session starts =================================================================
Hypothesis Seed: 101724005987071576949872594882807307139

warning: No auth method specified.
admin EEEE
base EE
dependencies EE
governance EEEE
instances EEEEEEEEEE
microservices EEDeleted breakpoint 1 at /home/nikita/fuzz-lightyear/fuzz_lightyear/main.py:147
display result.requests[0].id: 'microservices.addRule'
display result.requests[0].json(): {'method': 'POST', 'path': '/v4/d/registry/microservices/0/rules', 'body': {'rules': {'rules': [{'attribute': '', 'description': '', 'pattern': '', 'ruleType': ''}]}}}
EEEEEEEEEEEEEEEEEEEEDeleted breakpoint 2 at /home/nikita/fuzz-lightyear/fuzz_lightyear/main.py:147
display result.requests[0].id: 'rules.addRule'
display result.requests[0].json(): {'method': 'POST', 'path': '/v4/0/registry/microservices/0/rules', 'body': {'rules': {'rules': [{'attribute': '', 'description': '', 'pattern': '', 'ruleType': ''}]}}}
> /home/nikita/fuzz-lightyear/fuzz_lightyear/main.py(147)run_tests()
-> outputter.record_result(result)
(Pdb)

There are two interesting parts:

display result.requests[0].id: 'microservices.addRule'
display result.requests[0].json(): {'method': 'POST', 'path': '/v4/d/registry/microservices/0/rules', 'body': {'rules': {'rules': [{'attribute': '', 'description': '', 'pattern': '', 'ruleType': ''}]}}}

display result.requests[0].id: 'rules.addRule'
display result.requests[0].json(): {'method': 'POST', 'path': '/v4/0/registry/microservices/0/rules', 'body': {'rules': {'rules': [{'attribute': '', 'description': '', 'pattern': '', 'ruleType': ''}]}}}

As we can see, we have two different requests (we can identify it by id) that have the same endpoint under the hood (we can identify it by json())!

Expected results

Each endpoint should correspond to one request 😃

[domanchi]

Thanks for bringing this up! I would not be surprised that this happens -- however, it's currently unclear how this should be handled. I wonder whether endpoints should be deduplicated on sequence generation, but indexed by tags so that users can still test a single tag with fuzz-lightyear -t microservices.

[nikitastupin]

I think we can generate requests based solely on method + path combination. Tags can be added to request object later for later filtration.