search

znsio / specmatic

Turn your contracts into executable specifications. Contract Driven Development - Collaboratively Design & Independently Deploy MicroServices & MicroFrontends.
https://specmatic.io
MIT License
281 stars 52 forks source link

Secure schema not respected when generating test scenarios #932

Open mariusz-mc opened 10 months ago

mariusz-mc commented 10 months ago

Description According to the spec and the Swagger documentation

That is, security is an array of hashmaps, where each hashmap contains one or more named security schemes. Items in a hashmap are combined using logical AND, and array items are combined using logical OR.

For instances

security:
  - apiKey1: []
    apiKey2: []

Should generate only one test with two required headers, while specmatic generates two separate tests for each headers which is wrong. Specmatic behaviour is correct only for the following security configuration

security:
  - apiKey1: []
  - apiKey2: []

Steps to reproduce I wrote an example test

Expected behavior Create one test scenario when logical AND used

Screenshots N/A

System Information: N/A

haydenbanz commented 10 months ago

To address the issue of a secure schema not being respected when generating test scenarios, consider the following solutions:

  1. Validation Mechanism: Implement a robust validation mechanism in your test scenario generation process to ensure that generated data adheres to the secure schema. This can include checks for data types, length constraints, and any other security-related criteria.

  2. Enhanced Test Data Generation Tools: Explore or develop tools that provide more control over test data generation. Look for or create solutions that allow you to specify security constraints and generate data accordingly.

  3. Customized Data Generation Scripts: Write custom scripts for generating test data, taking into account the specific security requirements of your schema. This gives you fine-grained control over the data generation process.

  4. Collaboration with Developers: Collaborate closely with developers to align the test data generation process with the secure schema. This ensures that both testing and development efforts are in sync regarding security considerations.

  5. Regular Schema Audits: Conduct regular audits of the secure schema to identify any inconsistencies or changes. Update your test data generation process accordingly to reflect the most current security requirements.

joelrosario commented 10 months ago

@mariusz-mc Thanks for the bug report. We'll take a look.