Issue with Auto-Generated Karate Feature Files: Schema Validation and Field Generation

[Mohitkrishna12]

Hi @ivangsa , Auto-generated Karate feature files often include optional fields with fuzzy matching, despite being marked as required in the schema.

{
  "id": "##number",
  "name": "##string",
  "status": "##string"
}

This issue revolves around the utilization of the buildKarateSchema function found within the test-data-generator.ts file.

    const required = options.requiredProps && options.requiredProps.includes(options.name);
    const nullable = schema.nullable === true || (Array.isArray(schema.type) && schema.type.includes("'null'"));
    const isOptionalPrefix = !required || nullable ? '#' : '';

In the above code required is always false because in the function call options parameter is empty object '{}'. Please review this issue. Correct me if I'm mistaken. Additionally, could you confirm whether it's possible to implement max length, min length, and regex pattern validations based on the schema of the spec file during the generation process?. I'm also open to contributing to address this matter if such assistance is needed.

[ivangsa]

hi @Mohitkrishna12

yes, you are right about the options object being empty... Would you like to contribute a PR? It would be a matter of populating an array or required properties from the operation schema (arround here https://github.com/ZenWave360/karate-ide/blob/master/src/generators/openapi/test-data-generator.ts#L19)

You can also explore adding pattern, min, max-length validations on a separate PR, but for asserting all OpenAPI validation you can use a java class like this one and get all the validations at once https://github.com/ZenWave360/karate-openapi-petstore/blob/master/src/test/java/com/petstore/karate/OpenApiValidator.java#L49

I'm happy with your contributions, any path you find better..

[Mohitkrishna12]

Absolutely, I'm open to contributing and creating a pull request to address this issue. Please provide any necessary guidance or steps to get started. Looking forward to collaborating on a solution!

[ivangsa]

Currently there are not unit-test but since test-data-generator.ts does not depend on vscode it could be easy to add a test for this.

Any way to start the extension in debug mode you need two steps:

• Run: npm run watch and keep it open
• On vscode press F5 or from Run&Debug run Run Extension. This will open a new vscode instance with the extension in dev mode, you can open any workspace on this instance and use the extension while you can place breakpoints on the vscode instance holding your source code

For this issue you just need to build an array of "operation response required fields" you can get from operation parameter (which is the openapi operation) and pass it inside the options variable you pointed out is empty...

If you have any question just let me know..

[Mohitkrishna12]

Hello @ivangsa ,

I am planning to submit a PR shortly and I would like to confirm whether the matchResponse attribute is set to true by default only for 2xx status codes. In the test.feature.ejs file https://github.com/ZenWave360/karate-ide/blob/master/src/generators/openapi/templates/test.feature.ejs#L21, I noticed that it is currently hardcoded as true. Should I address this issue in the same PR?

[ivangsa]

yes you are right, matchResponse should be set to true only for 2xx because you don't usually need to match the response payload against the schema for codes like 400,404...

So you mean generate something like this:

Yes I think this is very useful.. go ahead!

[Mohitkrishna12]

Hi @ivangsa , I've submitted the Pull Request to address the discussed issues. Your feedback is much appreciated. Thanks!

[ivangsa]

Your PR has been merged and published to vscode marketplace: https://marketplace.visualstudio.com/items?itemName=KarateIDE.karate-ide

Thanks a lot!! If you find the extension useful maybe you can leave a review or rate it in the marketplace..