Test Case Visual Editor

[cjol]

Currently, test cases are defined as YAML and imported into the platform database. In future, we may have less technical users who are not comfortable editing yaml.

An interface should be created to allow creation of new test cases.

Requirements

• Where applicable, drop-down menus should be used to ensure that foreign keys are correctly entered (e.g. for the use_case, components, behaviour, test_steps[*].{method|source|target|api_spec} fields).
• Where possible, freeform JSON entry should be avoided, though it is acceptable in highly variable locations such as a request body definition.
• Existing test cases can be opened in the visual editor to allow them to be inspected and edited. Saving changes to a test case in this way should use the test-case editing procedure defined in #281.
• To save on development time, it would be acceptable (but it is not necessary) to define a JSON schema for the YAML structure, and use something like react-jsonschema-form to dynamically generate the interface (I don't know of an equivalent in the Vue world, but I assume something will exist).
• It should still be possible to import test-cases via YAML. It is not necessary to export testcases to YAML, but if it's easy then that would be a nice-to-have.

[brunosmm]

About the form generator, on Vue world I know 2: https://github.com/vue-generators/vue-form-generator (this one is based on bootstrap, the base of tabler) https://github.com/wotamann/vuetify-form-base (this one is linked with Vuetify design, which should not fit here since we use https://tabler.io)

[agalaguz]

Hi team, As UI part of the Editable Test Cases to be started next sprint, I started adding some wireframes so the expectations and our understaing of task can be a bit aligned. Can you please review the v1 wireframes here?

For now I see that, based, on #281 versioning rules, admins that uses UI to edit TC will create a new version once edited a single step or global TC data - this is from one side. From the other side I didn't want to push all those entries to a single long form. I will deeply appreciate you comments and suggestions. Thanks.

[cjol]

Hi Anton, I've added a few comments directly on the PDF, but mostly I think this is looking like the right approach. One thing I have thought of is that if an admin wants to edit a test case and make two changes, it would be nice to only create one new version, instead of creating a new version for every single incremental change. Perhaps the best solution would be to have a "draft" state to the new version when it is first created, and then to "commit" the draft when the user is happy with it. The workflow would be something like this:

1. User clicks the "edit test case" button.
2. In the database, the test case is cloned into a new "version" but it is marked in "draft" mode.
3. The user is shown the editing UI, and their edits are directly reflected in the record for the draft TC.
4. In the list view, the "draft" and the "original" test cases are both shown separately. In the draft row, there is a button to discard the draft, or a button to publish the draft.
5. the discard button would delete the new version of the test case, and the database would be in the same state as before step 1.
6. The publish button would remove the draft status from the new test case record. In the list view, the old version should no longer be displayed, because there is a newer non-draft version.

I don't know if this is the most elegant solution, so please let me know if you have better ideas! Also if you think this is overcomplicating things, then it's not the end of the world to have a new version for every new change.

[agalaguz]

Hi Christopher, @cjol I've updated the wireframes to include your comments (hope I didn't miss anything important). Could you please take a look at the new version here? Sure once we started, some changes can be applied due technical reasons, but this wireframe is just to show an approach.

[cjol]

Hi Anton, I've added comments there, and also on the Compliance session which I guess is more useful for #290

[agalaguz]

Thank you, checking it with the team.

[agalaguz]

Hi @brunosmm @cjol, I've added a few changes to the concept of Test Case and Test Step create/edit views (only these 2 page templates were updated). Please could you please have a look at a newer version here?

Key areas that were affected:

TC page: description and precondition are to use https://tiptap.dev/ as a plain text editor TS page: trigger, test_request_scripts, test_response_scripts, test_request_setups, test_response_setups, and response are to use https://jinkin1995.github.io/vue-json-edit/ as a JSON editor

I'm still to find out the way to make information links for users where they can find info about regex and how scripts, setups, and examples should be used.

[brunosmm]

Hey @agalaguz , Seems all good from my side. I added some comments on the layout, small points and suggestions. I like the 2 libs :)

[cjol]

Hi Anton, thanks for this. In principle this looks okay, although I have a concern about the JSON editor because it's not going to play nicely with templating for the "no simulator" use case. For example, if I want to output the following template, the JSON editor will prevent me:

{
   amount: {
      amount: {{ test_steps[0].request.body.amount.amount }}
  }
}

For now I think it would be acceptable to just use a textarea. Depending on how permissive vue-json-edit is, we could add a button to toggle the visual editor - but assuming that it will choke in invalid json, the button will need to be disabled in some circumstances. Furthermore, I anticipate that most templates will probably be invalid json in this way, so it might not be worth the effort to add the visual editor at all!

[agalaguz]

Hi Christopher @cjol,

Thank you for your comment.

We are to make proper grooming and define how to handle templating for the "no simulator" by the end of this sprint (it would be easier to implement it once #303, #309, and #310). For now, it seems that JSON editor may still work if we change templating syntax a bit, e.g.

A) add variables as a string that won't break json format:

{
  "amount":{
    "amount":"{{test_steps[0].request.body.amount.amount}}"
  }
}   

B) or add them as an object that includes special marker like type == "variable":

{
  "amount":{
    "amount":{
      "type":"variable",
      "value":"test_steps[0].request.body.amount.amount"
    }
  }
}

Seems like we'll need to write some parser to get such variables from YAML/test case edit UI and store them in the database -> to be later withdrawn during Test Step executions. Won't you mind if we keep JSON viewer for now? (If our suggestions/ideas above make small sense - could you please let us know? maybe we are missing something)

---

One more point re Status dropdown please - should we have it limited like on wireframes or show the full number of statuses like here https://httpstatuses.com/ ?

[cjol]

I think using a string as in approach A is a mistake because we lose type information. Our environment variables are all strings, but what if I want to insert the env var MY_FEE_AMOUNT=5 as a number into the JSON?

B could be functional, but I think it's going to be too much work to handle dynamic variables. The examples above are just for retrieving static variables, but we need the ability to do more advanced stuff like adding variables together (e.g. {{ test_steps[0].request.body.amount + test_steps[1].response.body.fee}} to simulate fees being added) or generating a new UUID (e.g. {{ uuidv4() }}). If we try to encode all this dynamic behaviour into an object with special marker types, it's going to get very complicated very quickly.

That's why my suggested flow was to use an existing template language like Twig. That would look something like this:

1. Define one big template string for the request body in the YAML
2. Store the template as a string in the database
3. During test execution, prepare a data structure that contains variable values (test_steps, environment, ...)
4. Pass the template string and the data to Twig, which will return a new string (hopefully containing valid JSON but tbh we could also support other structures like XML with this approach)
5. Use the resulting string as the request/response body.

---

For the status dropdown, we need to support the full range - but it would be acceptable to have a limited selection available for autocomplete, and let the user enter a custom number to override if they want. I don't know if that is easier, or if it's easier to just support the whole range with autocomplete.

[agalaguz]

Thank you, we'll have more detailed grooming of it once finished with the alpha ver of TC edit UI (second week of this sprint).