fhir-crucible / testscript-engine

Ruby FHIR TestScript execution engine
Apache License 2.0
5 stars 2 forks source link

Updates to TestReport Handling #27

Closed ms-k1ngk0ng closed 2 years ago

ms-k1ngk0ng commented 2 years ago

Created a new TestReportHandler module, which contains a TestReportBuilder class. Together, the module and class use some ruby mixin magic, allowing a new TestReport to be created any time a runnable is executed. It is now easier on the runnable side to update the TestReport as the report handling is abstracted away. I made some basic modifications to the TestScriptRunnable class to show how/where the TestReport is updated.

Next step is going to be fine-tuning the engine's control flow: i.e. what happens when an operation fails during setup? How about during test?

ms-k1ngk0ng commented 2 years ago

Great job!!! I have two comments:

  1. Compared to previous version, TestReport exported is missing language element. It's not essential so should be okay without it. Just let you know my finding.
  2. Some TestReports are missing action.assert.result. Its cardinality is 1..1 so should be included under assert. You can test with history_test_script.json
    "action": [
        {
          "operation": {
            "result": "fail",
            "message": "[.assert_path] Expected resource with id:  in fixtures, responses, or in last reply from server. No such resource found."
          }
        },
        {
          "assert": {
            "message": "Confirm that the returned HTTP status is 200(OK)."
          }
        },
        {
          "assert": {
            "message": "Confirm that the returned resource type is Bundle."
          }
        },

@jhlee-mitre My suspicion is that the second issue will be addressed through the bug fix merged in from yesterday. I will update this branch and confirm.