usnistgov / iheos-toolkit2

XDS Toolkit
https://ihexds.nist.gov
45 stars 26 forks source link

Making additional configuration information available to the Conformance Tool #443

Open iheos opened 5 years ago

iheos commented 5 years ago

The testing scenario triggering this comes out of Sequoia. For IHE Pre-Connectathon and Connectathon testing we have an established pattern of what configuration information is needed to run Conformance Tool tests. Sequoia has different requirements. Two specific issues they face are that homeCommunityId and Patient ID selections are forced by global configuration and they are used in the construction of messages in a different way from our current testing. Some examples.

HomeCommunityId must be inserted in XCPD messages. Actually this is done twice. The HomeCommunityId of both the sending station (Toolkit) and the receiving station (SUT) must be inserted into the XCPD messages. It is trivial to re-code the XCPD templates with variables for these values, the values are not currently available through the Report/UseReport facility.

The second key parameter is Patient ID. In current operation the Conformance Tool generates a Patient ID, publishes it on the display, and uses it in the collections of tests configured for that Actor type. There seems to be a need to allow an administrator to set a fixed Patient ID that will be used for a group of tests collected under a profile/actor/option grouping. Obviously this would not work for all profile/actor/options since uniqueness/isolation of data is sometimes critical. Testing a Registry actor is one place were this would not work.

For these two parameters HCI and PID it seems the HCI must come from the configuration of the selected SUT and the PID must come from some configuration area that is Test Session specific.

skbhaskarla commented 5 years ago

Is the status update issue with running independent Sections one-at-a-time documented somewhere else?

iheos commented 5 years ago

Not yet. Will be in a few minutes.

iheos commented 5 years ago

A third parameter needs to be managed, the HCI for the sending station (Toolkit). This would have to be established in the Test Session. All these parameters would need well known names so they could be coded into test plans.

iheos commented 5 years ago

Here is the design for this feature.

Additions to Orchestration

In the server-side orchestration code for each actor type...

class OrchestrationBuilder extends AbstractOrchestrationBuilder. AbstractOrchestrationBuider adds a new method

fillInGlobalParameters(AbstractOrchestrationRequest, RawResponse)

which is to be called just before returning. This method fills in the following parameters (in RawResponse.testParameters) which are extracted from AbstractOrchestrationRequest

SUT_homeCommunityId
Toolkit_homeCommunityId
Primary_PID

These parameters will now show up as Reports on all tests run under this orchestration. The last section below discusses how these parameters may be referenced when writing tests.

Supporting additions to Actor/Site

The testing of a system is organized in a Test Session. The Test Session references a single Actor/Site object which defines the SUT. These additional parameters will be created in the Actor/Site definition.

SUT_homeCommunityId is already present (under a slightly different name). Toolkit_homeCommunityId and Primary_PID are new.

A new optional parameter to an Actor/Site definition is Primary_PID. Its value is a String. This value will feed the above mechanism. The System Configuration tool will not validate the format of this parameter.

Another new optional parameter is Toolkit_Site. Its value is a String. This string, if not empty, is constrained to be the name of another Actor/Site. If not empty the System Configuration tool will validate that it reference an Actor/Site by name. If this parameter has a value, a site name, and the referenced site has a homeCommunityId value then that value will be declared the Toolkit_homeCommunityId.

The implication of this is that one or more Actor/Site definitions can be created representing the Toolkit system. This is only relevant when we need to know the homeCommunityId of the sending system (Toolkit) to encode it in a transaction. There can be multiple Actor/Site definitions for this purpose. They do not represent Toolkit (actually Toolkit Client) until they are reference by an Actor/Site Toolkit_Site parameter. If needed, each SUT can reference Toolkit by a different Toolkit_System and therefor use a different homeCommunityId for Toolkit if that is necessary.

Linking Actor/Site to Test Report/UseReport

The orchestration infrastructure referenced in the first section will place these parameters, as Report statements, in all log files produced by the Conformance Tool. That is provided for in the first section above. To be reference-able by UseReport statements in tests it must be possible to code Test/Section/Step elements in the UseReport statement which reference a real log file. There are two ways of making this happen. If the Conformance Tool Actor/Profile combination declares at least one Orchestration step (a testplan listed in the Orchestration section near the top of the Conformance Tool display) then that test can be referenced in the UseReport. If not then a special test is needed below in the tests section so it can referenced.

If such a special test is needed it is easy to write: a single section contained a single step referencing the NullTransaction. It will do nothing but generate a log file containing the parameters as Reports.

Providing for this special test (or not) is the responsibility of the test writer.