Ascertaining TRAPI and Biolink Model versions for tests

[RichardBruskiewich]

SRI_testing unit test data JSON templates currently have 'url' (string) and 'TRAPI' boolean arguments.

The former generally points to the service endpoint of the Translator resource. There is perhaps(?) a convention that the path of the URL should include a string encoding the version of TRAPI (e.g. v1.1) implemented by the API, although this convention is not yet universally implemented by all KP's and ARA's (?!).

The TRAPI argument stores a purely boolean value and (to frankly) pretty redundant here, in that all the SRI testing framework currently assumes that accessed services are TRAPI implementations.

Would it be therefore be sensible then, to repurpose the TRAPI tag value to instead explicitly encode the TRAPI version implemented by the given resource url? Do we even rename the tag from 'TRAPI' to 'TRAPI_version" (or 'trapi_version', if we are concerned about keeping all such tags as lower case)?

Concurrently, one might assume that a given knowledge resource (KP or ARA) being tested, is also complying with a given Biolink Model standard with a given set of test triples (i.e. may have specific categories and predicates found in a specified (earlier?) release of the Biolink Model than the 'latest' release that, say, the Biolink Model Toolkit defaults to)?

Would it therefore be sensible to record that version number as a new tag in the tests, perhaps 'Biolink_version' (or 'biolink_version' for a lower case tag) or perhaps Biolink_Model_version' (or 'biolink_model_version' lower case) to be more precise)?

[cbizon]

I'm wondering if maybe we should be taking more advantage of the smartAPI registrations here. Those registrations have the TRAPI versions and server urls in them. So maybe the test triples should be defined as (infores, x-maturity) as the scope. Then we can go dig around in smartAPI to find the other information?

[RichardBruskiewich]

Sounds sensible @cbizon.

@putmantime I guess we would use this in our efforts to pull in test data dynamically from every KP and ARA

[putmantime]

Yes agreed that smartapi is where to get those. The x-trapi extension has version as an attribute, that's even the sibling to the 'test_data_location' extension we are creating. So we will have access to the version and other info as we will already have the spec when getting test data

[RichardBruskiewich]

Have we already put the Biolink Model version there too?

[cbizon]

I don't think biolink shows up in the smartapi registration. We mostly haven't worried about minor version changes there, since they rarely seem to cause problems.

[RichardBruskiewich]

Minor (i.e. mostly 'patch'?) version changes generally add semantics, not remove it permanently (i.e. 'patch' versions only deprecate terms? Minor versions may remove them?). At the moment, I'm just defaulting to the 'latest' BMT default Biolink Model version. I suppose that we can cross the bridge of Biolink Model awareness when the tests start failing for that reason (they probably are already, with the older test data in the repo, which need updating...).

The TRAPI changes from <=1.0 to 1.2.* actually was a more severe breakage of the SRI tests (now mostly fixed, I think...).

The new current challenge in this project here may be to implement (and get community support for) SmartAPI-driven sample test data access (to replace the repo-accessioned test templates). I suppose TRAPI versioning itself is not too terribly relevant for the moment since the whole consortium are being encouraged to standardize around one TRAPI version at a time (e.g. 1.2... moving later this year to 1.3?). The SRI Test framework likely won't care about more than one major.minor TRAPI version at a time. Ditto the Biolink Model versions.

Bottom line: for now, the repo can be coded as TRAPI 1.2 and Biolink Model 'latest BMT default' for now. Perhaps we can tag the current release of the project as SRI_Testing 1.2, on that basis?

I'll close this issue for now, unless either of you say otherwise.