search

NCATSTranslator / reasoner-validator

Validation of Translator OpenAPI (TRAPI) messages both to TRAPI and Biolink Model standards. See https://ncatstranslator.github.io/reasoner-validator/
Other
2 stars 4 forks source link

Capture and use the 1.4.0 TRAPI Response biolink_version and (TRAPI) schema_version declarations #87

Closed RichardBruskiewich closed 1 year ago

RichardBruskiewich commented 1 year ago

These may/will override the global values in the Translator SmartAPI Registry(?) or may be identical (hence, more convenient for local validations of Response JSON?). In some cases, even if the Registry values are not updated, the local Response values may be correctly set?

There are currently four existing and possible sources of TRAPI / Biolink version setting for validation purposes in between reasoner-validator and SRI_Testing:

  1. the 'default' latest from TRAPI git release and from BMT
  2. the command line (for both tools) overriding any other assumed version
  3. (for SRI Testing only), the Translator SmartAPI Registry entry declared TRAPI and Biolink versions. I've just been noticing that in some cases, the Translator SmartAPI Registry doesn't always seem updated possibly even relative to the reported TRAPI Response, or the 'latest' standards
  4. The TRAPI Response JSON schema_version and biolink_version declarations
edeutsch commented 1 year ago

I was until recently more or less doing this in the ARAX code. However, it occured to me that if I get a Response that claims compatibility with TRAPI 1.4.0-beta1 and Biolink 3.0.1, that's all fine and dandy, but the current mandate is that we're compliant with TRAPI 1.4 release (TRAPI 1.4.1 in practice) and Biolink 3.5.0 so regardless of what the Response says it is, I'm going to tell you what isn't validate against the current mandate. So it's perhaps a bit complicated on which version the validator should use. And maybe it shouldn't be overriding what I tell it to validate against. I'm not certain

RichardBruskiewich commented 1 year ago

Hi @edeutsch, I think I mentioned this, but the reasoner-validator API defaults to the latest TRAPI and Biolink Model Toolkit (default) Biolink Model versions, if one doesn't specify otherwise.

An alternative here could be to default to the TRAPI Response specified biolink_version and schema_version in the JSON message and only override if the trapi_version and/or biolink_version are given as parameters to the API (or CLI or web service).

Alternately (or complementary), we could use the TRAPI response value but accept 'latest' as arguments to the above parameters to force validation against 'latest'.

A final option would be to assume 'latest' by default (the current implementation) but have some way to signal the use of the TRAPI Response JSON values.

I'm not 100% sure what would be the best policy (hence implementation) here.

I'm reminded of this because I came across a KP (service-provider-trapi) which is failing all validation due to an older model compliance (pre-dating 3.1.0). I guess this supports the idea of "compliance with latest" being the desired default(?).

edeutsch commented 1 year ago

okay, as long as what is specified by the caller is not overridden, then whatever the decisions are when the caller does not specify is fine with me.

RichardBruskiewich commented 1 year ago

@edeutsch I totally agree that the caller should always be able to override the TRAPI and Biolink versions as parameters to the key validation methods or (equivalently) on the command line.

I guess the uncertainty here is what to do if it is not given on the reasoner-validator "caller" (including CLI) versions.

I can point out, BTW, that option 3 - Translator SmartAPI Registry - isn't really a troubling option simply because reasoner-validator doesn't check the Registry (only SRI_Testing), although I'd suspect that the TRAPI Response JSON version should take precedence over the Registry version (the two ought to be identical but the Registry may occasionally lag implementation?)

The leaves the ambiguity of choosing either the TRAPI Response JSON stated schema_version and biolink_version or choosing the current default TRAPI and BMT versions, whenever the "caller" values are not specified.

In such cases, I reason that again, the "caller" is enforcing the validation to the latest... the ARA and KP need not always keep up with the evolution of 'latest'. Thus, what I think we might do is explicitly specify latest as the value for either or both "caller" parameters, to force use of the latest version.

Note: the software actually already accepts a special value of 'suppress' to the biolink_version when Biolink Model validation is not desired.

RichardBruskiewich commented 1 year ago

The final design is the following precedence: 1) Python method (API) level override (which includes SRI_Harness use of the Translator SmartAPI Registry recorded version by default, if the SRI_Testing CLI doesn't say otherwise) 2) TRAPI Response message schema_version and biolink_version 3) Default TRAPI and BMT biolink version otherwise

Fully implemented by next release (probably 3.7.6) which will incorporate code from commit 5d356db