search

NCATSTranslator / reasoner-validator

Validation of Translator OpenAPI (TRAPI) messages both to TRAPI and Biolink Model standards. See https://ncatstranslator.github.io/reasoner-validator/
Other
2 stars 4 forks source link

readme

Reasoner Validator

Pyversions Publish Python Package Sphinx Documentation Run tests License: MIT

This package provides software methods to Translator components (e.g. Knowledge Providers and Autonomous Relay Agents) using any version of the Translator Reasoner API (TRAPI) and the Biolink Model.

See the full documentation and the contributor guidelines.

Using the Package

Python Dependency

The Reasoner Validator now requires Python 3.9 or later (some library dependencies now force this).

Installing the Module

The module may be installed directly from pypi.org using (Python 3) pip or pip3, namely:

pip install reasoner-validator

Installing and working with the module locally from source

As of release 3.1.6, this project uses the poetry dependency management tool to orchestrate its installation and dependencies.

After installing poetry and cloning the project, the poetry installation may be run (within the available poetry shell):

git clone https://github.com/NCATSTranslator/reasoner-validator.git
cd reasoner-validator
poetry use 3.10
poetry shell
poetry install

Note that the poetry env can be set to either Python 3.10 or 3.11 at the present time.

This installation also installs testing dependencies (in the poetry 'dev' group in the pyproject.toml) and documentation dependencies (in the corresponding poetry 'docs' group). If you don't want the overhead of these dependencies, then the installation of these poetry group dependencies may be excluded:

poetry install --without dev,docs

If you plan to run the web service API, then install it with the optional web group:

poetry install reasoner-validator --with web

Running Validation against an ARS UUID Result(*) or using a Local TRAPI Request Query

A local script trapi_validator.py is available to run TRAPI Response validation against either a PK (UUID) indexed query result of the Biomedical Knowledge Translator "Autonomous Relay System" (ARS), a local JSON Response text file or a locally triggered ad hoc query Request against a directly specified TRAPI endpoint.

Note that it is best run within a poetry shell created by poetry install.

For script usage, type:

./trapi_validator.py --help

(*) Thank you Eric Deutsch for the prototype code for this script

Running tests

Run the available unit tests with coverage report:

poetry run pytest --cov

Note that poetry automatically uses any existing virtual environment, but you can otherwise also enter the one that is created by poetry by default:

poetry shell
# run your commands, e.g. the web service module
exit  # exit the poetry shell

The use of the Poetry shell command allows for running of the tests without the poetry run prefix. We will continue in this manner.

% poetry shell
(reasoner-validator-py3.9) % pytest --cov

Run the tests with detailed coverage report in a HTML page:

pytest --cov --cov-report html

Serve the report on http://localhost:3000:

python -m http.server 3000 --directory ./htmlcov

Building the Documentation Locally

All paths here are relative to the root project directory. The validation codes MarkDown file should first be regenerated if needed (i.e. if the codes.yaml was revised):

cd reasoner_validator
python ./validation_codes.py

Then build the documentation locally:

cd ../docs
make html

The resulting index.html and related pages describing the programmatic API are now available for viewing within the docs subfolder _build/html.

Validation Run as a Web Service

The Reasoner Validator is available wrapped as a simple web service. The service may be run directly or as a Docker container.

API

The web service has a single POST endpoint /validate taking a simple JSON request body, as follows:

{
  "trapi_version": "1.4.1",
  "biolink_version": "3.5.0",
  "target_provenance": {
    "ara_source": "infores:aragorn",
    "kp_source": "infores:panther",
    "kp_source_type": "primary"
  },
  "strict_validation": true,
  "response": "{<some full JSON object of a TRAPI query Response...>}"
}

The request body consists of JSON data structure with two top level tag:

Running the Web Service Directly

First install the web-specific dependencies, if not already done (e.g. by --all-extras above):

poetry install --extras web  # or poetry install --all-extras

The service may be run directly as a Python module. The web services module may be directly run, as follows. 

python -m api.main

Go to http://localhost/docs to see the service documentation and to use the simple UI to input TRAPI messages for validation.

Typical Output

As an example of the kind of output to expect, if one posts the following TRAPI Response JSON data structure to the /validate endpoint:

{
  "trapi_version": "1.4.2",
  "biolink_version": "4.1.5",
  "response": {
      "message": {
        "query_graph": {
            "nodes": {
                "type-2 diabetes": {"ids": ["MONDO:0005148"]},
                "drug": {"categories": ["biolink:Drug"]}
            },
            "edges": {
                "treats": {"subject": "drug", "predicates": ["biolink:treats"], "object": "type-2 diabetes"}
            }
        },
        "knowledge_graph": {
            "nodes": {
                "MONDO:0005148": {"name": "type-2 diabetes"},
                "CHEBI:6801": {"name": "metformin", "categories": ["biolink:Drug"]}
            },
            "edges": {
                "df87ff82": {"subject": "CHEBI:6801", "predicate": "biolink:treats", "object": "MONDO:0005148"}
            }
        },
        "results": [
            {
                "node_bindings": {
                    "type-2 diabetes": [{"id": "MONDO:0005148"}],
                    "drug": [{"id": "CHEBI:6801"}]
                },
                "edge_bindings": {
                    "treats": [{"id": "df87ff82"}]
                }
            }
        ]
      },
      "workflow": [{"id": "annotate"}]
  }
}

one should typically get a response body something like the following JSON validation result back:

{
  "messages": {
    "Validate TRAPI Response": {
      "Standards Test": {
        "info": {
          "info.query_graph.edge.predicate.mixin": {
            "global": {
              "biolink:treats": [
                {
                  "edge_id": "drug[biolink:Drug]--['biolink:treats']->type-2 diabetes[None]"
                }
              ]
            }
          }
        },
        "skipped": {},
        "warning": {},
        "error": {
          "error.query_graph.edge.predicate.invalid": {
            "global": {
              "biolink:treats": [
                {
                  "edge_id": "drug[biolink:Drug]--['biolink:treats']->type-2 diabetes[None]"
                }
              ]
            }
          }
        },
        "critical": {}
      }
    }
  },
  "trapi_version": "v1.4.2",
  "biolink_version": "4.1.5"
}

To minimize redundancy in validation messages, messages are uniquely indexed in dictionaries at two levels:

  1. the (codes.yaml recorded) dot-delimited validation code path string
  2. for messages with templated parameters, by a mandatory 'identifier' field (which is expected to exist as a field in a template if such template has one or more parameterized fields)

OpenTelemetry and Jaeger

NOTE: OpenTelemetry is temporarily disabled in this code release (to be updated later)

The web service may be monitored for OpenTelemetry by setting an environment variable TELEMETRY_ENDPOINT to a suitable trace collecting endpoint in an application like Jaeger (see also the Translator SRI Jaeger-Demo).

Note: the current system Docker (Compose) design only supports OpenTemplate tracing using the internal Jaeger container and may require further refinements to enable use of an external telemetry collector.

Running the Web Service within Docker

The Reasoner Validator web service may be run inside a docker container, using Docker Compose.

First, from the root project directory, build the local docker container

docker-compose build

Then, run the service:

docker-compose up -d

Once again, go to http://localhost/docs to see the service documentation.

To stop the service:

docker-compose down

Of course, the above docker-compose commands may be customized by the user to suit their needs. Note that the docker implementation assumes the use of uvicorn

Change Log

Summary of earlier releases and current Change Log is here.

Code Limitations (implied Future Work?)

Core Contributors