A shacl based validator backend service for validating catalogs against dcat-ap-no specification
The validator need the following input graphs:
The RDF graph containing the data to be validated
The RDF graph containing the SHACL shapes to validate with
The RDF graph containing extra ontological information. The validator will try to import ontologies referenced in owl:imports statements
Input graphs should be supplied in of the following ways:
The input may also contain a configuration record containing two options:
expand
(boolean: true
/false
): if set to true
, the validator will try to fetch remote triples referenced to in the data-graph.includeExpandedTriples
(boolean: true/false): if set to true
, the validator will include the remote triples and the ontologies in the response.Ref the openAPI specification. An example config record:
{
"expand": "true",
"includeExpandedTriples": "true"
}
The response will be a RDF graph consisting of
the input data graph, and
the report as a graph according to a SHACL validation report.
If includeExpandedTriples
is True
, the response will aslo include:
the additional triples added to the data graph, and
the ontology graph
% curl -i \
-H "Accept: text/turtle" \
-H "Content-Type: multipart/form-data" \
-F "data-graph-file=@tests/files/valid_catalog.ttl;type=text/turtle" \
-F "shapes-graph-file=@tests/files/mock_dcat-ap-no-shacl_shapes_2.00.ttl" \
-F "ontology-graph-file=@tests/files/ontologies.ttl" \
-X POST http://localhost:8000/validator
% curl -i \
-H "Accept: text/turtle" \
-H "Content-Type: multipart/form-data" \
-F "data-graph-url=https://example.com/mygraph" \
-F "shapes-graph-file=@tests/files/mock_dcat-ap-no-shacl_shapes_2.00.ttl" \
-F "ontology-graph-file=@tests/files/ontologies.ttl" \
-X POST http://localhost:8000/validator
% curl -i \
-H "Accept: text/turtle" \
-H "Content-Type: multipart/form-data" \
-F "data-graph-file=@tests/files/valid_catalog.ttl;type=text/turtle" \
-F "shapes-graph-file=@tests/files/mock_dcat-ap-no-shacl_shapes_2.00.ttl" \
-F "ontology-graph-file=@tests/files/ontologies.ttl" \
-F "config=@tests/files/config.json;type=application/json" \
-X POST http://localhost:8000/validator
% curl -i \
-H "Accept: application/json" \
-X GET http://localhost:8000/shapes
% curl -i \
-H "Accept: application/json" \
-X GET http://localhost:8000/shapes/1
% git clone https://github.com/Informasjonsforvaltning/dcat-ap-no-validator-service.git
% cd dcat-ap-no-validator-service
% pyenv install 3.10.6
% pyenv local 3.10.6
% python get-poetry.py
% pipx install nox
% pipx inject nox nox-poetry
% poetry install
Start the server locally:
% poetry run adev runserver dcat_ap_no_validator_service
% poetry run gunicorn dcat_ap_no_validator_service:create_app --bind localhost:8000 --worker-class aiohttp.GunicornWebWorker
To build and run the api in a Docker container:
% docker build -t digdir/dcat-ap-no-validator-service:latest .
% docker run --env-file .env -p 8000:8080 -d digdir/dcat-ap-no-validator-service:latest
The easier way would be with docker-compose:
docker-compose up --build
We use pytest for contract testing.
To run linters, checkers and tests:
% nox
To run tests with logging, do:
% nox -s integration_tests -- --log-cli-level=DEBUG
REDIS_HOST
Hostname where the remote redis is reachable on default port (6379). Default: localhost
REDIS_PASSWORD
Password to the remote redis is reachable.
Default: 6379
LOGGING_LEVEL
One of the supported levels found here.
Default: INFO
CONFIG
One of
dev
: will not use cache backendtest
: will not use cache backendproduction
: will require and use a redis backend, cf docker-compose.yml
Default: production
An example .env file for local development without use of redis cache:
LOGGING_LEVEL=DEBUG
CONFIG=dev