DTS API validator

[mromanello]

Proponent / Mentor

Matteo Romanello (@mromanello)

Brief Description

(I believe this came up during the hackathon info session, but I can't recall exactly by whom, sorry!) The idea is to develop a validation tool for the DTS API, along the lines of IIIF's validators for the image- and presentation-APIs (see https://github.com/IIIF/image-validator and https://github.com/IIIF/presentation-validator). Such a tool would be extremely valuable to developers for testing new DTS API implementations. The front-end can be kept extremely simple. The bulk of the work would be define validation tests based on the current specs.

Coding skills needed

Javascript, Python

Team members (if applicable)

Anyone with the above (or a comparable) set of skills is welcome to join this hack.

[Grantglass]

I would love to help out!

[PietroLiuzzo]

Hi @mromanello and @Grantglass ! As a user I would really love to have this. I do not find the IIIF one terribly useful however, if I may. beside checking json syntax and the ld part by making sure the context is there and is real I would love if a validator would inform me on

• completeness : e.g. give me an INFO if dts:references and dts:passages are not present
• coherence : e.g. tell me that my navigation is non uniform in the format of references with respect to dts:citeStructure, etc.
• functionality : if document api is available try it for the references provided (possibly not all?? and tell if it returns as expected)
• minimal information : tell me if the minimum information to make the API useful are present. This would help me a lot for my Hack, because I could rely on the validator response to decide without prior knowledge what I can print out of an existing resource available via the dts endpoint.

[mromanello]

Unfortunately, this hack idea did not take off (for lack of time). But I'm pasting here the preparatory notes/brainstorming I've jotted down while thinking about implementation.

Aspects to be tested/validated:

• syntax/formal aspects

  • JSON-LD: context is there and it's real
  • the response is in a valid format

• coherence

  • test for uniqueness of collection/resource ids

• validity/completeness

  • test for validity of IDs (collection/resource)
  • test for things that must be there (as per specs)

• functionality

  • give a summary of error types per endpoint?
  • can I retrieve collections and resources? if so, how many?
  • is pagination available (when browsing collections), does it work?
  • document endpoint: if available, can I retrieve all documents (or all in the sample)?

---

Implementation idea

Basically an adaption of the IIIF image validator

Core classes:

Class	Description
TestSuite	A collection of tests to run against a given DTS API. It has a run_test() method to run each test, as well as methods to find and list them (list_tests())
DTS_API	Object responsible for building and making API calls to the DTS instance under validation. TestSuite.run_test() passes as input the test name and an instance of DTS_API
BaseTest	Class that each test class will have to extend. Defines basic behaviour and information related to each test. Each test has a run() method which takes in input an instance of DTS_API and tests a certain aspect.
Validator	This classes uses the Bottle Python framework to create an API/web service.
ValidationInfo	object that gets passed around to each test by TestSuite, so that validation infos are stored in it via ValidationInfo.check()

---

Other stuff

• validating API responses for all resources vs validating for a sample

  • runtime will change drastically for large corpora
  • perhaps worth having a flag to switch between the two modes

• how to present the validation results? ideally a JSON summary that is both machine- and human- readable/renderable. This way it can be displayed for humans and also read by machines (clients, services, etc.)