MetaDIG API Additions/Revisions

[gothub]

Existing API

The current API available via the metadig-webapp contains these services. Should all these be retained?

Service URL	Description	Java Method Called	Keep? *
/checks	lists names of available quality checks	ChecksResource.listChecks()	[x]
/checks/{id}	retrieves check XML	ChecksResource.getCheck()	[x]
/checks/{id}/run	runs check with provided metadata, sysmeta	ChecksResource.run()	[x]
/suites	lists names of available quality suites	SuitesResource.listSuites()	[x]
/suites/{id}	retrieves a suite as XML	SuitesResource.getSuite()	[x]
/suite/{id}/run	runs all checks in suite against provided metadata, sysmeta	SuitesResource.run()	[x]
/runs	lists all cached runs	RunsResource.listRuns()	[x]
/runs/{id}	retrieves report for specified run	RunsResource.getRun()	[x]

• Currently GFM doesn't allow for checkboxes in tables, so these will have to be changed manually

Proposed Additions to API

The following services are being considered for addition to the webapp:

Service URL	Description	Java Method Called
/queue/start	Start quality engine processing	QueueResource.start()
/queue/stop	Stop quality engine processing	QueueResource.stop()
/queue/info	Retrieve info about engine	QueueResource.info()
/queue/pid/status	Retrieve info about metadata pid in queue	QueueResource.status()
/queue/pid/remove	Remove an entry from processing queue	QueueResource.remove()
/queue/pid/add	Add an entry to processing queue	QueueResource.add()
/queue/test	Send test entry to processing queue	QueueResource.test()
/suite/{id}/add	Add a new suite definition to the engine	SuiteResources.addSuite()
/check/{id}/add	Add a new check definition to the engine	ChecksResources.addCheck()

These were first discussed in the MetaDIG meeting on 2018/01/10 and documented here The original list of functions that the above list was derived from is:

• queueSize()
• startJobProcessing()
• stopJobProcessing()
• workerCount()
• addToQueue()
• removeFromQueue()
• listQueue()
• addCheck()
• addSuite()
• getCheck()
• getSuite()

[mbjones]

These look great. Would there be an API for adding and removing worker instances?

[gothub]

Yes, definitely. The configuration of the engine should have a default configuration but the configuration should be able to be changed dynamically, such as the number of workers, the member node that is being queried for generation of quality documents, etc.

[tedhabermann]

Peter,

I agree this looks cool… I note that there is a capability to create a new suite but I don’t see something for adding check(s) to a suite…? Am I missing something?

Ted

[mbjones]

@gothub I also just noticed that you have separate endpoints for add and remove operations, such as:

• /queue/pid/remove
• /queue/pid/add

Under the REST model, these should not be different endpoints, but rather use HTTP verbs to indicate the operation to be performed on the resource. So, in this example:

• /queue
  • GET to list the pids in the collection
  • POST to add a new pid to the queue
• /queue/pid
  • GET to retrieve info about the queued pid
  • DELETE to remove the pid
  • PUT to modify the queue entry for this pid

This style of change should occur throughout the API.

[mbjones]

Ted suggested we should consider using Swagger and OpenAPI to describe this API. I think that is a great idea. Started a new issue #4 for that work.

[gothub]

The current API definition is available at https://app.swaggerhub.com/apis/mbjones/metadig/2.0.0-a1-oas3

[gothub]

The info from this issue is covered in https://github.com/NCEAS/metadig-webapp/issues/4