Merritt SWORD more strict about REST request types

CDLUC3 / mrt-doc

Documentation and Information regarding the Merritt repository

8 stars 4 forks source link

Describe the bug Dryad sends two types of REST requests to Merritt-sword. We send POST requests (insert) for new datasets and PUT requests (update) for version updates. This is verified from reading our code and seeing where we make requests. We do use different types of requests against Merritt-Sword.

It would be nice if SWORD rejected PUT requests where it cannot find the previous object that it should update.

This may depend on whether PUT requests are ever used for anything besides updates in sword.

To Reproduce

Put Merritt into a state where it cannot determine whether a local id (DOI) already exists.
Do a PUT request to an existing dataset with a DOI to Merritt sword.
Merritt-Sword creates a new object in the Merritt system rather than returning an error or adding a version to the already-existing object. It results in objects being split into multiple objects in Merritt and requiring manual cleanup to fix the objects.

Expected behavior If updates come in where the object being updated cannot be determined then they are rejected.

Note: This may not be possible if PUT requests are also used for upserts (insert and update), but I do not believe we use upserts in Dryad when communicating to Merritt-Sword, but IDK if they may be used elsewhere or if they are a part of a spec for it.

Additional context

This may not be a general solution to all the problems, but might prevent problems with submissions coming in from the SWORD interface.

If this is an easy change, it may save manual cleanup time from versions split across objects by rejecting them. Cleanup is much easier for failed submissions (just resubmit when the system is working) rather than doing manual data analysis and cleanup.

Ingest

add - All of the content sent to ingest is used as the new version. All previous versions are ignored for construction of the current version
update - Content sent to ingest is used to determine what to replace if already exists, or add if does not exist

Sword

Sword collection - POST command - sent to ingest as an update. This is treated as an update because the update behavior would be the same as add because the POST container should contain all content needed to build a new version. No arks are provided in the request.
Sword container - PUSH command - again sent to ingest as an update because the container should contain only content needed for changing existing current version. No arks are provided in the request.

Ingest

Ingest uses a profile containing the owner for a the collection in the submission. Using this owner with the submitted local identifier ingest sends a request to inventory to get confirmation whether a previous ingest command had created an ark for an already existing object.

if no existing object - get new ark - send ingest a build for the new owner-localid->ark - creat new object
if existing object - update the current ark with a new version

Some issues for Sword to detect previous submission

Sword currently does not have the owner of an existing collection - this is in the ingest profile. This prevents sword from doing a local id test. Arks are not provided as input to the Sword update
This particular problem appears to have occurred as part of the localid lookup not being successfully processed. This could happen if a previous collection call had not completed.

Possible solution

If an ark could be returned as part of the container call, the resulting ingest request would be made on that ark and not the localid lookup in ingest.

CDLUC3 / mrt-doc