Available Scripts are not documented in the REST Contract

[abollini]

The script endpoints provide access to the list of configured / enabled scripts and each script provide a machine readable description of the script purpose and of the expected parameters. Nevertheless, such information are not tracked in the REST Contract by their-self so

• they are not available "offline" for consultation
• the are not (directly) tracked over version, it is hard to tell which script was available in a specific dspace version and how it was expected to be used
• it is not possible to discuss with the community the design of a new script other than in the code implementation, this could pass unnoticed. Despite to haven't yet a clear statement about the REST contract and the support versioning model changes to an existing script parameters or behavior is very similar to a change to an endpoint as external system or custom UI could depend on this script

I have opened this issue to see what other things, and if you agree to create a page in our REST contract to list all the existing script and their configuration. For each script I suggest to track

• name
• description
• parameters
• who has access to it (see also https://github.com/DSpace/RestContract/issues/219 )

[abollini]

@tdonohue on this task I would like to get an initial feedback from you and anyone else about the idea. If you agree that is good to have the script documented in the REST Contract we will work on the PR

[hardyoyo]

I like this idea, it seems like a good fit for the kinds of conversations that need to occur about script design, and those sorts of conversations already take place around changes in the REST Contract. And since script design does impact the REST-API, via the script endpoint, this makes sense. 👍

[mwoodiupui]

While the "script" mechanism is exposed through the REST contract, it seems to me that the individual "scripts" are not properly part of the REST layer, since they live in the dspace-api layer and can also be run from the command line.

Each "script" is partially self-documenting -- it will tell you what options and parameters are available. I feel that elaboration of the meanings of these, and suggestions for proper use, belong in the manual, which would provide the "offline" access that is sought.

Where to discuss the design of new features before spending a lot of resources on coding them is a problem now because DSpace is spread across three code repositories, none of which really has an over-arching scope. I would probably file an issue against DSpace/DSpace, but that is due to habit and perhaps not the best way to get proper attention from all stakeholders. Maybe we need to take such discussions back to a mailing list, until a design is firm enough to divide the work and file issues against all of the affected repo.s.

I agree with the request for more and better documentation of "scripts". I will note that the manual has quite a lot of version information scattered through it already, so that good documentation of a "script" there would be a natural place to note when it became available. This is probably something we could do better across all new features.

[mwoodiupui]

I want to add that I have noticed from time to time in the manual that extended discussion of settled designs is jumbled together with "how to use it" in an unclear manner. Once or twice I have pulled a page apart into two pages to disentangle these aspects and put them in their proper sections (with cross-links, of course).

[tdonohue]

In Slack discussion, it was pointed out by @abollini that there are some good reasons to move Script documentation into the REST Contract:

• we cannot propose/discuss changes to the wiki documentation via a git PR
• we have less control about this important changes, it is easy to get them unnoted if they are discussed just in a REST API PR/issue but they really affect our REST API. If we change the parameter of the curation, the import, export we will broken the Angular UI
• external developer that just make an integration are likely to use also this extra endpoints and would need to have access to the documentation without accessing a real instance (this is solved also putting the details in the DSpace documentation but in any case it would be a duplicate too)

Overall, I think we all agree that Scripts are not officially part of the "Contract". But, they are important to document alongside the contract because changes to their params and usage will impact the user interface & any clients that use those scripts via the REST API.

So, I'm fine with moving documentation for Scripts to the REST Contract (and @artlowel agreed as well). But, I feel that the Wiki Docs should then be updated to link to the Scripts docs in the REST Contract. We may also want to keep the Scripts docs in a slightly "separate" area of https://github.com/DSpace/RestContract by creating a /scripts subdirectory for their docs, or similar.

[mwoodiupui]

Apparently I don't understand how "scripts" are plumbed into the UI. I thought that the UI provided a framework for carrying information about the availability of any abstract "script" to the user, and carrying the user's option choices to any abstract "script". If the UI is actually dependent on any "script" then I view that as a problem until I can understand why it isn't.

[mwoodiupui]

Yup, my understanding was incomplete. The UI has reused several scripts in a way which requires it to understand their options, unlike the "run a script" UI feature for which the user provides that understanding. I consider this close coupling a Bad Thing. Reforming it would require a fair amount of work.

[tdonohue]

@mwoodiupui and I had a detailed & very useful discussion about this (in Slack) which is worth documenting here.

Essentially, the goals of this ticket are to simply document what we have better to make it easier to use/understand (both for ourselves, as well as for any REST API clients, including our own UI).

• What we currently have is a model where each script (defined in scripts.xml) is represented as a sub-endpoint of /server/api/system/scripts/. For example, /server/api/system/scripts/export is the "export" script endpoint.
• This is very powerful as it means each newly defined script has an immediate REST endpoint, and all those endpoints work similarly.
• However, this design requires the REST client (e.g. our UI) to have some basic understanding of the script, namely the script name (which defines its endpoint) and what parameters are required.

What @mwoodiupui has suggested/noted, is that this current design requires the REST client (e.g. UI) to know more about individual scripts than it should. If a script's required parameters change, then the REST client could throw errors if it isn't aware. He's suggested we may want to consider future work to redesign what we have to no longer require as much script information to be hardcoded into REST clients.

• One basic example, could be that we could consider an endpoint like /server/api/collection/[:uuid]/metadata-export to allow a quick/easy way to trigger the "metadata-export" script for a specific Collection (without having to require the REST Client to "know" that the Collection UUID must be passed to the "metadata-export" script via the -i param)
• However, it would require creating some duplicative endpoints which make specific scripts easier to run _without a detailed understanding of all the scripts' parameters.

I think @mwoodiupui 's suggestion is worth further consideration...but it's definitely out-of-scope for this ticket. It'd require a deeper discussion of each of the scripts, and how to make them easier to "trigger" without knowing all the params. But, I think that discussion needs to be had separate from this initial "quick fix" of simply documenting what we currently have.

[Sondissimo]

Pull Request has been opened: https://github.com/DSpace/RestContract/pull/221