ucsdlib / damsmanager

DAMS Manager
Other
3 stars 1 forks source link

Address indexing queue wait time #266

Closed arwenhutt closed 6 years ago

arwenhutt commented 6 years ago

Descriptive summary

Editing a CLR launches a re-index of all objects in the collection and all linked sub-collections, creating a very large queue which delays indexing of unrelated edits and ingests. While some edits, such as Title value, do have an impact on all linked objects, many other edits do not and therefore should not trigger a full re-index.

Long indexing queues, whether triggered by a CLR edit, or any other edit or ingest job, can have negative impacts on time sensitive changes to records (i.e. embargo content, correct erroneous information) DOMM is asked to make. We would like to be able to push re-indexing of an item to the front of the queue.

Request

gamontoya commented 6 years ago

Per Longshou:


this may cause the inconsistent issue.

Maybe we can make the collection the `urgent` priority, while the objects in the 
collection are `high` priority. 

The `urgent` one is more important. I think we can add a checkbox `Urgent` in the 
RDF Import form to do the trick if the checkbox is checked.
lsitu commented 6 years ago

@arwenhutt / @mcritchlow I've added PR https://github.com/ucsdlib/damsmanager/pull/282 that will give the item edited the highest level of priority 9, and all other linked records will be indexed in a lower priority. I also update our dev tool to index single items in the highest priority so that we can index some urgent items at any time. The PR is ready for review now.

Attached is the screenshot of the optional checkbox on the bottom for urgent indexing in the RDF Edit tool:

screen shot 2018-11-02 at 1 46 05 pm

Per my experience in the past, I think the item with highest priority should be index in around 20 minutes. Does it sound good to you?