add object indexer service

[phish108]

the sdg-indexer operates on the entire corpus. This is highly inefficient, if only a few new objects are added to the corpus. In those cases a variation of the initial indexer should operate only on those few objects.

The algorithm is pretty straight forward:

• Tokenise and clean the object.
• concatenate title, abstract and extras
• drop duplicate terms
• fetch all keywords that have anyofterms in the keyword field. This reduces the search space of potential keywords to those that appear in the infoObject already.
• Run the normal indexer steps on the result set

[phish108]

This can be nicely integrated into the regular indexer via the message queue.

The algorithm will run all index terms against the reduced corpus of the added objects.

[phish108]

We use only indexer.add for corpus indexing via the UI We use only indexer.update

We use only importer.object for indexing newly imported objects