search

mitdbg / aurum-datadiscovery

MIT License
74 stars 49 forks source link

Idempotent indexing of profiled documents into elastic #116

Open raulcf opened 6 years ago

raulcf commented 6 years ago

Indexing the same relation more than once should update the profiles associated to that relation, but not duplicate them.

raulcf commented 6 years ago

This is the relevant info: https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html

Basically, this requires having an 'id' generated by aurum in a consistent way, so when a document is reindexed, we assign the same id as before, and the document gets updated correctly.