Related to #122, something that will help address the problem of identifiers will be slightly changing how they are handled internally. This issue outlines the process of adding a new column combine_id, a randomly generated UUID hash at a record's creation.
It will keep the record_id as a string that is semantically meaningful, and will be used for publishing. What this allows for, is altering the record_id if desirable, perhaps even a dedicated task in Combine.
Current progress:
Done
adding combine_id to Record model
updated combine_tables_prime.sql
BUILD: need to update this?
both tables
have linkage between records look for combine_id not record_id
make sure ES indexes with combine_id, and not record_id
having Spark jobs write random uuid to combine_id
still writing record_id as before
using withColumn likely...
ES doc retrieval
use record_id for OAI publishing
ToDo
create workflows for updating record_id
adding as "tack on" like validation scenarios and index mapping for jobs
with live preview as well (consider moving test_record_id_transform() view to models?)
find and address where Records were selected or disambiguated with record_id and change to combine_id if it makes sense
add combine_id to search in tables?
*moving considerations about uniqueness to new issue: #125
Related to #122, something that will help address the problem of identifiers will be slightly changing how they are handled internally. This issue outlines the process of adding a new column
combine_id
, a randomly generated UUID hash at a record's creation.It will keep the
record_id
as a string that is semantically meaningful, and will be used for publishing. What this allows for, is altering therecord_id
if desirable, perhaps even a dedicated task in Combine.Current progress:
Done
combine_id
toRecord
modelcombine_tables_prime.sql
combine_id
notrecord_id
combine_id
, and notrecord_id
combine_id
record_id
as beforewithColumn
likely...record_id
for OAI publishingToDo
record_id
test_record_id_transform()
view to models?)record_id
and change tocombine_id
if it makes sensecombine_id
to search in tables?*moving considerations about uniqueness to new issue: #125