Exported URN should be able to be imported to another database

[simonreed]

At the moment the URN on export e.g. urn:ddi:uk.cls.ncds:ncds_00_cai-ca-321157:1.0.0 has the database id in this case '321157' however when you import it and the category URN will change.

We'd like to replace the database ID part (which is a primary key which is sequential so we can't set it when importing) to a slug which we can easily set to make sure when you import a URN it will be the same when exporting.

[HayleyMills]

Yes exactly, this will mean that the IDs will be stable when exporting and importing to different instances. The URNs generated on export will continue to use the current study agency and prefix.

Relates to #814

[simonreed]

@HayleyMills When importing an instrument or fragment and the URN already exists should we create a new Question (or anything else) or should we update the existing record?

As it stands at the moment we'll create a new record for Questions when we import either a whole instrument or a fragment.

[simonreed]

As it works currently because it creates a new db record the URN isn't duplicated between the 2 questions. However as we go to the DDI slug approach creating a new question record as well as keeping the existing one will mean we could potentially have 2 questions with the same URN as it'll be persisted upon import and export.

[spuddybike]

I don’t think should happen as a question name can’t be duplicated and if it is detached and replaced the old question would not be attached to a question construct so won’t be exported. On 7 Feb 2024, at 16:59, Simon Reed @.***> wrote: As it works currently because it creates a new db record the URN isn't duplicated between the 2 questions. However as we go to the DDI slug approach creating a new question record as well as keeping the existing one will mean we could potentially have 2 questions with the same URN as it'll be persisted upon import and export.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were assigned.Message ID: @.***>

[simonreed]

@HayleyMills this is now on staging, if there's any issue can you add the xml file that causes an issue.

[HayleyMills]

@simonreed are you able to export from staging? It's not currently exporting any instruments?

[simonreed]

@simonreed are you able to export from staging? It's not currently exporting any instruments?

No let me take a look

[simonreed]

@HayleyMills fixed, you can now export on staging

[HayleyMills]

@simonreed I have exported an instrument, imported into another instance and then exported from there. Should I be expecting the URNs to remain unchanged?

I think this was the aim so after we have moved the instruments to another instance and then export to replace in Discovery it doesn't think everything has been updated, and version everything, instead of only versioning what has actually changed? @spuddybike

Exported alspac_10_yayl from staging, imported to heaf and exported from heaf alspac_10_yayl_staging.txt alspac_10_yayl_staging_heaf.txt

[simonreed]

Looking at this now, I can use that xml to perform a full automated test.

[simonreed]

@HayleyMills can you test again on staging please. I've changed a lot of the db views and triggers to allow the URN to remain the same on import and export and we have an automated test to cover this.

[HayleyMills]

@simonreed I am unable to download the export again? I'm testing on staging, I can create the export, but the download export link cannot be found e.g. https://closer-archivist-staging.herokuapp.com/instruments/2504/export/6364

[simonreed]

@HayleyMills download of export is now working on staging

[HayleyMills]

@simonreed I can now download from staging but I can't download from other instances to complete the test, This is a new problem, this was previously happening for the wirral instance see #823, but it now happening for alspac, heaf, main, us and genscotland which were working.

[HayleyMills]

I've been able to export from staging bcs_80_pqf_staging.txt

Import then export from onsls, but the urn numbers still don't match? bcs_80_pqf_onsls.txt

[simonreed]

@HayleyMills the importing fix for the URNs are only available on the staging instance so you can only import this on staging at the moment. So you could export from onsls and then import and export from staging.

[HayleyMills]

OK thanks that makes sense, I've exported and then imported into staging instead and receive this error? https://closer-archivist-staging.herokuapp.com/admin/imports/1166

I exported from onsls and imported into staging and received this error onsls_71_hhw https://closer-archivist-staging.herokuapp.com/admin/imports/1167

[simonreed]

@HayleyMills sorry I was running a database migration could you try again please

[HayleyMills]

I exported from onsls and imported into staging and the URNs are the same (expect the date/time as expected).

[simonreed]

@HayleyMills closer-archivist-heaf has now been updated

[HayleyMills]

I exported from heaf and imported into staging and the URNs are the same (expect the date/time as expected).