search

CLOSER-Cohorts / archivist

Archivist is designed to be an easy to use editor that allows documention of questionnaires and data in DDI-Lifecycle
MIT License
17 stars 8 forks source link

"Copying" instrument fails due to slug and instrument ID #838

Closed HayleyMills closed 1 month ago

HayleyMills commented 2 months ago

There is no copy function in Archivist (maybe we could add this if straightforward), so I download an instrument and change the file name and the prefix in the file, then upload. This has caused the error https://closer-archivist-alspac.herokuapp.com/admin/imports/341

Record Invalid : PG::UniqueViolation: ERROR: duplicate key value violates unique constraint "index_codes_on_ddi_slug_and_instrument_id" DETAIL: Key (ddi_slug, instrument_id)=(, 1813) already exists.

Downloaded alspac_16_la24.txt

Edited alspac_16_la24_test.txt

simonreed commented 2 months ago

@HayleyMills the issue here was actually that the original export didn't have the correct URN in them. I've updated both staging and alspac so you should be able to now export it correctly.

HayleyMills commented 2 months ago

Tested again by exporting and importing the same instrument in alspac instance and the import worked.

Do I need to test the other instances, or will the fix need to be rolled out to the others?

HayleyMills commented 2 months ago

Tested on staging, creating new response domains, code lists, question items (question grids not working), question construct, condition, loop, statement and sequence, exported both xml and imported both. Both have errors

https://closer-archivist-staging.herokuapp.com/admin/imports/1258 https://closer-archivist-staging.herokuapp.com/admin/imports/1257

simonreed commented 2 months ago

@HayleyMills Can you try to export and import again on staging please.

HayleyMills commented 2 months ago

Will do - The same one, or a newly created one?

simonreed commented 2 months ago

Can you export the new same one and try to import it again. The exported XML should change.

HayleyMills commented 2 months ago

Stilled failed. https://closer-archivist-staging.herokuapp.com/admin/imports/1259 https://closer-archivist-staging.herokuapp.com/admin/imports/1260

simonreed commented 2 months ago

Which instrument did you export this from?

HayleyMills commented 2 months ago

The same one test111

simonreed commented 2 months ago

ok found something, bear with me

simonreed commented 2 months ago

@HayleyMills can you try to export and import again please on staging. You'll need to do fresh export as that's what is primarily affected.

HayleyMills commented 2 months ago

They uploaded successfully.

HayleyMills commented 2 months ago

Sorry reopening, as although it said it was successful, it didn't upload everything, it is missing a sequence and statement. it looks like it's in the xml which was exported.

simonreed commented 2 months ago

which staging instrument was this? could i get a link the import

HayleyMills commented 2 months ago

https://closer-archivist-staging.herokuapp.com/admin/imports/1264

If you look at the build totals and compare original https://closer-archivist-staging.herokuapp.com/instruments/test111/build/ with the imported one https://closer-archivist-staging.herokuapp.com/instruments/test111_4/build/

HayleyMills commented 2 months ago

@spuddybike to check export xml is valid.

test111_5.txt

test111_all.txt

HayleyMills commented 2 months ago

Issue related to Sequences being inside loops, they have the incorrect URN (one is padded with 000) and so the new slug ID was not correct either. HM to check that there are no occasions of this which have been exported and loaded into Discovery.

simonreed commented 1 month ago

@HayleyMills this is on staging if you'd like to try again?

HayleyMills commented 1 month ago

Re-exported and imported as _may, and all constructs are there.

HayleyMills commented 1 month ago

Double checked, and there are no issues in Discovery with missing sequences inside loops.