Retaining foreign record keys in iRecord

[DavidHepper]

How best to can I retain foreign record keys through iRecord? I believe this was also a recent issue for Natural England keys: see #304.

Example: BDS has received 100,000 dragonfly records from BTO for import and verification in iRecord. Each has a source survey (either BirdTrack or Garden BirdWatch) and source record key that needs to be preserved separately from the iRecord record ID. Both fields need to be propagated to the Atlas, too.

Implementation: This should mean simply adding two columns to the 'DRN dragonfly imports' CSV format, with Null/Empty being the default for ordinary records. If I lump these values into the existing Comments field they won't be searchable. I can't do more towards importing BTO records until this is resolved.

[kitenetter]

@DavidHepper I don't see any problem with adding these two columns to the DRN imports format. The more difficult question is how they can then be included in the atlas export. I'm happy to look into this but cannot do so before week beginning 14 May, sorry.

[DavidHepper]

Thank you. I suggest that the same fields as chosen for the NE records, as discussed in #304, should be used in the export to the Atlas for ours, too.

[DavidHepper]

How can I go about this? Please tell me which column headings I should add to the DRN imports CSV. I didn't follow all of the discussion in #304 .

[johnvanbreda]

@DavidHepper in my opinion you should consider importing the record key into occurrence -> external key. This will then automatically appear in the Atlas export as OtherCatalogNumbers.

If you then add the survey reference code (ID 1144) to the DRN imports dataset then you can use this to import the source survey. This will then appear in the Atlas as DatasetID.

[DavidHepper]

@johnvanbreda, yes I'll take your advice on this. Please make Occurrence::External Key and Occurrence::Survey Reference Code available to Survey DRN dragonfly imports. I'm expecting to be able to upload Keys of at least 12 alphanumerics and Survey Refs of "BTO BT" (for BirdTrack) or "BTO GBW" (for Garden BirdWatch) unless these are relational and I need to conform with different codes.

[johnvanbreda]

@DavidHepper I've added the survey reference code to the DRN dragonfly import dataset - note that it is a sample attribute, not an occurrence attribute. You should find that occurrence:external_key is already available as it is a core field.

[DavidHepper]

Testing these imports (in Training Mode) but am unable to match either of these fields on survey 'DRN dragonfly imports'. All the usual fields match automatically whereas the new fields on the CSV trigger a manual match and are both absent from the 'Maps to attribute' drop-down lists. I'm stuck. Is this a regression problem or were they never enabled fully?

[kitenetter]

@DavidHepper I think this is now working - there is a filter in the iRecord import form settings that also needed adjusting.

[DavidHepper]

@kitenetter - thanks; nearly there! My test record imported without a squeak. Example ID: 8489835 (Training Mode). I can see the Survey Reference Code set to "BTO BT" as expected but External Key (which should be "OBS181992351") is missing. Is this a permissions issue or is the display form not set to show it to verifiers? (There is a field on the Comments tab labelled External reference or other source but this is blank.)

[DavidHepper]

@kitenetter - Please let us see External Key after import - see above. It's the last step of the process. (I'm unable to reopen this issue in GitHub.)

[kitenetter]

@DavidHepper where is it that you need to see it, are we talking about the "details" pane on the right of the verification grid, or something else?

[DavidHepper]

Yes, showing in 'details' at Verify is the most important, so that verifiers can query with BTO by quoting a record ID that BTO will recognise. When this is done I'll have confidence to do the big import.

These two fields should be shown in the details at Explore and must be eligible for export to NBN, too.

[kitenetter]

I'll have to ask @johnvanbreda to advise on the verifiers view, I'm not sure what controls that.

I'll add the NBN export issue to the list of things we need to sort out for exporting BDS data, will get back to you as soon as I can.

[johnvanbreda]

I've added the external key to the verifier details pane when present - the survey reference code should already have been there. The external key is included in the NBN Atlas export under OtherCatalogNumbers.

[DavidHepper]

Excellent! I'll test one more and then import the large BTO dataset.