castuofa / dataarc-source

4 stars 5 forks source link

Field issues with Cairns and Icelandic Sagas #4

Open manonwilson opened 3 years ago

manonwilson commented 3 years ago

@ropitz For Mar Oliver's cairn data, there are two things happening that she says aren't in her data, so she doesn't understand why they are showing up in the field list. We need to remove the cairns_nw_iceland_id field from the cairn dataset. It is basically a duplicate field of cairns_nw_iceland_objectid except that the id starts with 0 and the objectid starts with 1. Also, the confidence field has negative numbers? Her data only shows positive 1-5. Angie and I asked a lot of questions about her dataset and there are no negative numbers and there are not two id fields. Chris has said that it appears it did come from her dataset upload. This is also true for two of Emily's fields: sagas_sagaid and sagas_saganame. They are duplicating sagas_id and sagas_name.

christopher-angel commented 3 years ago

For some clarification, the fields id and OBJECTID currently exist in the dataset. I'm not sure where these came from initially, but I am under the impression these were direct exports from the previous database.

If one of the semi-duplicate id fields, or both needs to be removed, is this something you'd prefer us to handle in this specific case? Trying to keep in mind longer term strategies for data updates/additions as well.

My only guess about the OBJECTID field is that it was generated by some GIS software during some export along the way before being imported into the previous database.

I also don't know what to do about the negative values. Essentially, I didn't want us to start making data decisions without a consult.

It's likely the real solution here is to extract and add the latest cairns data, via a PR, and that should solve this issue and others like it in the future.

ropitz commented 3 years ago

Yes, the objectid was generated via a GIS conversion and can be removed. Confidence : -1 is, I think, the Null value, so unranked ones.