gbif / model-material

Data model research focused on richer data for a material catalogue
7 stars 8 forks source link

Specify: image duplication? #70

Closed MortenHofft closed 1 year ago

MortenHofft commented 1 year ago

I notice that this IMAGE OF,dcc04c84-1ed3-11e3-bfac-90b11c41863e is repeated 54 times in the entity relations file.

Given that https://ichthyology.specify.ku.edu/specify/view/collectionobject/38640/ (which have that GUID) only has 9 images it looks wrong. It isn't illegal, but it does look like an error.

acbentley commented 1 year ago

That is duplication caused by the Collecting Event images and the 1:M relationship to collection objects. I have turned on the distinct toggle in the query to get rid of that duplication - hopefully - in the below ENTITY_RELATIONSHIP.csv

acbentley commented 1 year ago

However, important to note in the example record you gave that there are 9 collection object images, 1 collecting event image, and 6 sequences so that record should still be duplicated 16 times across the three concatenated files. If it had a CT scan it would be duplicated another 1 time too.

tucotuco commented 1 year ago

The EntityRelationship record would not be duplicated 16 times (15 actually, because we can't connect to the collecting_event in this version), the entity_relationship_type='IMAGE OF'/subject_entity_id combination would be duplicated.

acbentley commented 1 year ago

Yes, sorry, that is what I meant. However, collecting event images have still been included - just linked to the collection objects and thus duplicated based on the number of collection objects that are connected to the collecting event. Does that not work?

tucotuco commented 1 year ago

The schema doesn't actually prevent it, but it should. There should be a UNIQUE constraint

ALTER TABLE entity_relationship ADD CONSTRAINT entity_relationship_unique_key UNIQUE(depends_on_entity_relationship_id, subject_entity_id, entity_relationship_type, object_entity_id, object_entity_iri, entity_relationship_date);

Before I commit such a change, can I get confirmation from @MortenHofft and @timrobertson100 ?

acbentley commented 1 year ago

Looking through the issues it looks like there are some open Specify questions but no open issues as such. Am I to assume that we are now good to go and that all files are in order? @MortenHofft @tucotuco ?

tucotuco commented 1 year ago

I defer to @MortenHofft on this question, as he has been systematically trying to load the data and test them in a UI.

MortenHofft commented 1 year ago

I will close this issue - feel free to open if any of you believe that you haven't changed what you needed to.