molgenis / molgenis-emx2

MOLGENIS EMX2, the latest version of the MOLGENIS data platform.
GNU Lesser General Public License v3.0
11 stars 16 forks source link

fix(backend): foreign key check when import ref when refTable has a composite key #3876

Closed BrendaHijmans closed 10 hours ago

BrendaHijmans commented 2 weeks ago

What version of EMX2 are you using (see footer) 10.81.2, not in for instance 10.60.1

Describe the bug When you make a reference from a table to another table which has a composite key, you are required to refer to all those composite keys when you import data. That data should be present in the table you are refering to. It is now possible to upload any data in these composite key references.

This excel contains broken data that can be uploaded: refBug_example.xlsx, see the red highlights in sheet Pet. The inserted data is also present in a download of table Pet after upload.

To Reproduce Steps to reproduce the behavior:

  1. Go to https://emx2.dev.molgenis.org/
  2. Log in
  3. Make a new schema
  4. Try to upload the provided Excel
  5. Faulty data is uploaded.

Expected behavior Import should fail with warning that "reference to Colours table is incorrect'"

Additional context Morris supposes this needs to be fixed in SqlTable where also other not null checks happen. I.e.

BrendaHijmans commented 2 weeks ago

See also this file, which is a download from a staging area on the data catalogue server: Variables (4).csv. There are multiple instances where the column 'collection event.name' is filled out, but 'collection event.resource' is not. This is very problematic behaviour.

mswertz commented 2 weeks ago

I think this has never worked. Can you check 10.60.1 again? I suppose that there it was a required field so then the error doesn't occur. Issue is that postgresql skips foreign checks when they contain nulls (don't ask me why). This is no issue for singular foreign keys but is for composite foreign keys.