molgenis / projects-rd-cosas

0 stars 0 forks source link

COSAS Mapping Script Fixes #31

Closed davidruvolo51 closed 2 years ago

davidruvolo51 commented 2 years ago

I noticed that there are a couple of issues in the nightly mapping job. These occur during the import step, but there are likely minor changes in the nightly export that have effected the processing script.

Issues

Improvments

While I'm at it, I might as well fix a few things that have been bothering me.

davidruvolo51 commented 2 years ago

For Null id error when importing data into the subjects table. Add the following filter when the subjectsToRegister object is binded to the main subjects dataset.

-subjects = dt.rbind(...)[...][...]
+subjects = dt.rbind(...)[...][...][f.subjectID != None, :]
davidruvolo51 commented 2 years ago

Errors with xref attributes were likely due to the Null id error. All identifiers exist in the raw/processed data, but the import failed as the subjects data failed to import properly.