search

ISA-tools / isa-api

ISA tools API
https://isa-tools.org
Other
40 stars 38 forks source link

Add tracking to data file type column names 2. #553

Closed ptth222 closed 2 months ago

ptth222 commented 6 months ago

Rebasing the original branch for this did not go well, so I manually created another one. This PR should replace #510. I was able to get this to pass all of the tests and now multiple ".* File" columns won't cause an issue. We don't need this capability like I thought we did originally, but I figured I would go on and make it work anyway.

coveralls commented 6 months ago

Coverage Status

coverage: 81.341% (+0.08%) from 81.257% when pulling e50e65e6c8eb29706560e5b820de181ad07db9f2 on ptth222:fix-data-file-name-bug2 into 7d5f19f9c7d2738849b2ac62e6d3c71603da8151 on ISA-tools:issue-511.

proccaserra commented 6 months ago

as discussed today:

"NMR Assay Name" with "Free Induction Decay Data File" with 'protocol type' = "NMR spectroscopy" (or declared synonyms) "MS Assay Name" with "Raw Spectral Data File" with 'protocol type' = "mass spectrometry" (or declared synonyms)

ptth222 commented 6 months ago

I did an example more like what was discussed in the meeting. I had to also add tracking for the columns that are added for known protocol types. The problem that this PR solves is that for any ".* File" column, if there is more than 1 then the values don't get written out correctly. Solving this wasn't as easy as it was for a similar issue with Protocol REF columns. Due to how graph nodes were previously handled.

Old Table Output: Sample Name Protocol REF MS Assay Name Raw Data File Protocol REF Data Transformation Name Derived Data File Protocol REF Data Transformation Name Derived Data File
sample1 protocol1 process1 datafile1.raw protocol2 process3 datafile3.raw protocol3 process3 datafile3.raw
sample2 protocol1 process4 datafile4.raw protocol3 process5 datafile5.raw process5 datafile5.raw
New Table Output: Sample Name Protocol REF MS Assay Name Raw Data File Protocol REF Data Transformation Name Derived Data File Protocol REF Data Transformation Name Derived Data File
sample1 protocol1 process1 datafile1.raw protocol2 process2 datafile2.raw protocol3 process3 datafile3.raw
sample2 protocol1 process4 datafile4.raw protocol3 process5 datafile5.raw
proccaserra commented 6 months ago

PRS to test this situation, i.e., several files generated by one data acquisition:

Sample Name Protocol REF MS Assay Name Raw Data File Protocol REF Data Transformation Name Derived Data File Protocol REF Data Transformation Name Derived Data File
sample1 protocol1 process1 datafile1.raw protocol2 process2 datafile2.raw protocol3 process3 datafile3.raw
sample1 protocol1 process1 datafile4.raw protocol2 process2 datafile2.raw
ptth222 commented 4 months ago

I took a small break from what I got pulled away to work on to address the comments raised when we last met. I added comments and broke up _build_paths_and_indexes() into smaller pieces and removed all of the logic that identified data nodes with 'File'. Let me know if this is still not enough.

ptth222 commented 3 months ago

The only conflict is where I deleted a line in testing that was printing. Can this be resolved?

proccaserra commented 3 months ago

The only conflict is where I deleted a line in testing that was printing. Can this be resolved?

@ptth222 there is indeed an outstanding merge conflict but resolving it wouldn't allow to merge still as the PR does not solve the issue. We have not found a fix yet unfortunately

ptth222 commented 3 months ago

What issue? I addressed everything mentioned in the history above, and all of the tests pass. If there is an issue can we create a test for it?

proccaserra commented 2 months ago

@ptth222 finally managed to find time and issue causing the problem while merging into my local branch. All tests are now passing locally but need to investigate one more case. Apologies for the delay

proccaserra commented 2 months ago

@ptth222 closing this as your changes have been integrated in issue-511.