search

monarch-initiative / koza

Data transformation framework for LinkML data models
https://koza.monarchinitiative.org/
BSD 3-Clause "New" or "Revised" License
47 stars 4 forks source link

CSV Reader: Handle varying line lengths #122

Open glass-ships opened 4 months ago

glass-ships commented 4 months ago

Currently varying line lengths will raise an exception and hard fail.

We should probably make these warnings and report out which lines vary.

Could also create a custom exception and allow the client code to determine what to do here

@kevinschaper do you have any additional insight into this issue?

caufieldjh commented 4 months ago

Does this include cases in which the transform parses multiple input files and column headings vary between the two (but they're all subsets of what the config specifies)? Or is that a different issue?

glass-ships commented 4 months ago

to be honest I'm not too sure, this was basically copy pasted from a TODO comment in the code. I'm hoping Kevin has a better idea of what was meant there