There is a good discussion to be had about how permissive or strict a parser for a file standard should be and if it is permissive which errors in the format should be tolerated and which not. To me, the answer to this question for the case of SDRF files is not yet clear and I would welcome a discussion about that from contributors and users of sdrf-pipelines. It follows a list of questions (not comprehensive, at all):
What are permissible errors?
Can a trailing whitespace always be stripped? Or can a trailing whitespace have meaning?
Can an empty line be tolerated? At the beginning? At the end? In the middle?
Can we make valid assumptions about strings? Is the encoding UTF-8? Are file names supposed to be composed of only a limited charset?
Filenames?
Column names?
Fields in the SDRF table?
How thoroughly is the content checked?
Are empty fields allowed? Or filled with some value?
Do we need the same number of value X and Y in a column?
Is invalid content detected? e.g. labelling information in a fraction column?
Which detected issues are how severe?
What do they affect?
Should they be handled silently, trigger warning or raise an error?
Some of these questions have clear answers, others not so much. I would very much welcome a discussion around and about
These questions might also have different answers for different use cases. The SDRF is a tool expected to be applied in a broad range of environments and use cases. Discussing these questions will help us anticipate the requirements better and help in the design and implementation of the next iteration of the sdrf-pipelines package.
Since I am new to this project, such a discussion will also help me get going with contributions. Or in other words, keep me from straying into territories that are better left uncharted.
There is a good discussion to be had about how permissive or strict a parser for a file standard should be and if it is permissive which errors in the format should be tolerated and which not. To me, the answer to this question for the case of SDRF files is not yet clear and I would welcome a discussion about that from contributors and users of sdrf-pipelines. It follows a list of questions (not comprehensive, at all):
Some of these questions have clear answers, others not so much. I would very much welcome a discussion around and about
These questions might also have different answers for different use cases. The SDRF is a tool expected to be applied in a broad range of environments and use cases. Discussing these questions will help us anticipate the requirements better and help in the design and implementation of the next iteration of the
sdrf-pipelinespackage.Since I am new to this project, such a discussion will also help me get going with contributions. Or in other words, keep me from straying into territories that are better left uncharted.