bids-standard / legacy-validator

Validator for the Brain Imaging Data Structure
https://bids-standard.github.io/legacy-validator/
MIT License
186 stars 111 forks source link

deno: be able to trigger only specific testing #2144

Closed yarikoptic closed 1 month ago

yarikoptic commented 1 month ago

In our particular case we would like to "quickly" (i.e. more quick than running full validation) address HED issues for which we got

        [WARNING] HED_WARNING The validation on this HED string returned a warning.
                /sub-0001/ses-01/func/sub-0001_ses-01_task-alignvideo_acq-mb8_run-01_events.tsv - WARNING: [UNITS_MISSING] No unit specified. Using "m" as the default - "X-position/45.62". TSV line: 11. (For more information on this HED warning, see https://hed-specification.readthedocs.io/en/latest/Appendix_B.html#units-missing.)
                /sub-0001/ses-01/func/sub-0001_ses-01_task-alignvideo_acq-mb8_run-01_events.tsv - WARNING: [UNITS_MISSING] No unit specified. Using "m" as the default - "X-position/35.0". TSV line: 12. (For more information on this HED warning, see https://hed-specification.readthedocs.io/en/latest/Appendix_B.html#units-missing.)

                75698 more files with the same issue

        Please visit https://neurostars.org/search?q=HED_WARNING for existing conversations about this issue.

but it seems that there is no easy way ATM to instruct deno bids-validator to run only the tests concerning with HED validation.

VisLab commented 1 month ago

The example just has warnings. We would like to have a way of suppressing HED warnings for BIDS validation. We warn of a number of things like -- when you extend tags or there are column values that appear in a .tsv but don't appear in the JSON file containing annotations for other values in that column. None of these are HED errors --- really information rather than errors. Should we turn them off by default in the BIDS validator or is there a mechanism for specifying that HED warnings should be turned off?

FYI @happy5214

VisLab commented 1 month ago

but it seems that there is no easy way ATM to instruct deno bids-validator to run only the tests concerning with HED validation.

We would really like to have this feature so that users could more easily debug their HED independently. The other option is to run a separate HED validation as explained in hed-standard/hed-examples#405, but it would be ideal to allow the BIDS validator to run a HED validator step.

happy5214 commented 1 month ago

Is there no general way to suppress warnings? If not, all HED warnings fall under two issue codes: 107 (HED_INTERNAL_WARNING), which to my knowledge is entirely unused and probably should be deleted; and 105 (HED_WARNING), which includes all other (i.e. all) warnings.

yarikoptic commented 1 month ago

Thanks everyone for chiming in! I finally found the issue I was looking for when filing this one: