hed-standard / hed-python

Python validation, summary, and analysis tools for HED (Hierarchical Event Descriptors).
https://hed-python.readthedocs.io/en/latest/
MIT License
10 stars 10 forks source link

WARN, don't blow up on _events.tsv without events #1030

Closed yarikoptic closed 1 month ago

yarikoptic commented 1 month ago
❯ datalad run -m "Run hed-validator" hed-validator -o logs/hed-validator.log .
[INFO   ] == Command start (output follows) ===== 
Traceback (most recent call last):
  File "/home/yoh/proj/HED/hed-python/venvs/dev3/bin/hed-validator", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/yoh/proj/HED/hed-python/hed/scripts/hed_validator.py", line 27, in main
    issue_list = validate_dataset(args)
                 ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/yoh/proj/HED/hed-python/hed/scripts/hed_validator.py", line 41, in validate_dataset
    issue_list = bids.validate(check_for_warnings=args.check_for_warnings)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/yoh/proj/HED/hed-python/hed/tools/bids/bids_dataset.py", line 84, in validate
    issues += files.validate_datafiles(self.schema, check_for_warnings=check_for_warnings)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/yoh/proj/HED/hed-python/hed/tools/bids/bids_file_group.py", line 155, in validate_datafiles
    data_obj.set_contents(overwrite=False)
  File "/home/yoh/proj/HED/hed-python/hed/tools/bids/bids_tabular_file.py", line 31, in set_contents
    self._contents = TabularInput(file=self.file_path, sidecar=self.sidecar._contents,
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/yoh/proj/HED/hed-python/hed/models/tabular_input.py", line 40, in __init__
    super().__init__(file, file_type=".tsv", worksheet_name=None, has_column_names=True, mapper=new_mapper,
  File "/home/yoh/proj/HED/hed-python/hed/models/base_input.py", line 65, in __init__
    self._open_dataframe_file(file, has_column_names, input_type)
  File "/home/yoh/proj/HED/hed-python/hed/models/base_input.py", line 480, in _open_dataframe_file
    raise HedFileError(HedExceptions.INVALID_DATAFRAME, "Invalid dataframe(malformed datafile, etc)", file)
hed.errors.exceptions.HedFileError: ('INVALID_DATAFRAME', 'Invalid dataframe(malformed datafile, etc)', '/home/yoh/datasets/1076_spacetop/sub-0133/ses-04/func/sub-0133_ses-04_task-fractional_acq-mb8_run-01_events.tsv')
[INFO   ] == Command exit (modification check follows) ===== 
[INFO   ] The command had a non-zero exit code. If this is expected, you can save the changes with 'datalad save -d . -r -F .git/COMMIT_EDITMSG' 
run(error): /home/yoh/datasets/1076_spacetop (dataset) [hed-validator -o logs/hed-validator.log ...]
datalad run -m "Run hed-validator" hed-validator -o logs/hed-validator.log .  13.58s user 2.40s system 108% cpu 14.728 total

where the file it points to is

❯ cat /home/yoh/datasets/1076_spacetop/sub-0133/ses-04/func/sub-0133_ses-04_task-fractional_acq-mb8_run-01_events.tsv
onset   duration    subtask_type    event_type  value   response_accuracy   stim_file   button_press
❯ hexdump -C /home/yoh/datasets/1076_spacetop/sub-0133/ses-04/func/sub-0133_ses-04_task-fractional_acq-mb8_run-01_events.tsv
00000000  6f 6e 73 65 74 09 64 75  72 61 74 69 6f 6e 09 73  |onset.duration.s|
00000010  75 62 74 61 73 6b 5f 74  79 70 65 09 65 76 65 6e  |ubtask_type.even|
00000020  74 5f 74 79 70 65 09 76  61 6c 75 65 09 72 65 73  |t_type.value.res|
00000030  70 6f 6e 73 65 5f 61 63  63 75 72 61 63 79 09 73  |ponse_accuracy.s|
00000040  74 69 6d 5f 66 69 6c 65  09 62 75 74 74 6f 6e 5f  |tim_file.button_|
00000050  70 72 65 73 73 0a                                 |press.|
00000056

which is likely a legit warning for HED but nothing there really violates BIDS or HED, hence should not lead to a crash AFAIK.

VisLab commented 1 month ago

@yarikoptic agreed --- can you attach a small sample of the actual file that caused the crash so we can be sure we fixed.

yarikoptic commented 1 month ago

sure -- you can get it from http://www.oneukrainian.com/tmp/sub-0133_ses-04_task-fractional_acq-mb8_run-01_events.tsv

VisLab commented 1 month ago

Addressed with PR #1032