extra empty header/column in logfile

[CerenB]

Describe your problem in detail

I cannot pass the bids-validator with the logfile of visual motion localiser output.

Describe what you got instead

Bids validator throws an error

Error message

File Path: All rows must have the same number of columns as there are headers.

    Type:       Error
    File:       sub-001_ses-001_task-auditoryLocalizer_run-001_events.tsv
    Location:       raw/sub-001/ses-001/func/sub-001_ses-001_task-auditoryLocalizer_run-001_events.tsv
    Reason:     All rows must have the same number of columns as there are headers.
    Line:       2
    Evidence:   row 1: 5.543655 1.200499    static  -1.000000   0.000000    0.000000    1.000000    1.000000    n/a 

    Type:       Error
    File:       sub-001_ses-001_task-visualLocalizer_run-001_events.tsv
    Location:       raw/sub-001/ses-001/func/sub-001_ses-001_task-visualLocalizer_run-001_events.tsv
    Reason:     All rows must have the same number of columns as there are headers.
    Line:       2
    Evidence:   row 1: 5.320559 1.217551    static  -1.000000   15.000000   0.000000    1.000000    1.000000    n/a 

======================================================

File Path: Empty cell in TSV file detected: The proper way of labeling missing values is "n/a".

    Type:       Error
    File:       sub-001_ses-001_task-auditoryLocalizer_run-001_events.tsv
    Location:       raw/sub-001/ses-001/func/sub-001_ses-001_task-auditoryLocalizer_run-001_events.tsv
    Reason:     Missing value at column # 10
    Line:       2
    Evidence:   row 1: 5.543655 1.200499    static  -1.000000   0.000000    0.000000    1.000000    1.000000    n/a 

    Type:       Error
    File:       sub-001_ses-001_task-visualLocalizer_run-001_events.tsv
    Location:       raw/sub-001/ses-001/func/sub-001_ses-001_task-visualLocalizer_run-001_events.tsv
    Reason:     Missing value at column # 10
    Line:       2
    Evidence:   row 1: 5.320559 1.217551    static  -1.000000   15.000000   0.000000    1.000000    1.000000    n/a 

[CerenB]

heya! Since both @iqrashahzad14 and @fcerpe will start acquiring data, I want to ask this issue again.

@Remi-Gau @marcobarilari and whoever is using cpp-bids currently, when you try passing the bids validator, do you get error messages for the .tsv files? For the TMS project, I had issues with the last column being empty (see above). So I was deleting it. I wonder if someone else faced with the same issue and/or it is something to be changed in our cpp-bids repo? In my projects, I created a small script to read and write .tsv files, which was doing the trick. nothing complicated.

[iqrashahzad14]

I got the same error. But I am using @Ceren Battal @.***> code to re-organise my tsv.

Iqra

On Thu, Oct 28, 2021 at 2:38 PM CerenB @.***> wrote:

heya! Since both @iqrashahzad14 https://github.com/iqrashahzad14 and @fcerpe https://github.com/fcerpe will start acquiring data, I want to ask this issue again.

@Remi-Gau https://github.com/Remi-Gau @marcobarilari https://github.com/marcobarilari and whoever is using cpp-bids currently, when you try passing the bids validator, do you get error messages for the .tsv files? For the TMS project, I had issues with the last column being empty (see above). So I was deleting it. I wonder if someone else faced with the same issue and/or it is something to be changed in our cpp-bids repo? In my projects, I created a small script https://github.com/CerenB/motionTMS_fMRI_analysis/blob/master/src/organiseTsvFile.mto read and write .tsv files, which was doing the trick. nothing complicated.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cpp-lln-lab/CPP_BIDS/issues/174#issuecomment-953804470, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASBKONC66BD6GIRU5WW6IJ3UJFG6XANCNFSM46LQ3PUQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[Remi-Gau]

Ok I ran this test and indeed the validator does give error code 22 and 23 on the output of this.

https://github.com/cpp-lln-lab/CPP_BIDS/blob/dev/tests/manualTests/test_makeRawDataset.m

I need to check but this seems due to the fact that the tsvwrite adds an extra tab at the end of the line.

If that's the case then this is a bids matlab bug not a CPP BIDS bug.

Will investigate.

[Remi-Gau]

This seems to rules out a pure bids-matlab issue

logFile.onset = 5;
logFile.duration = 3;

bids.util.tsvwrite(fullfile(pwd, 'tmp.tsv'), logFile)

FID = fopen(fullfile(pwd, 'tmp.tsv'), 'r');
content = fread(FID, 19, 'uint8=>char')

content =

  19×1 char array

    'o'
    'n'
    's'
    'e'
    't'
    '   '
    'd'
    'u'
    'r'
    'a'
    't'
    'i'
    'o'
    'n'
    '↵'  < ---- no extra tab at the end of the line
    '5'
    '   '
    '3'
    '↵'

[Remi-Gau]

definitely an extra is added by CPP_BIDS

    cfg.subject.subjectNb = 1;
    cfg.subject.runNb = 1;

    cfg.task.name = 'testtask';

    cfg = createFilename(cfg);

    logFile = saveEventsFile('init', cfg);
    logFile = saveEventsFile('open', cfg, logFile);

    logFile.onset = 2;
    logFile.duration = 3;

    logFile = saveEventsFile('save', cfg, logFile);

    saveEventsFile('close', cfg, logFile);

    funcDir = fullfile(cfg.dir.outputSubject, cfg.fileName.modality);
    eventFilename = cfg.fileName.events;

    FID = fopen(fullfile(funcDir, eventFilename), 'r');
    content = fread(FID, 49, 'uint8=>char')

content =

  49×1 char array

    'o'
    'n'
    's'
    'e'
    't'
    '   '
    'd'
    'u'
    'r'
    'a'
    't'
    'i'
    'o'
    'n'
    '   '
    't'
    'r'
    'i'
    'a'
    'l'
    '_'
    't'
    'y'
    'p'
    'e'
    '↵'
    '2'
    '.'
    '0'
    '0'
    '0'
    '0'
    '0'
    '0'
    '   '
    '3'
    '.'
    '0'
    '0'
    '0'
    '0'
    '0'
    '0'
    '   '
    'n'
    '/'
    'a'
    '   '  <--- this extra tab should not be there
    '↵'

[Remi-Gau]

179 should have fixed this. Will close for now but reopen if this is not fixed.

I will update the submodule in the different repos we have (I really need to automate this).

I think that your localizers repos are diverging from the one the upstream repo of CPP_LAB: so you might have to update your submodule manually.

We will have to find a way to avoid our different repos diverging that much: I don't like it because it makes it hard to maintain.

[Remi-Gau]

@all-contributors please add @iqrashahzad14 for bug test

[allcontributors[bot]]

@Remi-Gau

I've put up a pull request to add @iqrashahzad14! :tada: