search

brainlife / app-freesurfer-stats

0 stars 1 forks source link

Output validation error. #5

Closed jlhanson5 closed 3 years ago

jlhanson5 commented 3 years ago

Hello all-

I was attempting to use the following pipeline:

Previously, I could get this pipeline to work well, but recently on the last "Parcellation Statistics - Surface - Deprecated", I started getting validation errors. It looks like there's now an issue output-ing an roi.csv file that previously worked/was output.

I've attached the slurm.err and .log output. I wondered if others had run into this? slurm-18693.err.txt slurm-18693.log.txt

Any thoughts are much appreciated! Jamie.

francopestilli commented 3 years ago

@jlhanson5 is this being run locally or via BL?

jlhanson5 commented 3 years ago

via BL.

soichih commented 3 years ago

We recently added neuro/parc-stats validator and I am guessing that it's uncovering some inconsistencies with existing apps.

@bacaron This App outputs those csv files directly under ./parc-stats not ./parc-stats/parc-stats. (similar issue that you fixed recently for other app?)

image

Also, it's failing to find the rois.csv

Traceback (most recent call last):
  File "/export/prod/60c8c5632b8bed2c50835cfd/60c8c5642b8bedbcdd835d00/create_data_csv.py", line 103, in <module>
    rois = pd.read_csv('rois.csv')
  File "/usr/local/lib/python3.6/dist-packages/pandas/io/parsers.py", line 676, in parser_f
    return _read(filepath_or_buffer, kwds)
  File "/usr/local/lib/python3.6/dist-packages/pandas/io/parsers.py", line 448, in _read
    parser = TextFileReader(fp_or_buf, **kwds)
  File "/usr/local/lib/python3.6/dist-packages/pandas/io/parsers.py", line 880, in __init__
    self._make_engine(self.engine)
  File "/usr/local/lib/python3.6/dist-packages/pandas/io/parsers.py", line 1114, in _make_engine
    self._engine = CParserWrapper(self.f, **self.options)
  File "/usr/local/lib/python3.6/dist-packages/pandas/io/parsers.py", line 1891, in __init__
    self._reader = parsers.TextReader(src, **kwds)
  File "pandas/_libs/parsers.pyx", line 374, in pandas._libs.parsers.TextReader.__cinit__
  File "pandas/_libs/parsers.pyx", line 674, in pandas._libs.parsers.TextReader._setup_parser_source
FileNotFoundError: [Errno 2] File rois.csv does not exist: 'rois.csv'

from create_data.csv.py line 102.

# append subject ID to data with coordinates from dan's code
rois = pd.read_csv('rois.csv')
rois['subjectID'] = [ subjectID for x in range(len(rois['ROI_name'])) ]
rois = rois[ ['subjectID'] +  [ f for f in rois.columns if f != 'subjectID']] 
rois.to_csv('rois.csv',index=False)
bacaron commented 3 years ago

@jlhanson5 and @soichih,

Thank you for bringing my attention to this. I'm not exactly sure what happened, but it looks like some legacy code has snuck it's way into the app. That rois.csv file is better suited for the volumated stats, as it comes from some code that one of our group members wrote.

I've commented out those lines. @jlhanson5 could you please try it again and let me know if it fails? I'm terribly sorry for the delays.

Brad

bacaron commented 3 years ago

I believe this has been officially resolved.