search

PennLINC / aslprep

Preprocessing of arterial spin labeling (ASL) data
https://aslprep.readthedocs.io
Other
41 stars 15 forks source link

unhelpful crash with UnicodeDecodeError if json sidecar has non-ascii character #180

Open dmd opened 2 years ago

dmd commented 2 years ago

Hi,

At my site, someone edited a nifti json sidecar and accidentally inserted a smart-quote. I obviously don't expect ASLprep to run, but it would be really nice if when it crashes it were to tell you why - i.e., at least say what file is causing it to choke.

[pkumar@micc NIFTItest]$ singularity run -B /data/pkumar/ --cleanenv /data/singularity-images/aslprep-0.2.7.simg /data/pkumar/MFS/Imaging_
Data/NIFTItest /data/pkumar/asdf participant --participant-label sub-MFS113 -w /data/pkumar/asdfw --fs-license-file /data/pkumar/MFS/Imagi
ng_Data/license.txt --basil --output-spaces MNI152NLin6Asym:res-2 --scorescrub --dummy-vols 4 --skip-bids-validation --use-syn-sdc
Traceback (most recent call last):
  File "/usr/local/miniconda/bin/aslprep", line 10, in <module>
    sys.exit(main())
  File "/usr/local/miniconda/lib/python3.7/site-packages/aslprep/cli/run.py", line 17, in main
    parse_args()
  File "/usr/local/miniconda/lib/python3.7/site-packages/aslprep/cli/parser.py", line 496, in parse_args
    config.from_dict(vars(opts))
  File "/usr/local/miniconda/lib/python3.7/site-packages/aslprep/config.py", line 537, in from_dict
    execution.load(settings)
  File "/usr/local/miniconda/lib/python3.7/site-packages/aslprep/config.py", line 185, in load
    cls.init()
  File "/usr/local/miniconda/lib/python3.7/site-packages/aslprep/config.py", line 395, in init
    "derivatives", re.compile(r'^\.')))
  File "/usr/local/miniconda/lib/python3.7/site-packages/bids/layout/layout.py", line 288, in __init__
    indexer.index_metadata()
  File "/usr/local/miniconda/lib/python3.7/site-packages/bids/layout/index.py", line 234, in index_metadata
    payload = json.load(handle)
  File "/usr/local/miniconda/lib/python3.7/json/__init__.py", line 293, in load
    return loads(fp.read(),
  File "/usr/local/miniconda/lib/python3.7/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x93 in position 3: invalid start byte
geowk commented 2 years ago

Hello,

I receive a similar messsage. Aslprep runs for some time. It does registration, calculates dvars and carpet plots and then I get an error message about Preprocessing not finishing.

Here is log. Our json noted there was no vascular crushing, but in the aslprep processing log it says true.

Not sure what is going and we're having a difficult time pinpointing the problem. The data is BIDS validated. Any help would be appreciated.

Thanks.

g.


Node: aslprep_wf.single_subject_1006_wf.asl_preproc_ses_01_acq_HEAT_run_01_wf.compt_cbf_wf.extractcbf
Working directory: /work/aslprep_wf/single_subject_1006_wf/asl_preproc_ses_01_acq_HEAT_run_01_wf/compt_cbf_wf/extractcbf

Node inputs:

asl_file = /work/aslprep_wf/single_subject_1006_wf/asl_preproc_ses_01_acq_HEAT_run_01_wf/asl_asl_trans_wf/merge/vol0000_xform-00000_merged.nii.gz
bids_dir = /data/sub-1006
dummy_vols = 0
fwhm = 5.0
in_file = /data/sub-1006/ses-01/perf/sub-1006_ses-01_acq-HEAT_run-01_asl.nii
in_mask = /work/aslprep_wf/single_subject_1006_wf/asl_preproc_ses_01_acq_HEAT_run_01_wf/compt_cbf_wf/refinemask/ref_bold_corrected_brain_mask_maths_refinemask.nii.gz
in_metadata = {'AcquisitionMatrixPE': 76, 'AcquisitionNumber': 1, 'AcquisitionTime': '13:59:48.380000', 'AcquisitionVoxelSize': [3.026316, 3.026316, 6.0], 'ArterialSpinLabelingType': 'PCASL', 'BackgroundSuppression': True, 'BandwidthPerPixelPhaseEncode': 53.706, 'BaseResolution': 76, 'BodyPartExamined': 'BRAIN', 'ConsistencyInfo': 'N4_VE11C_LATEST_20160120', 'ConversionSoftware': 'dcm2niix', 'ConversionSoftwareVersion': 'v1.0.20200331', 'DerivedVendorReportedEchoSpacing': 0.000489997, 'DeviceSerialNumber': '167023', 'DwellTime': 2.5e-06, 'EchoTime': 0.013, 'EchoTrainLength': 37, 'EffectiveEchoSpacing': 0.000244999, 'FlipAngle': 90, 'ImageOrientationPatientDICOM': [0.995717, -0.0711251, -0.0590702, 0.0486987, 0.946557, -0.318839], 'ImageType': ['ORIGINAL', 'PRIMARY', 'M', 'ND', 'NORM', 'MOSAIC'], 'ImagingFrequency': 123.24, 'InPlanePhaseEncodingDirectionDICOM': 'COL', 'InstitutionAddress': 'Street', 'InstitutionName': 'University', 'InstitutionalDepartmentName': 'Department', 'LabelOffset': 90, 'LabelingDuration': 1.8, 'M0Type': 'Absent', 'MRAcquisitionType': '2D', 'MagneticFieldStrength': 3, 'Manufacturer': 'Siemens', 'ManufacturersModelName': 'Prisma_fit', 'MeanGzx10': 6, 'Modality': 'MR', 'NumRFBlocks': 98, 'ParallelReductionFactorInPlane': 2, 'PartialFourier': 1, 'PatientPosition': 'HFS', 'PercentPhaseFOV': 100, 'PercentSampling': 100, 'PhaseEncodingDirection': 'j-', 'PhaseEncodingSteps': 75, 'PhaseResolution': 1, 'PhiAdjust': 100, 'PixelBandwidth': 2630, 'PostLabelDelay': 1.2, 'PostLabelingDelay': 1.2, 'ProcedureStepDescription': 'Seminowicz_MANTRA', 'ProtocolName': 'ep2d_pcasl_HEAT1', 'PulseSequenceDetails': '%CustomerSeq%_ep2d_pcasl_ve11c', 'RFGap': 0.00036, 'ReceiveCoilActiveElements': 'HC1-7;NC1', 'ReceiveCoilName': 'HeadNeck_64', 'ReconMatrixPE': 76, 'RefLinesPE': 24, 'RepetitionTime': 4, 'RepetitionTimePreparation': 4.0, 'SAR': 0.257949, 'ScanOptions': 'FS', 'ScanningSequence': 'EP', 'SequenceName': 'epfid2d1_76', 'SequenceVariant': 'SK', 'SeriesDescription': 'ep2d_pcasl_HEAT1', 'SeriesNumber': 5, 'ShimSetting': [3206, -11943, 6685, 430, -26, -66, -25, -126], 'SliceThickness': 5, 'SliceTiming': [0.3725, 0, 0.41, 0.0375, 0.445, 0.075, 0.4825, 0.11, 0.52, 0.1475, 0.5575, 0.185, 0.595, 0.2225, 0.6325, 0.26, 0.67, 0.2975, 0.7075, 0.335], 'SoftwareVersions': 'syngo_MR_E11', 'SpacingBetweenSlices': 6, 'StationName': 'MRC35035', 'TotalReadoutTime': 0.0183749, 'TxRefAmp': 226.704, 'VascularCrushing': True}
out_avg = <undefined>
out_file = <undefined>

Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/plugins/multiproc.py", line 344, in _send_procs_to_workers
    self.procs[jobid].run(updatehash=updatehash)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 521, in run
    result = self._run_interface(execute=True)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 639, in _run_interface
    return self._run_command(execute)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 751, in _run_command
    f"Exception raised while executing Node {self.name}.\n\n{result.runtime.traceback}"
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node extractcbf.

Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/core.py", line 398, in run
    runtime = self._run_interface(runtime)
  File "/usr/local/miniconda/lib/python3.7/site-packages/aslprep/interfaces/cbf_computation.py", line 99, in _run_interface
    idasl = pd.read_csv(aslcontext1)['volume_type'].tolist()
  File "/usr/local/miniconda/lib/python3.7/site-packages/pandas/util/_decorators.py", line 311, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/pandas/io/parsers/readers.py", line 586, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "/usr/local/miniconda/lib/python3.7/site-packages/pandas/io/parsers/readers.py", line 482, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/usr/local/miniconda/lib/python3.7/site-packages/pandas/io/parsers/readers.py", line 811, in __init__
    self._engine = self._make_engine(self.engine)
  File "/usr/local/miniconda/lib/python3.7/site-packages/pandas/io/parsers/readers.py", line 1040, in _make_engine
    return mapping[engine](self.f, **self.options)  # type: ignore[call-arg]
  File "/usr/local/miniconda/lib/python3.7/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 69, in __init__
    self._reader = parsers.TextReader(self.handles.handle, **kwds)
  File "pandas/_libs/parsers.pyx", line 542, in pandas._libs.parsers.TextReader.__cinit__
  File "pandas/_libs/parsers.pyx", line 642, in pandas._libs.parsers.TextReader._get_header
  File "pandas/_libs/parsers.pyx", line 843, in pandas._libs.parsers.TextReader._tokenize_rows
  File "pandas/_libs/parsers.pyx", line 1917, in pandas._libs.parsers.raise_parser_error
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 78: invalid start byte`
tsalo commented 1 year ago

@dmd Shouldn't this be checked by the BIDS validator? Why did you include the --skip-bids-validation? If you don't include that, does it fail with a more interpretable message at the beginning?

@geowk the exception seems to come from reading your aslcontext.tsv file, rather than the sidecar JSON file.