niftiwrapper IO

[bpinsard]

I ran into an issue with specific MPRAGE dicoms unenhanced from Philips 3D to 2D dicoms. Using dcmstack, I am able to parse, create niftiwrapper and save it, but when trying to load it using nibabel.load or dcmstack.NiftiWrapper.from_filename it raise a JSON error:

 dcmstack.NiftiWrapper.from_filename('test.nii')
/home_local/bpinsard/.virtualenvs/processing_virtenv/lib/python2.6/site-packages/dcmstack-0.7.dev-py2.6.egg/dcmstack/dcmmeta.pyc in from_filename(klass, path)
   1480             The path to the Nifti file to load.
   1481         '''
-> 1482         return klass(nb.load(path))
   1483 
   1484     @classmethod

/home_local/bpinsard/.virtualenvs/processing_virtenv/lib/python2.6/site-packages/nibabel/loadsave.pyc in load(filename)
     52         else:
     53             klass =  spm2.Spm2AnalyzeImage
---> 54     return klass.from_filename(filename)
     55 
     56 

/home_local/bpinsard/.virtualenvs/processing_virtenv/lib/python2.6/site-packages/nibabel/spatialimages.pyc in from_filename(klass, filename)
    409     def from_filename(klass, filename):
    410         file_map = klass.filespec_to_file_map(filename)
--> 411         return klass.from_file_map(file_map)
    412 
    413     @classmethod

/home_local/bpinsard/.virtualenvs/processing_virtenv/lib/python2.6/site-packages/nibabel/analyze.pyc in from_file_map(klass, file_map)
    868         hdr_fh, img_fh = klass._get_fileholders(file_map)
    869         hdrf = hdr_fh.get_prepare_fileobj(mode='rb')
--> 870         header = klass.header_class.from_fileobj(hdrf)
    871         if hdr_fh.fileobj is None: # was filename
    872             hdrf.close()

/home_local/bpinsard/.virtualenvs/processing_virtenv/lib/python2.6/site-packages/nibabel/nifti1.pyc in from_fileobj(klass, fileobj, endianness, check)
    580             extsize = hdr._structarr['vox_offset'] - fileobj.tell()
    581         byteswap = endian_codes['native'] != hdr.endianness
--> 582         hdr.extensions = klass.exts_klass.from_fileobj(fileobj, extsize, byteswap)
    583         return hdr
    584 

/home_local/bpinsard/.virtualenvs/processing_virtenv/lib/python2.6/site-packages/nibabel/nifti1.pyc in from_fileobj(klass, fileobj, size, byteswap)
    497             # a particular extension type
    498             try:
--> 499                 ext = extension_codes.handler[ecode](ecode, evalue)
    500             except KeyError:
    501                 # unknown extension type

/home_local/bpinsard/.virtualenvs/processing_virtenv/lib/python2.6/site-packages/nibabel/nifti1.pyc in __init__(self, code, content)
    261             # XXX or fail or at least complain?
    262             self._code = code
--> 263         self._content = self._unmangle(content)
    264 
    265     def _unmangle(self, value):

/home_local/bpinsard/.virtualenvs/processing_virtenv/lib/python2.6/site-packages/dcmstack-0.7.dev-py2.6.egg/dcmstack/dcmmeta.pyc in _unmangle(self, value)
    694         if sys.version_info >= (2, 7):
    695             kwargs['object_pairs_hook'] = OrderedDict
--> 696         return json.loads(value, **kwargs)
    697 
    698     def _mangle(self, value):

/usr/lib64/python2.6/json/__init__.pyc in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, **kw)
    305             parse_int is None and parse_float is None and
    306             parse_constant is None and not kw):
--> 307         return _default_decoder.decode(s)
    308     if cls is None:
    309         cls = JSONDecoder

/usr/lib64/python2.6/json/decoder.pyc in decode(self, s, _w)
    317 
    318         """
--> 319         obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    320         end = _w(s, end).end()
    321         if end != len(s):

/usr/lib64/python2.6/json/decoder.pyc in raw_decode(self, s, **kw)
    334         kw.setdefault('context', self)
    335         try:
--> 336             obj, end = self._scanner.iterscan(s, **kw).next()
    337         except StopIteration:
    338             raise ValueError("No JSON object could be decoded")

/usr/lib64/python2.6/json/scanner.pyc in iterscan(self, string, idx, context)
     53             action = actions[m.lastindex]
     54             if action is not None:
---> 55                 rval, next_pos = action(m, context)
     56                 if next_pos is not None and next_pos != matchend:
     57                     # "fast forward" the scanner

/usr/lib64/python2.6/json/decoder.pyc in JSONObject(match, context, _w)
    181         end = _w(s, end + 1).end()
    182         try:
--> 183             value, end = iterscan(s, idx=end, context=context).next()
    184         except StopIteration:
    185             raise ValueError(errmsg("Expecting object", s, end))

/usr/lib64/python2.6/json/scanner.pyc in iterscan(self, string, idx, context)
     53             action = actions[m.lastindex]
     54             if action is not None:
---> 55                 rval, next_pos = action(m, context)
     56                 if next_pos is not None and next_pos != matchend:
     57                     # "fast forward" the scanner

/usr/lib64/python2.6/json/decoder.pyc in JSONObject(match, context, _w)
    181         end = _w(s, end + 1).end()
    182         try:
--> 183             value, end = iterscan(s, idx=end, context=context).next()
    184         except StopIteration:
    185             raise ValueError(errmsg("Expecting object", s, end))

/usr/lib64/python2.6/json/scanner.pyc in iterscan(self, string, idx, context)
     53             action = actions[m.lastindex]
     54             if action is not None:
---> 55                 rval, next_pos = action(m, context)
     56                 if next_pos is not None and next_pos != matchend:
     57                     # "fast forward" the scanner

/usr/lib64/python2.6/json/decoder.pyc in JSONObject(match, context, _w)
    181         end = _w(s, end + 1).end()
    182         try:
--> 183             value, end = iterscan(s, idx=end, context=context).next()
    184         except StopIteration:
    185             raise ValueError(errmsg("Expecting object", s, end))

/usr/lib64/python2.6/json/scanner.pyc in iterscan(self, string, idx, context)
     53             action = actions[m.lastindex]
     54             if action is not None:
---> 55                 rval, next_pos = action(m, context)
     56                 if next_pos is not None and next_pos != matchend:
     57                     # "fast forward" the scanner

/usr/lib64/python2.6/json/decoder.pyc in JSONArray(match, context, _w)
    215     while True:
    216         try:
--> 217             value, end = iterscan(s, idx=end, context=context).next()
    218         except StopIteration:
    219             raise ValueError(errmsg("Expecting object", s, end))

/usr/lib64/python2.6/json/scanner.pyc in iterscan(self, string, idx, context)
     53             action = actions[m.lastindex]
     54             if action is not None:
---> 55                 rval, next_pos = action(m, context)
     56                 if next_pos is not None and next_pos != matchend:
     57                     # "fast forward" the scanner

/usr/lib64/python2.6/json/decoder.pyc in JSONObject(match, context, _w)
    183             value, end = iterscan(s, idx=end, context=context).next()
    184         except StopIteration:
--> 185             raise ValueError(errmsg("Expecting object", s, end))
    186         pairs[key] = value
    187         end = _w(s, end).end()

ValueError: Expecting object: line 274 column 46 (char 10553)

The nifti file is otherwise loaded when dcmstack is not in the library path. It is also opened in any other software. Any idea about what could cause this? Thanks.

[nicholsn]

@bpinsard, you might want to take a look at our fork of dcmstack at https://github.com/IBIC/dcmstack that has some support for Philips Enhanced DICOM. Hoping to get these changes incorporated into the main branch of dcmstack soon.

[moloney]

I haven't seen this before. Could you send me the JSON? For example you could do "nitool dump myfile.nii.gz > myjson.txt".

[bpinsard]

I had tried the guess-dim branch directly on the enhanced data but the same thing happens. here are the lines I extracted that make json library crash at loading:

            "DimensionIndexSequence": [
                {
                    "DimensionOrganizationUID": "2.16.124.113543.6006.99.02479373120167955565", 
                    "DimensionIndexPointer": (0020, 9056), 
                    "FunctionalGroupPointer": (0020, 9111)
                }, 
                {
                    "DimensionOrganizationUID": "2.16.124.113543.6006.99.02479373120167955565", 
                    "DimensionIndexPointer": (0020, 9057), 
                    "FunctionalGroupPointer": (0020, 9111)
                }
            ], 

maybe these self referential dicom tags code are the problem?

[moloney]

It looks like those tags are not being serialized correctly. What is the VR of those elements?

[bpinsard]

the VR is AT

[moloney]

Could you test the referenced branch, I think it should fix the issue. You will need to reconvert the files.

[bpinsard]

it works, thanks, maybe these tags should be filtered out as they do refers to dicoms ?

[moloney]

OK, merged it into master.

Currently, the default filtering is just intended to remove PHI. It should however be easy enough to filter any tag with a given VR if a user wants that.