mhe / pynrrd

Simple pure-python module for reading and writing nrrd files.
https://pynrrd.readthedocs.io/
MIT License
116 stars 51 forks source link

Zlib Incorrect Data Check on Windows #29

Closed addisonElliott closed 7 years ago

addisonElliott commented 7 years ago

Hello,

Just wanted to say that I like this library and it has made my life a lot easier than having to write one myself.

I kept having the following issue when trying to read specific NRRD files from memory:

zlib.error: Error -3 while decompressing data: incorrect data check

I simplified my code down into the following that should trigger the error:

import os
import nrrd
import numpy as np

path = 'D:\\Users\\addis\\Desktop'
path2 = 'D:\\Users\\addis\\Desktop\\RectifyTest\\Subject0001_Final\\debug\\fatImageBiasCorrection\\'

headerDict1 = {}
headerDict2 = {'space': 'right-anterior-superior'}
headerDict3 = {'space': 'right-anterior-superior', 'space directions': ((1.0, 0.0, 0.0), (0.0, 1.0, 0.0), (0.0, 0.0, 1.0))}
headerDict4 = {'space': 'right-anterior-superior', 'space directions': ((1, 0, 0), (0, 1, 0), (0, 0, 1)), 'space origin': (100.0, 200.0, 300.0)}

image3 = np.random.random_sample((12, 12, 24))

def test(image):
    nrrd.write(os.path.join(path, 'testNRRD1.nrrd'), image, headerDict1)
    nrrd.write(os.path.join(path, 'testNRRD2.nrrd'), image, headerDict2)
    nrrd.write(os.path.join(path, 'testNRRD3.nrrd'), image, headerDict3)
    nrrd.write(os.path.join(path, 'testNRRD4.nrrd'), image, headerDict4)

    data, header = nrrd.read(os.path.join(path, 'testNRRD1.nrrd'))
    data, header = nrrd.read(os.path.join(path, 'testNRRD2.nrrd'))
    data, header = nrrd.read(os.path.join(path, 'testNRRD3.nrrd'))
    data, header = nrrd.read(os.path.join(path, 'testNRRD4.nrrd'))

print('Running test')
test(image3)

My machine is Windows 10 x64 with 8GB RAM.

Issue

After much debugging, I found the problem with the code. The problem is with the memory mapped section of code causing corruption. This problem is likely specific to Windows due to the "nt" if statement. This is probably why it was not caught.

    elif seek_past_header:
        # Efficiently find the header-data seperator line no matter how big
        # any header line is
        datafilehandle.seek(0)
        if os.name == "nt":
            m = mmap.mmap(datafilehandle.fileno(), 0, access=mmap.ACCESS_READ)
        else:
            m = mmap.mmap(datafilehandle.fileno(), 0,
                      mmap.MAP_PRIVATE, mmap.PROT_READ)

        seek_past_header_pos = m.find(b'\n\n')
        if seek_past_header_pos == -1:
            raise NrrdError('Invalid NRRD: Missing header-data separator line')
        datafilehandle.seek(seek_past_header_pos + 2)

I am confident this is the issue because setting seek_past_header to False makes the test case work.

I will submit a PR to fix this soon hopefully.