marksgraham / OCT-Converter

Tools for extracting the raw optical coherence tomography (OCT) and fundus data from proprietary file formats.
https://pypi.org/project/oct-converter/
MIT License
199 stars 72 forks source link

Error with fds.read_oct_volume #29

Closed ghost closed 2 years ago

ghost commented 3 years ago

oct_volume = fds.read_oct_volume() Traceback (most recent call last): File "", line 1, in File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/oct_converter/readers/fds.py", line 86, in read_oct_volume raise ValueError('Could not find OCT header @IMG_SCAN_03 in chunk list') ValueError: Could not find OCT header @IMG_SCAN_03 in chunk list

I am not sure why I am getting this error. Any help would be very appreciated!

This is what I get when I do fds = FDS(filepath) b'@FDA_FILE_INFO' b'@HW_INFO_03' b'@PATIENT_INFO_03' b'@CAPTURE_INFO_02' b'@IMG_JPEG' b'@PARAM_SCAN_04' b'@IMG_TRC_02' b'@PARAM_TRC_02' b'@IMG_MOT_COMP_03' b'@IMG_PROJECTION' b'@REGIST_INFO' b'@ALIGN_INFO' b'@CONTOUR_INFO' b'@FAST_Q2_INFO' b'@THUMBNAIL' b'@GLA_LITTMANN_01' b'@PATIENTEXT_INFO' b'@EFFECTIVE_SCAN_RANGE' b'@MAIN_MODULE_INFO' b'@REPORT_INFO' b'@CONTOUR_MASK_INFO' b'@TOPQEXT_INFO' b'@PARAM_ANGIOGRAPHY' b'@REF_IMG_SCAN' b'@FOCUS_MOTOR_INFO' b'@ANGIO_TRACKING_INFO' b'@IMG_ANGIOGRAPHY_02' b'@TABIL_REGAVG_INFO'

marksgraham commented 3 years ago

Hi, Are you sure this is an OCT file - are you able to view OCT data in this file through other means, e.g. the Topcon viewer?

ghost commented 3 years ago

Yes, I am sure that it is a OCT file. A team member confirmed that the file can be viewed through TOPCON. I don't actually have access to TOPCON personally

antoniohupa commented 3 years ago

Hi dpuruwo

Looking at the chunks you're extracting from your file, I'm afraid the file is not a .fds but a .fda. Could you try with the fda reader?

ghost commented 3 years ago

@antoniohupa AHA! Good call... still getting an error though:

oct_volume = fda.read_oct_volume() Traceback (most recent call last): File "", line 1, in File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/oct_converter/readers/fda.py", line 110, in read_oct_volume slice = decode(raw_slice) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pylibjpeg/utils.py", line 68, in decode raise ValueError("Unable to decode the data") ValueError: Unable to decode the data

if I use fda.read_oct__volume_2() I get only 9 slices which seems a lot less than normal.

marksgraham commented 3 years ago

Hi, Would you be able to share a file for me to take a look at?

ghost commented 3 years ago

Yes, thank you for taking a look. Could I e-mail it to you?

marksgraham commented 3 years ago

Yeah - I think my email is on this github right?

On Wed, 23 Jun 2021 at 12:41, Daiana Pur @.***> wrote:

Yes, thank you for taking a look. Could I e-mail it to you?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/marksgraham/OCT-Converter/issues/29#issuecomment-866764370, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB4UIM6L67XBMKZILSIHQ6TTUHB7XANCNFSM466DTBLQ .

antoniohupa commented 3 years ago

@dpuruwo are you working under a virtual environment? I had a similar issue and re-setting my virtual environment resolved all the problems..

ghost commented 3 years ago

I am not, i'm using a mac with python 3.8

marksgraham commented 3 years ago

So might be worth starting a fresh virtual environment, installing of the requirements and trying then. Or as a quick diagnosis you could send me the scan and see if it works for me

marksgraham commented 3 years ago

I have a similar error to you when I try with your file. I also get 9 slices if I use read_oct_volume_2. Are you sure it is a conventional OCT? I noticed it has the chunk b'@IMG_ANGIOGRAPHY_02' in it. So I wonder if if is an angio and they have different structures that my code is unable to parse?

ghost commented 3 years ago

It is a conventional OCT but also includes angio, I will actually try to obtain one with just the conventional OCT including fundus. Thanks again for looking into it!

marksgraham commented 3 years ago

Would be good to try that. It may also be that you have a different version of Topcon scanner that encodes data differently - do you know what scanner you're using?

yvonneng996 commented 3 years ago

Hi Mark,

I also faced this problem where there is this b'@IMG_ANGIOGRAPHY_02' folders present and it shows 'Unable to decode the data'. Is there a need to change the code to under the read_oct_volume section?

Thank You

marksgraham commented 3 years ago

Hi there, It is possible these volumes with angio data have their OCT data encoded differently. Unfortunately, because I'm reliant on others reverse-engineering the Topcon file format rather than actual knowledge of the format, it is very hard to work out how the code might have to change in order to accomodate these new volumes. Mark

yvonneng996 commented 3 years ago

Hi Mark,

Is there a why to detect the structure where they are stored in chunks? Thank You

Yvonne

marksgraham commented 3 years ago

Hi,

If you're prepared to spend some time playing with a scan, it is sometimes possible to work our the structure of a chunk. It is likely that the OCT data in your scan is encoded in a pretty similar way to the OCT data my reader current works on, so a header of this format:

        self.oct_header = Struct(
            'type' / PaddedString(1, 'ascii'),
            'unknown1' / Int32un,
            'unknown2' / Int32un,
            'width' / Int32un,
            'height' / Int32un,
            'number_slices' / Int32un,
            'unknown3' / Int32un,
        )

This header seems to give me the correct height and width - they are the same values I get when I use read_oct_volume_2, which does sucesfully manage to extract a small number of slices, just not all of them. (Have you tried to use this function and see if it works for you?)

So it does seem to just be the jpeg decoding that is failing...

All the code I've written is based on this guide to the format found here

Mark

marksgraham commented 2 years ago

Closing as stale.