FDA files

[antoniohupa]

Hi Mark!

First of all, congrats for developing code to read .fda files. You made a great job! I previously used your tool to extract .fds files to png but now it would be even more helpful to me to work directly with .fda (as it is the default file for storage in my hospital) without intermediate transformations. I have downloaded your code but I'm having some problems to read the images. Applying your example codes to one of my .fda files it outputs the following error:

""""" 2 ----> 3 from oct_converter.readers import FDA 4 5 # a sample .fda file can be downloaded from the Biobank resource here:

ImportError: cannot import name 'FDA' from 'oct_converter.readers' (/Users/antonio_mac/OCT-Converter/oct_converter/readers/init.py) """"""" I have reinstalled all the code but it didn't help.. I hope you can help me cause making this code work would be great for me..

Thanks in advance!

[marksgraham]

Hi there, I can't reproduce this issue. Is there a fda.py file in your readers folder? Are you able to extract .fds files using the fds example, after reinstallation?

[antoniohupa]

Hi Mark

Yes, there's a fda.py file in the readers folder but my older code for .fds files extraction don't run after reinstallation..

[antoniohupa]

There's the output:

---

ModuleNotFoundError Traceback (most recent call last) /anaconda3/lib/python3.7/site-packages/pylibjpeg/init.py in 34 try: ---> 35 import pydicom.pixel_data_handlers.pylibjpeg_handler 36 except ImportError:

ModuleNotFoundError: No module named 'pydicom.pixel_data_handlers.pylibjpeg_handler'

During handling of the above exception, another exception occurred:

ValueError Traceback (most recent call last)

in ----> 1 from oct_converter.readers import FDS ~/OCT-Converter/oct_converter/readers/__init__.py in 3 from .e2e import E2E 4 from .dcm import Dicom ----> 5 from .fda import FDA ~/OCT-Converter/oct_converter/readers/fda.py in 2 import numpy as np 3 from oct_converter.image_types import OCTVolumeWithMetaData, FundusImageWithMetaData ----> 4 from pylibjpeg import decode 5 from pathlib import Path 6 /anaconda3/lib/python3.7/site-packages/pylibjpeg/__init__.py in 35 import pydicom.pixel_data_handlers.pylibjpeg_handler 36 except ImportError: ---> 37 add_handler() 38 39 except ImportError: /anaconda3/lib/python3.7/site-packages/pylibjpeg/utils.py in add_handler() 19 If *pydicom* is not available. 20 """ ---> 21 from .pydicom import pixel_data_handler as handler 22 import pydicom.config 23 /anaconda3/lib/python3.7/site-packages/pylibjpeg/pydicom/pixel_data_handler.py in 55 } 56 ---> 57 _DECODERS = get_pixel_data_decoders() 58 SUPPORTED_TRANSFER_SYNTAXES = list(_DECODERS.keys()) 59 /anaconda3/lib/python3.7/site-packages/pylibjpeg/pydicom/utils.py in get_pixel_data_decoders() 34 return { 35 val.name: val.load() ---> 36 for val in iter_entry_points('pylibjpeg.pixel_data_decoders') 37 } 38 /anaconda3/lib/python3.7/site-packages/pylibjpeg/pydicom/utils.py in (.0) 34 return { 35 val.name: val.load() ---> 36 for val in iter_entry_points('pylibjpeg.pixel_data_decoders') 37 } 38 /anaconda3/lib/python3.7/site-packages/pkg_resources/__init__.py in load(self, require, *args, **kwargs) 2444 if require: 2445 self.require(*args, **kwargs) -> 2446 return self.resolve() 2447 2448 def resolve(self): /anaconda3/lib/python3.7/site-packages/pkg_resources/__init__.py in resolve(self) 2450 Resolve the entry point from its module and attrs. 2451 """ -> 2452 module = __import__(self.module_name, fromlist=['__name__'], level=0) 2453 try: 2454 return functools.reduce(getattr, self.attrs, module) /anaconda3/lib/python3.7/site-packages/openjpeg/__init__.py in 1 """Set package shortcuts.""" 2 ----> 3 from ._config import DICOM_ENCODERS, DICOM_DECODERS 4 from ._version import __version__ 5 from .utils import decode, decode_pixel_data, get_parameters /anaconda3/lib/python3.7/site-packages/openjpeg/_config.py in 1 ----> 2 from openjpeg.utils import decode_pixel_data 3 4 5 DICOM_DECODERS = { /anaconda3/lib/python3.7/site-packages/openjpeg/utils.py in 4 import warnings 5 ----> 6 import _openjpeg 7 8 /anaconda3/lib/python3.7/site-packages/openjpeg/_openjpeg.pyx in init _openjpeg() ValueError: numpy.ufunc size changed, may indicate binary incompatibility. Expected 216 from C header, got 192 from PyObject

[marksgraham]

Could you try a clean install of the converter on a fresh virtual environment?

On Tue, 17 Nov 2020 at 16:45, Antonio notifications@github.com wrote:

There's the output:

ModuleNotFoundError Traceback (most recent call last) /anaconda3/lib/python3.7/site-packages/pylibjpeg/init.py in 34 try: ---> 35 import pydicom.pixel_data_handlers.pylibjpeg_handler 36 except ImportError:

ModuleNotFoundError: No module named 'pydicom.pixel_data_handlers.pylibjpeg_handler'

During handling of the above exception, another exception occurred:

ValueError Traceback (most recent call last) in ----> 1 from oct_converter.readers import FDS

~/OCT-Converter/oct_converter/readers/init.py in 3 from .e2e import E2E 4 from .dcm import Dicom ----> 5 from .fda import FDA

~/OCT-Converter/oct_converter/readers/fda.py in 2 import numpy as np 3 from oct_converter.image_types import OCTVolumeWithMetaData, FundusImageWithMetaData ----> 4 from pylibjpeg import decode 5 from pathlib import Path 6

/anaconda3/lib/python3.7/site-packages/pylibjpeg/init.py in 35 import pydicom.pixel_data_handlers.pylibjpeg_handler 36 except ImportError: ---> 37 add_handler() 38 39 except ImportError:

/anaconda3/lib/python3.7/site-packages/pylibjpeg/utils.py in add_handler() 19 If pydicom is not available. 20 """ ---> 21 from .pydicom import pixel_data_handler as handler 22 import pydicom.config 23

/anaconda3/lib/python3.7/site-packages/pylibjpeg/pydicom/pixel_data_handler.py in 55 } 56 ---> 57 _DECODERS = get_pixel_data_decoders() 58 SUPPORTED_TRANSFER_SYNTAXES = list(_DECODERS.keys()) 59

/anaconda3/lib/python3.7/site-packages/pylibjpeg/pydicom/utils.py in get_pixel_data_decoders() 34 return { 35 val.name: val.load() ---> 36 for val in iter_entry_points('pylibjpeg.pixel_data_decoders') 37 } 38

/anaconda3/lib/python3.7/site-packages/pylibjpeg/pydicom/utils.py in (.0) 34 return { 35 val.name: val.load() ---> 36 for val in iter_entry_points('pylibjpeg.pixel_data_decoders') 37 } 38

/anaconda3/lib/python3.7/site-packages/pkg_resources/init.py in load(self, require, *args, *kwargs) 2444 if require: 2445 self.require(args, **kwargs) -> 2446 return self.resolve() 2447 2448 def resolve(self):

/anaconda3/lib/python3.7/site-packages/pkg_resources/init.py in resolve(self) 2450 Resolve the entry point from its module and attrs. 2451 """ -> 2452 module = import(self.module_name, fromlist=['name'], level=0) 2453 try: 2454 return functools.reduce(getattr, self.attrs, module)

/anaconda3/lib/python3.7/site-packages/openjpeg/init.py in 1 """Set package shortcuts.""" 2 ----> 3 from ._config import DICOM_ENCODERS, DICOM_DECODERS 4 from ._version import version 5 from .utils import decode, decode_pixel_data, get_parameters

/anaconda3/lib/python3.7/site-packages/openjpeg/_config.py in 1 ----> 2 from openjpeg.utils import decode_pixel_data 3 4 5 DICOM_DECODERS = {

/anaconda3/lib/python3.7/site-packages/openjpeg/utils.py in 4 import warnings 5 ----> 6 import _openjpeg 7 8

/anaconda3/lib/python3.7/site-packages/openjpeg/_openjpeg.pyx in init _openjpeg()

ValueError: numpy.ufunc size changed, may indicate binary incompatibility. Expected 216 from C header, got 192 from PyObject

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/marksgraham/OCT-Converter/issues/13#issuecomment-729054242, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB4UIM2XBBEGEU4HRDPQZTDSQKSC3ANCNFSM4TYXEZOA .

[antoniohupa]

I have reinstalled all fresh and I'm able to extract .fda files. I cannot save .avi cause it throws an error but that is not important for me at this point. Now what I cannot open is the .fds files.. outputs the following:

File 59286.fds contains the following chunks: b'_INFO_03\xb4\x00\x00\x00Triton plus\x00\x00\x00\x00\x00207093\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'

ValueError Traceback (most recent call last)

in 1 filepath = './59286.fds' 2 fds = FDS(filepath) ----> 3 oct_volume = fds.read_oct_volume() # returns an OCT volume with additional metadata if available 4 oct_volume.peek() # plots a montage of the volume 5 oct_volume.save('fds_testing.avi') # save volume as a movie ~/OCT-Converter/oct_converter/readers/fds.py in read_oct_volume(self) 84 """ 85 if b'@IMG_SCAN_03' not in self.chunk_dict: ---> 86 raise ValueError('Could not find OCT header @IMG_SCAN_03 in chunk list') 87 with open(self.filepath, 'rb') as f: 88 chunk_location, chunk_size = self.chunk_dict[b'@IMG_SCAN_03'] ValueError: Could not find OCT header @IMG_SCAN_03 in chunk list

[marksgraham]

Hm, the chunks do not look right. Were you able to open this file previously? Have you tried more than one .fds files - could it be this one is corrupted?

On Tue, 17 Nov 2020 at 17:37, Antonio notifications@github.com wrote:

I have reinstalled all fresh and I'm able to extract .fda files. I cannot save .avi cause it throws an error but that is not important for me at this point. Now what I cannot open is the .fds files.. outputs the following: File 59286.fds contains the following chunks: b'_INFO_03\xb4\x00\x00\x00Triton plus\x00\x00\x00\x00\x00207093\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'

ValueError Traceback (most recent call last) in 1 filepath = './59286.fds' 2 fds = FDS(filepath) ----> 3 oct_volume = fds.read_oct_volume() # returns an OCT volume with additional metadata if available 4 oct_volume.peek() # plots a montage of the volume 5 oct_volume.save('fds_testing.avi') # save volume as a movie

~/OCT-Converter/oct_converter/readers/fds.py in read_oct_volume(self) 84 """ 85 if b'@IMG_SCAN_03' not in self.chunk_dict: ---> 86 raise ValueError('Could not find OCT header @IMG_SCAN_03 in chunk list') 87 with open(self.filepath, 'rb') as f: 88 chunk_location, chunk_size = self.chunk_dict[b'@IMG_SCAN_03']

ValueError: Could not find OCT header @IMG_SCAN_03 in chunk list

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/marksgraham/OCT-Converter/issues/13#issuecomment-729086897, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB4UIM6BQ7U6U72Q4A3WMXTSQKYD7ANCNFSM4TYXEZOA .

[antoniohupa]

Hi Mark I'm almost sure that I could open that file previously. However, the code works with the example .fds file that can be downloaded from the UK biobank.. Let me try with a sort of new .fds files from my hospital and I'll come back to you...

[antoniohupa]

Hi again Mark

I tried to open other .fds files without any problem, so maybe the other file above was corrupted.

There's another issue I'm dealing with. I'm trying to extract the date of capture, the eye (left o right eye) and maybe the number of identification of the patient. I guess that information is contained in the chunk named '@FDA_FILE_INFO', isn't it? At least I can see the patient's name, so that's why I guess such information is there... Do you think it is possible to extract that information from that chunk? How could I do that? It is an important issue for me as I need to link the images with other information in the electronic medical record of the patients...

Thanks!

[marksgraham]

Hi there,

It should be possible to extract this info. If you have time to work on this now, I can try to give some guidance.

This wiki here has information on where various info is located: https://bitbucket.org/uocte/uocte/wiki/Topcon%20File%20Format

So for example, patient_id and name are in the chunk @PATIENT_INFO_02. The wiki is either a little out of date or is talking about a different file type to the one you are examining, because you have found the patient name in '@FDA_FILE_INFO', but with a bit of trial and error you the wiki should help you find the bits you need.

Also this issue points out that laterality is encoded in the first byte of @CAPTURE_INFO_02: https://bitbucket.org/uocte/uocte/issues/1/topcon-reader-should-read-sex-and

Give it a go and let me know how you get on. I can try and take a look too if you get stuck but I may not be able to find time to work on this for a little while.

Mark

On Thu, 19 Nov 2020 at 18:31, Antonio notifications@github.com wrote:

Hi again Mark

I tried to open other .fds files without any problem, so maybe the other file above was corrupted.

There's another issue I'm dealing with. I'm trying to extract the date of capture, the eye (left o right eye) and maybe the number of identification of the patient. I guess that information is contained in the chunk named '@FDA_FILE_INFO', isn't it? At least I can see the patient's name, so that's why I guess such information is there... Do you think it is possible to extract that information from that chunk? How could I do that? It is an important issue for me as I need to link the images with other information in the electronic medical record of the patients...

Thanks!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/marksgraham/OCT-Converter/issues/13#issuecomment-730558348, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB4UIMYB4WKLFPBRFHMKSRTSQVP7DANCNFSM4TYXEZOA .

[antoniohupa]

Hi Mark!

Thank for the info.

I'm almost done with it but I need to prove it in a larger sample of .fda files. Once it runs correctly I'll come back to you

Have a nice day!

Antonio