Reading E2E, UnicodeDecode error

[ivanslootweg]

Hi,

First of all thank you for providing this tool.

I want to read an .E2E file with OCT data from a Heidelberg scanner and run into the following error:

  File "XX\ipykernel_36836\1681970171.py", line 4, in <module>
    file.read_oct_volume()
  File "XX\venv\lib\site-packages\oct_converter\readers\e2e.py", line 80, in read_oct_volume
    directory_chunk = e2e_binary.main_directory_structure.parse(raw)
  File "XX\venv\lib\site-packages\construct\core.py", line 289, in parse
    return self.parse_stream(io.BytesIO(data), **contextkw)
  File "XX\venv\lib\site-packages\construct\core.py", line 301, in parse_stream
    return self._parsereport(stream, context, "(parsing)")
  File "XX\venv\lib\site-packages\construct\core.py", line 313, in _parsereport
    obj = self._parse(stream, context, path)
  File "XX\venv\lib\site-packages\construct\core.py", line 2123, in _parse
    subobj = sc._parsereport(stream, context, path)
  File "XX\venv\lib\site-packages\construct\core.py", line 313, in _parsereport
    obj = self._parse(stream, context, path)
  File "XX\venv\lib\site-packages\construct\core.py", line 2656, in _parse
    return self.subcon._parsereport(stream, context, path)
  File "XX\venv\lib\site-packages\construct\core.py", line 313, in _parsereport
    obj = self._parse(stream, context, path)
  File "XXs\venv\lib\site-packages\construct\core.py", line 705, in _parse
    return self._decode(obj, context, path)
  File "XX\venv\lib\site-packages\construct\core.py", line 1612, in _decode
    return obj.decode(self.encoding)
UnicodeDecodeError: 'ascii' codec can't decode byte 0x96 in position 0: ordinal not in range(128)

What I have tried:

• read other .E2E files
• pass different encoding arguments to PaddedString() in main_directory_structure : Struct( "magic2" / PaddedString(12, "ascii"), "version" / Int32un, "unknown" / Array(10, Int16un), "num_entries" / Int32un, "current" / Int32un, "prev" / Int32un, "unknown3" / Int32un, )
• Find out the encoding type of the object with chardet. This yields the error: embedded null character

I am happy to learn about a solution to this.

Ivan

[marksgraham]

Hi,

You're not the first to report this issue, see here. In that case I made an update which seemed to help - I found it was necessary to skip some bytes until you find the CMDb string that signifies the start of the file, see here. Can you confirm you're using the latest version of the converter, 0.5.11?

If you are and it still isn't working, are you maybe able to share a file for me to take a look at?

[ivanslootweg]

Hi, My apologies for the duplicate issue. The update does not seem to resolve the problem as I encounter the same error. I will have to discuss what file information I can share with you. Would it be of help to send you (part of) the raw string?

Edit: I have a file I can share with you. Is it possible to send this over email?

[marksgraham]

Hi,

Yeah please send via email!

Best, Mark

[marksgraham]

Hi,

I've experimented a bit - could you try installing the PR with it+https://github.com/marksgraham/OCT-Converter.git@refs/pull/119/merge and let me know if it works? If so could you try on a few files?

[ivanslootweg]

HI,

Yes it works indeed. Thank you !

[marksgraham]

Ok - I'm going to leave it as that PR for now because I need to find a solution that works for all scans, so you'll have to install from them for now.

Mark

On Tue, Jul 25, 2023 at 1:04 PM Ivan Slootweg @.***> wrote:

HI,

Yes it works indeed. Thank you !

— Reply to this email directly, view it on GitHub https://github.com/marksgraham/OCT-Converter/issues/118#issuecomment-1649705563, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB4UIM4XLOEAZRTUUMU4CVDXR6Y5VANCNFSM6AAAAAA2TCDWWQ . You are receiving this because you commented.Message ID: @.***>

[marksgraham]

Hi @ivanslootweg

I think i've got it working more generally, and i've merged into main. It would be really helpful for me if you could try it out on a few files and let me know if it is still working.

[ivanslootweg]

Hi, sorry my delayed response. With the version on main branch I get the error: 'ascii' codec can't decode byte 0xd3 in position 0: ordinal not in range(128)'

edit: Is it possible to republish intermediate version of https://github.com/marksgraham/OCT-Converter.git@refs/pull/119/merg such that I can continue using the model? I am now stuck with the updated version

[marksgraham]

The current version on main is the version you tested and said worked... are you sure you have installed the latest version?

[ivanslootweg]

Indeed, all works now. Thank you!