Stream Error when reading E2E file

[SaeedShurrab]

Hi Mark

Thank you so much for the great tool. I am trying to extract data from E2E file but I get an error when I read the file:

from oct_converter.readers import E2E
filepath = '76543017.E2E'
file = E2E(filepath)
oct_volumes =file.read_oct_volume() 

then I get the following response:

---------------------------------------------------------------------------
StreamError                               Traceback (most recent call last)
Cell In[13], line 1
----> 1 oct_volumes =file.read_oct_volume() 

File /usr/local/lib/python3.8/site-packages/oct_converter/readers/e2e.py:219, in E2E.read_oct_volume(self)
    217 f.seek(start)
    218 raw = f.read(60)
--> 219 chunk = self.chunk_structure.parse(raw)
    221 if chunk.type == 9:  # patient data
    222     raw = f.read(127)

File /usr/local/lib/python3.8/site-packages/construct/core.py:288, in Construct.parse(self, data, **contextkw)
    274 def parse(self, data, **contextkw):
    275     r"""
    276     Parse an in-memory buffer (often bytes object). Strings, buffers, memoryviews, and other complete buffers can be parsed with this method.
    277 
   (...)
    286     :raises ConstructError: raised for any reason
    287     """
--> 288     return self.parse_stream(io.BytesIO(data), **contextkw)

File /usr/local/lib/python3.8/site-packages/construct/core.py:300, in Construct.parse_stream(self, stream, **contextkw)
    298 context._params = context
    299 try:
--> 300     return self._parsereport(stream, context, "(parsing)")
    301 except CancelParsing:
    302     pass

File /usr/local/lib/python3.8/site-packages/construct/core.py:312, in Construct._parsereport(self, stream, context, path)
    311 def _parsereport(self, stream, context, path):
--> 312     obj = self._parse(stream, context, path)
    313     if self.parsed is not None:
    314         self.parsed(obj, context)

File /usr/local/lib/python3.8/site-packages/construct/core.py:2120, in Struct._parse(self, stream, context, path)
   2118 for sc in self.subcons:
   2119     try:
-> 2120         subobj = sc._parsereport(stream, context, path)
   2121         if sc.name:
   2122             obj[sc.name] = subobj

File /usr/local/lib/python3.8/site-packages/construct/core.py:312, in Construct._parsereport(self, stream, context, path)
    311 def _parsereport(self, stream, context, path):
--> 312     obj = self._parse(stream, context, path)
    313     if self.parsed is not None:
    314         self.parsed(obj, context)

File /usr/local/lib/python3.8/site-packages/construct/core.py:2653, in Renamed._parse(self, stream, context, path)
   2651 def _parse(self, stream, context, path):
   2652     path += " -> %s" % (self.name,)
-> 2653     return self.subcon._parsereport(stream, context, path)

File /usr/local/lib/python3.8/site-packages/construct/core.py:312, in Construct._parsereport(self, stream, context, path)
    311 def _parsereport(self, stream, context, path):
--> 312     obj = self._parse(stream, context, path)
    313     if self.parsed is not None:
    314         self.parsed(obj, context)

File /usr/local/lib/python3.8/site-packages/construct/core.py:703, in Adapter._parse(self, stream, context, path)
    702 def _parse(self, stream, context, path):
--> 703     obj = self.subcon._parsereport(stream, context, path)
    704     return self._decode(obj, context, path)

File /usr/local/lib/python3.8/site-packages/construct/core.py:312, in Construct._parsereport(self, stream, context, path)
    311 def _parsereport(self, stream, context, path):
--> 312     obj = self._parse(stream, context, path)
    313     if self.parsed is not None:
    314         self.parsed(obj, context)

File /usr/local/lib/python3.8/site-packages/construct/core.py:4832, in FixedSized._parse(self, stream, context, path)
   4830 if length < 0:
   4831     raise PaddingError("length cannot be negative", path=path)
-> 4832 data = stream_read(stream, length, path)
   4833 if self.subcon is GreedyBytes:
   4834     return data

File /usr/local/lib/python3.8/site-packages/construct/core.py:91, in stream_read(stream, length, path)
     89     raise StreamError("stream.read() failed, requested %s bytes" % (length,), path=path)
     90 if len(data) != length:
---> 91     raise StreamError("stream read less than specified amount, expected %d, found %d" % (length, len(data)), path=path)
     92 return data

StreamError: Error in path (parsing) -> magic3
stream read less than specified amount, expected 12, found 0

What might be the problem??

[SaeedShurrab]

Any updates @marksgraham

[marksgraham]

This seems to fail very early on, reading the first header. Are you sure this is an e2e scan? Do you know which scanner it came from (maybe very old or very new?), and I'd it is a standard OCT or some different modality?

El mié, 25 de ene de 2023 00:08, Saeed Shurrab @.***> escribió:

Any updates @marksgraham https://github.com/marksgraham

— Reply to this email directly, view it on GitHub https://github.com/marksgraham/OCT-Converter/issues/83#issuecomment-1403182541, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB4UIM2BUWYEHR6PCGV6YOTWUDGNZANCNFSM6AAAAAAUDNQ2DM . You are receiving this because you were mentioned.Message ID: @.***>

[SaeedShurrab]

Hi Mark,

This is the first line of the e2e file, could you check the problem?

434d 4462 0000 0000 0000 0000 6400 0000 CMDb........d...

[marksgraham]

This matches the start of my .e2e files so it is an .e2e .. what is the filesize?

[SaeedShurrab]

around 50 Mb

[marksgraham]

Hm this is very difficult to debug remotely.. is it a normal OCT scan? If you can send me the scan I can take a look

[SaeedShurrab]

hmmm. It is not allowed to send the file elsewhere due to privacy issues.

[marksgraham]

Yeah fair enough. And have you had any luck with eyepy which has a similar but seperately implemented .e2e reader - will help us work out if the problem is a code bug or a problem with us not fully understanding the .e2e specification.

[SaeedShurrab]

Yeah fair enough. And have you had any luck with eyepy which has a similar but seperately implemented .e2e reader - will help us work out if the problem is a code bug or a problem with us not fully understanding the .e2e specification.

even eyepie does not work for the file that I have, Thanks Mark

[Oli4]

The eyepy reader seems to fail later, because it does not find the Bscan meta data. Parsing of the file contents seems to work, so the file should be fine.

[SaeedShurrab]

Hi guys

We managed to open the .E2E and .sdb files using convert_oct_data c++ library to .img, then we use oct-converter to from .img to JpeG. Thanks for your help

[Oli4]

Thank you for sharing your solution!

[marksgraham]

Hi,

Thanks, good to know. Looks like there is something we can learn about the .e2e format from that library - will look into when I get a chance.

Best, Mark

[marksgraham]

Closed because without the ability to access the data I don't think I can work on fixing this

[ivanslootweg]

Hi, I am experiencing the same error with an .E2E file. I can send you a file if you want

[marksgraham]

Sorry for missing this - this is separate from the previous unicode issue you were having? If you're still having the issue you can send me the file and I'll take a look