search

sid5432 / pyOTDR

Simple OTDR SOR file parser written in Python
GNU General Public License v3.0
71 stars 28 forks source link

ValueError: read length must be non-negative or -1 #44

Closed bruniii closed 7 months ago

bruniii commented 7 months ago

Hi and thank you for this module!

When I try to read .sor files, sometimes the following script:

import pyotdr

filename = "fail.sor"
_, results, tracedata = pyotdr.read.sorparse(filename=filename)

return the error "ValueError: read length must be non-negative or -1" but only for some files, while for others the script complete, even if all of them are generated by the same OTDR instruments, the same day and, to my knowledge, with no meaningful differences.

I attach two examples, in a zip file: sor example.zip. I'm using python 3.12.1. The error:

File ......\pyotdr\supparams.py:45, in process(fh, results)
     43 # read the rest of the block (just in case)
     44 endpos = results["blocks"][bname]["pos"] + results["blocks"][bname]["size"]
---> 45 fh.read(endpos - fh.tell())
     46 status = "ok"
...
     39     xlen = len(buf)
     40     if xlen > self.spaceleft:
     41         # process then clear buffer

ValueError: read length must be non-negative or -1

The debug log from your package:

2024-02-21 16:16:06,370 pyOTDR     DEBUG    MAIN: bellcore 2.x version
2024-02-21 16:16:06,370 pyOTDR     DEBUG    MAIN: Version 2.00, block size 124 bytes; next position 0XA
2024-02-21 16:16:06,371 pyOTDR     DEBUG    MAIN: 7 blocks to follow; next position 0XCX
2024-02-21 16:16:06,371 pyOTDR     DEBUG    --------------------------------------------------------------------------------
2024-02-21 16:16:06,371 pyOTDR     DEBUG    MAIN: BLOCKS:
2024-02-21 16:16:06,371 pyOTDR     DEBUG    MAIN: GenParams  block: version 2.00,
2024-02-21 16:16:06,372 pyOTDR     DEBUG    block size 67 bytes,
2024-02-21 16:16:06,372 pyOTDR     DEBUG    start at pos 0X7C
2024-02-21 16:16:06,372 pyOTDR     DEBUG    MAIN: SupParams  block: version 2.00,
2024-02-21 16:16:06,373 pyOTDR     DEBUG    block size 55 bytes,
2024-02-21 16:16:06,374 pyOTDR     DEBUG    start at pos 0XBF
2024-02-21 16:16:06,374 pyOTDR     DEBUG    MAIN: FxdParams  block: version 2.00,
2024-02-21 16:16:06,374 pyOTDR     DEBUG    block size 140 bytes,
2024-02-21 16:16:06,375 pyOTDR     DEBUG    start at pos 0XF6
2024-02-21 16:16:06,375 pyOTDR     DEBUG    MAIN: KeyEvents  block: version 2.00,
2024-02-21 16:16:06,375 pyOTDR     DEBUG    block size 1102 bytes,
2024-02-21 16:16:06,376 pyOTDR     DEBUG    start at pos 0X182
2024-02-21 16:16:06,376 pyOTDR     DEBUG    MAIN: DataPts  block: version 2.00,
2024-02-21 16:16:06,376 pyOTDR     DEBUG    block size 32788 bytes,
2024-02-21 16:16:06,376 pyOTDR     DEBUG    start at pos 0X5D0
2024-02-21 16:16:06,378 pyOTDR     DEBUG    MAIN: SpclProprietary  block: version 2.00,
2024-02-21 16:16:06,378 pyOTDR     DEBUG    block size 352 bytes,
2024-02-21 16:16:06,378 pyOTDR     DEBUG    start at pos 0X85E4
2024-02-21 16:16:06,379 pyOTDR     DEBUG    MAIN: Cksum  block: version 2.00,
2024-02-21 16:16:06,379 pyOTDR     DEBUG    block size 8 bytes,
2024-02-21 16:16:06,380 pyOTDR     DEBUG    start at pos 0X8744
2024-02-21 16:16:06,380 pyOTDR     DEBUG    --------------------------------------------------------------------------------

2024-02-21 16:16:06,381 pyOTDR     DEBUG    MAIN: next position 0X7C
2024-02-21 16:16:06,381 pyOTDR     DEBUG    --------------------------------------------------------------------------------

2024-02-21 16:16:06,383 pyotdr.read DEBUG
MAIN:  GenParams block: 67 bytes, start pos 0X7C (124)
2024-02-21 16:16:06,383 pyotdr.genparams DEBUG        :  language: 'EN', next pos 136
2024-02-21 16:16:06,383 pyotdr.genparams DEBUG        : 0. cable ID:
2024-02-21 16:16:06,384 pyotdr.genparams DEBUG        : 1. fiber ID: 4
2024-02-21 16:16:06,384 pyotdr.genparams DEBUG        : 2. fiber type: G.652 (standard SMF)
2024-02-21 16:16:06,386 pyotdr.genparams DEBUG        : 3. wavelength: 1310 nm
2024-02-21 16:16:06,386 pyotdr.genparams DEBUG        : 4. location A: start postion
2024-02-21 16:16:06,386 pyotdr.genparams DEBUG        : 5. location B: end postion
2024-02-21 16:16:06,387 pyotdr.genparams DEBUG        : 6. cable code/fiber type:
2024-02-21 16:16:06,387 pyotdr.genparams DEBUG        : 7. build condition: BC (as-built)
2024-02-21 16:16:06,387 pyotdr.genparams DEBUG        : 8. user offset: 0
2024-02-21 16:16:06,387 pyotdr.genparams DEBUG        : 9. user offset distance: 0
2024-02-21 16:16:06,388 pyotdr.genparams DEBUG        : 10. operator: handler namSupParams
2024-02-21 16:16:06,388 pyotdr.genparams DEBUG        : 11. comments: FIBERCLOUD
Traceback (most recent call last):
  File "...\for_debug.py", line 25, in <module>
    _, results, tracedata = pyotdr.read.sorparse(filename=filename)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "...\pyotdr\read.py", line 53, in sorparse
    status = genparams.process(fh, results)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "...\pyotdr\genparams.py", line 48, in process
    fh.read(endpos - fh.tell())
  File "...\pyotdr\parts.py", line 38, in read
    buf = self.filehandle.read(*args, **kargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: read length must be non-negative or -1

Could you please have a look at the sor files and help me?

Thank you!

RemiDesgrange commented 7 months ago

I did not debug it, but I think endpos - fh.tell() generates a negative integer. I'll have a look.

RemiDesgrange commented 7 months ago

Ok I fixed the problem by commenting out the line 44-45 in genparams.py and 44-45 in supparams.py tests are passing. this read was performed "in case" but we are seeking the file upside down during the process so can't see the point of doing this. Maybe you could check that values are ok for your fail.sor file. If so I'll publish a new fixed version.

bruniii commented 7 months ago

Thank you, I will try tomorrow morning and let you know!

Ottieni Outlook per Androidhttps://aka.ms/ghei36

From: Rémi Desgrange @.> Sent: Wednesday, February 21, 2024 7:45:54 PM To: sid5432/pyOTDR @.> Cc: bruniii @.>; Author @.> Subject: Re: [sid5432/pyOTDR] ValueError: read length must be non-negative or -1 (Issue #44)

Ok I fixed the problem by commenting out the line 44-45 in genparams.py and 44-45 in supparams.py tests are passing. this read was performed "in case" but we are seeking the file upside down during the process so can't see the point of doing this. Maybe you could check that values are ok for your fail.sor file. If so I'll publish a new fixed version.

— Reply to this email directly, view it on GitHubhttps://github.com/sid5432/pyOTDR/issues/44#issuecomment-1957678203, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEENDNFPUUVEA2USULKS4V3YUY6GFAVCNFSM6AAAAABDTJDDRGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJXGY3TQMRQGM. You are receiving this because you authored the thread.Message ID: @.***>

bruniii commented 7 months ago

I can confirm that commenting the lines:

endpos = results["blocks"][bname]["pos"] + results["blocks"][bname]["size"]
fh.read(endpos - fh.tell())

in genparams.py and supparams.py fix the error and now the fail.sor file can be read by your module.

bruniii commented 7 months ago

Dear @RemiDesgrange, could you please realese a version with the fix? I tested locally commenting the two lines as suggested and finished the development of a small tool based on your module, but to give it to my collogues I'm waiting for the new official version of pyOTDR. Sorry to push for it, I'm already very thankful for your quick support!