search

ngageoint / sarpy

A basic Python library to demonstrate reading, writing, display, and simple processing of complex SAR data using the NGA SICD standard.
MIT License
262 stars 87 forks source link

Handle BaseReader going away before an open file. #526

Closed kjurka closed 2 weeks ago

kjurka commented 4 months ago

Our application doesn't close any open SICD files explicitly and maintains a reference to them throughout the application, so they're not garbage collected until shutdown. At that point BaseReader has been destructed before our open file and we got a ton of errors like:

Exception ignored in: <function BaseReader.del at 0x7fd05b307a30> Traceback (most recent call last): File "/share/home/kjurka@emagsys.com/git/kjurka/sarpy/sarpy/io/general/base.py", line 463, in del File "/share/home/kjurka@emagsys.com/git/kjurka/sarpy/sarpy/io/general/nitf.py", line 2450, in close AttributeError: 'NoneType' object has no attribute 'close'

I haven't been able to come up with a standalone replication of this problem, but this works for me.

bombaci-vsc commented 4 months ago

I don't know why the garbage collection would set the BaseReader reference to None. If you replace BaseReader.close(self) with super().close(), does the error go away?

kjurka commented 4 months ago

Gets further, but now DataSegment is None at a later point https://github.com/ngageoint/sarpy/blob/3354207e6e8d8920416d1248c9c626c7d5a86b47/sarpy/io/general/base.py#L439

Exception ignored in: <function BaseReader.__del__ at 0x7f87f307f910>
Traceback (most recent call last):
  File "/share/home/kjurka@emagsys.com/git/kjurka/sarpy/sarpy/io/general/base.py", line 463, in __del__
  File "/share/home/kjurka@emagsys.com/git/kjurka/sarpy/sarpy/io/general/nitf.py", line 2450, in close
  File "/share/home/kjurka@emagsys.com/git/kjurka/sarpy/sarpy/io/general/base.py", line 439, in close
TypeError: isinstance() arg 2 must be a type, a tuple of types, or a union
bombaci-vsc commented 1 month ago

Unfortunately python does not guarantee that __del__ methods will work during shutdown. See the second bullet of the warning on __del__’s documentation

image

Unfortunately, I don’t think it is practical to make SARPy's __del__ methods accommodate running during shutdown. The exact order of events during shutdown isn't defined or guaranteed, which could lead to a cat-and-mouse game of trying to make the __del__ methods run without error.