pauldex
commented
11 months ago It sounds like with fdb you can still read the BlobReader objects after the cursor is closed, but with firebird_driver the BlobReader objects are closed along with the cursor and can no longer be read. Is that right?
A possible workaround is to make sure to have read everything you need before to cursor is closed (e.g., with .all() or .first()) before trying to process the result set.
I’ll add reviewing the cursor closing behavior to the list of things to do.
Thanks,
Paul
From: Bryan Cole @.> Sent: Tuesday, October 10, 2023 9:02 AM To: pauldex/sqlalchemy-firebird @.> Cc: Subscribed @.***> Subject: [pauldex/sqlalchemy-firebird] SqlAlchemy handling of firebird stream blobs (a.k.a. BlobReader objects) (Issue #58)
I'm having a problem reading large blobs from a firebird database, using the new firebird-driver.
For large blobs, the firebird driver (both new and old) returns BlobReader objects, rather than the fully materialised python bytes-objects. These BlobReader objects are file-like and can be read to obtain the binary data. The thing that has changed in the new firebird driver is that the Cursor.close() method now closes all BlobReader objects associated with that cursor. Unfortunately, when sqlalchemy executes a statement returning data (i.e. calls fetchXXX() on the cursor), it always closes the cursor after iterating over it, but before accessing any of the data in the returned rows. Hence, later on when the data is passed to the Dialect TypeDecorator to handle type-conversions, the cursor has been closed so all the BlobReader objects are closed, so reading the data fails.
Although I can't see a way to fix this in the Dialect, I'm wondering if you have any ideas, before I look at modifying the sqlalchemy core to add some sort of hook to customise Cursor closing behaviour. The author of the new firebird driver seems adamant that the BlobReaders ought to be closed when the Cursor is closed.
— Reply to this email directly, view it on GitHub https://github.com/pauldex/sqlalchemy-firebird/issues/58 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ABX7Z67JWG5EHUMGAJRIPG3X6VWPTAVCNFSM6AAAAAA52TFJRSVHI2DSMVQWIX3LMV43ASLTON2WKOZRHEZTKNZQGYZTGOI . You are receiving this because you are subscribed to this thread. https://github.com/notifications/beacon/ABX7Z63SE4XT4VPKEODP5Z3X6VWPTA5CNFSM6AAAAAA52TFJRSWGG33NNVSW45C7OR4XAZNFJFZXG5LFVJRW63LNMVXHIX3JMTHHGYEI4M.gif Message ID: @. @.> >
bryancole
fdcastel
I'm having a problem reading large blobs from a firebird database, using the new firebird-driver.
For large blobs, the firebird driver (both new and old) returns
BlobReaderobjects, rather than the fully materialised python bytes-objects. These BlobReader objects are file-like and can be read to obtain the binary data. The thing that has changed in the new firebird driver is that theCursor.close()method now closes all BlobReader objects associated with that cursor. Unfortunately, when sqlalchemy executes a statement returning data (i.e. calls fetchXXX() on the cursor), it always closes the cursor after iterating over it, but before accessing any of the data in the returned rows. Hence, later on when the data is passed to the Dialect TypeDecorator to handle type-conversions, the cursor has been closed so all the BlobReader objects are closed, so reading the data fails.Although I can't see a way to fix this in the Dialect, I'm wondering if you have any ideas, before I look at modifying the sqlalchemy core to add some sort of hook to customise Cursor closing behaviour. The author of the new firebird driver seems adamant that the BlobReaders ought to be closed when the Cursor is closed.