search

FirebirdSQL / fdb

Firebird Driver for Python
https://www.firebirdsql.org/en/devel-python-driver/
Other
60 stars 26 forks source link

Error "segment buffer length shorter than expected" when saving db-blob-field containing a blob > 64kB to a file [PYFB71] #87

Closed firebird-automations closed 6 years ago

firebird-automations commented 7 years ago

Submitted by: Rajko Thon (rthon)

Attachments: saveBlobToFile.py K4.zip

When executing the attached python-demo I get the following error:

Traceback (most recent call last): File "D:\Entwicklung\k4\firebird\entw\standard\tmp\http://saveBlobToFile.py", line 15, in <module> pic.write( readerA.read() ) File "C:\Apps\Python\lib\site-packages\fdb\http://fbcore.py", line 4751, in read self.__BLOB_get() File "C:\Apps\Python\lib\site-packages\fdb\http://fbcore.py", line 4694, in __BLOB_get "BlobReader.__BLOB_get/isc_get_segment:") fdb.fbcore.DatabaseError: ('BlobReader.__BLOB_get/isc_get_segment:\n- SQLCODE: 101\n- segment buffer length shorter than expected', 101, 335544366)

The error only happens for blobs > 64kB.

-- Analysis (fdb, version 1.7) The error is reported from line 4694 in http://fbcore.py. As it seems it will always happen when there is more than one segment to read. Segments seem to have predefined a size of 64k, hence the program works for blobs < 64k. The behaviour is caused by the setting line 4676: allow_incomplete_segment_read = False Setting this variable to "False" forces the code to always raise an exception as soon as more than one segment has to be read. Since this variable is never changed in the programm, the result is guaranteed.

-- Workaround There is a simple workaround: change line 4676 to allow_incomplete_segment_read = True For my application this is enough and everything works fine. This can be seen by executing the attached python-file again after changing line 4676 accordingly. The jpeg image is now successfully exported. I have not further examined what was intended by the code, so I have no idea if this is already a solution.

PS: Thanks for making the fdb module! Regardless of the reported bug fdb is great work and its existence very much appreciated!

Commits: FirebirdSQL/fdb@a92cd81071a73c19f4c68604befbcd79fde1dd37 FirebirdSQL/fbt-repository@083578714e4cee3cdec316b434e42d3d8c0e4227

firebird-automations commented 7 years ago

Commented by: Rajko Thon (rthon)

Attached files - k4.zip: zipped database file (FB 2.5.6, x64), containing a single table exp_obj with a single row, containing 1 blob (cheetah-picture from freepascal, public domain)

- http://saveBlobToFile.py python demo which saves the blob to a file in local directory

firebird-automations commented 7 years ago
Modified by: Rajko Thon (rthon) Attachment: \[ 13077 \] Attachment: K4\.zip \[ 13078 \]
firebird-automations commented 6 years ago

Commented by: @pcisar

This was an artifact of backward compatibility to KInterbasDB. It was (sort off lame) safeguard to prevent memory exhaustion from very large materialized blobs. The bug was that this safeguard was also used by BlobReader. Blob reader was fixed and new memory exhaustion safeguard was implemented.

firebird-automations commented 6 years ago
Modified by: @pcisar status: Open \[ 1 \] =\> Resolved \[ 5 \] resolution: Fixed \[ 1 \] Fix Version: 1\.8 \[ 10855 \]
firebird-automations commented 6 years ago
Modified by: @pcisar status: Resolved \[ 5 \] =\> Closed \[ 6 \]