wrong negative XSQLVAR.sqllen returned when using fb25 isql with fb30 [CORE5595]

[firebird-automations]

Submitted by: Holger Klemt (klemmo)

create the database on fb3 and use a isql version from firebird 2.5 with fb3 client lib and execute

SELECT cast('any text' as varchar(32765)) || 'abcd' FROM rdb$database

Result:

Statement failed, SQLSTATE = HY001 unable to allocate memory from operating system

[firebird-automations]

Modified by: Holger Klemt (klemmo)

description: just tried to use a udf replacement using the stored function and received very strange out of memory errors from firebird 3\.0\.2 can be easy reproduced when creating this function in an empty fb302 database CREATE OR ALTER FUNCTION RTRIM \( S1 VARCHAR\(32765\)\) RETURNS VARCHAR\(10889\) AS DECLARE VARIABLE S VARCHAR\(32765\); begin s=trim\(S1\); return S; end when executing the following statement, the error "out of memory" is shown workaround: change return declaration to VARCHAR\(10888\) or any other length smaller than 10888 and all works fine\. \(the exact number is always different when doing different statements I think this is an error in the handling of expressions in stored functions params, since when doing the same statement with trim insted of rtrim, all works fine =\> just tried to use a udf replacement using the stored function and received very strange out of memory errors from firebird 3\.0\.2 can be easy reproduced when creating this function in an empty fb302 database CREATE OR ALTER FUNCTION RTRIM \( S1 VARCHAR\(32765\)\) RETURNS VARCHAR\(10889\) AS DECLARE VARIABLE S VARCHAR\(32765\); begin s=trim\(S1\); return S; end when executing the following statement, the error "out of memory" is shown SELECT \(R\.RDB$RELATION\_NAME \|\| ' \(' \|\| TRIM\(CASE WHEN \(R\.RDB$VIEW\_SOURCE IS NULL\) THEN 'VIEW ' ELSE 'No view' END\) \|\| '\)'\) FROM RDB$RELATIONS R workaround: change return declaration to VARCHAR\(10888\) or any other length smaller than 10888 and all works fine\. \(the exact number is always different when doing different statements I think this is an error in the handling of expressions in stored functions params, since when doing the same statement with trim insted of rtrim, all works fine

[firebird-automations]

Modified by: Holger Klemt (klemmo)

description: just tried to use a udf replacement using the stored function and received very strange out of memory errors from firebird 3\.0\.2 can be easy reproduced when creating this function in an empty fb302 database CREATE OR ALTER FUNCTION RTRIM \( S1 VARCHAR\(32765\)\) RETURNS VARCHAR\(10889\) AS DECLARE VARIABLE S VARCHAR\(32765\); begin s=trim\(S1\); return S; end when executing the following statement, the error "out of memory" is shown SELECT \(R\.RDB$RELATION\_NAME \|\| ' \(' \|\| TRIM\(CASE WHEN \(R\.RDB$VIEW\_SOURCE IS NULL\) THEN 'VIEW ' ELSE 'No view' END\) \|\| '\)'\) FROM RDB$RELATIONS R workaround: change return declaration to VARCHAR\(10888\) or any other length smaller than 10888 and all works fine\. \(the exact number is always different when doing different statements I think this is an error in the handling of expressions in stored functions params, since when doing the same statement with trim insted of rtrim, all works fine =\> just tried to use a udf replacement using the stored function and received very strange out of memory errors from firebird 3\.0\.2 can be easy reproduced when creating this function in an empty fb302 database CREATE OR ALTER FUNCTION RTRIM \( S1 VARCHAR\(32765\)\) RETURNS VARCHAR\(10889\) AS DECLARE VARIABLE S VARCHAR\(32765\); begin s=trim\(S1\); return S; end when executing the following statement, the error "out of memory" is shown SELECT \(R\.RDB$RELATION\_NAME \|\| ' \(' \|\| TRIM\(CASE WHEN \(R\.RDB$VIEW\_SOURCE IS NULL\) THEN 'VIEW ' ELSE 'No view' END\) \|\| '\)'\) FROM RDB$RELATIONS R workaround: change return declaration to VARCHAR\(10888\) or any other length smaller than 10888 and all works fine\. \(the exact number is always different when doing different statements I think this is an error in the handling of expressions in stored functions params, since when doing the same statement with trim instead of rtrim, all works fine

[firebird-automations]

Commented by: @asfernandes

Cannot reproduce in Linux.

[firebird-automations]

Commented by: Holger Klemt (klemmo)

changed original tracker entry to a more detailed version

[firebird-automations]

Modified by: Holger Klemt (klemmo)

description: just tried to use a udf replacement using the stored function and received very strange out of memory errors from firebird 3\.0\.2 can be easy reproduced when creating this function in an empty fb302 database CREATE OR ALTER FUNCTION RTRIM \( S1 VARCHAR\(32765\)\) RETURNS VARCHAR\(10889\) AS DECLARE VARIABLE S VARCHAR\(32765\); begin s=trim\(S1\); return S; end when executing the following statement, the error "out of memory" is shown SELECT \(R\.RDB$RELATION\_NAME \|\| ' \(' \|\| TRIM\(CASE WHEN \(R\.RDB$VIEW\_SOURCE IS NULL\) THEN 'VIEW ' ELSE 'No view' END\) \|\| '\)'\) FROM RDB$RELATIONS R workaround: change return declaration to VARCHAR\(10888\) or any other length smaller than 10888 and all works fine\. \(the exact number is always different when doing different statements I think this is an error in the handling of expressions in stored functions params, since when doing the same statement with trim instead of rtrim, all works fine =\> just tried to use a udf replacement using the stored function and received very strange out of memory errors from firebird 3\.0\.2 can be easy reproduced when creating this function in an empty fb302 database CREATE OR ALTER FUNCTION RTRIM \( S1 VARCHAR\(32765\)\) RETURNS VARCHAR\(10889\) AS DECLARE VARIABLE S VARCHAR\(32765\); begin s=trim\(S1\); return S; end when executing the following statement, the error "out of memory" is shown SELECT \(R\.RDB$RELATION\_NAME \|\| ' \(' \|\| RTRIM\(CASE WHEN \(R\.RDB$VIEW\_SOURCE IS NULL\) THEN 'VIEW ' ELSE 'No view' END\) \|\| '\)'\) FROM RDB$RELATIONS R workaround: change return declaration to VARCHAR\(10888\) or any other length smaller than 10888 and all works fine\. \(the exact number is always different when doing different statements I think this is an error in the handling of expressions in stored functions params, since when doing the same statement with trim instead of rtrim, all works fine

[firebird-automations]

Commented by: @asfernandes

I saw it and still cannot reproduce even in 3.02 in w64 as you reported.

[firebird-automations]

Commented by: Holger Klemt (klemmo)

changed original tracker entry to a more detailed version

[firebird-automations]

Commented by: Holger Klemt (klemmo)

at least now we know the problem, but not the reason, when doing a prepare for the statement, we get a value for

XSQLVAR.sqllen

of -32765 which is in fact a stupid value to allocate memory.

Anyone have any idea why this can happen with a stored function used and not when a udf is used?

[firebird-automations]

Commented by: Holger Klemt (klemmo)

Easy way to reproduce (except in isql, which still does not create this error, but almost any other app will):

SELECT cast('any text' as varchar(32765)) || 'abcd' FROM rdb$database

Firebird returns value of lenght which is greater than max value of sqllen defined as shortint in ibase.pas (signed short in ibase.h). This makes it negative. From my point of view Firebird should give 'implementation limit exceeded' error like for following:

SELECT cast('any text' as varchar(32766)) || 'abcd' FROM rdb$database

Maybe in the new API (AFAIK, isql uses new API) something is different and therefore isql doesn't give an error, but for old API it also should return positive length anyway or 'implementation limit exceeded' error instead.

[firebird-automations]

Commented by: Holger Klemt (klemmo)

one easy way to reproduce the error with firebirds own tools:

create the database on fb3 and use a isql version from firebird 2.5 with fb3 client lib and execute again

SELECT cast('any text' as varchar(32765)) || 'abcd' FROM rdb$database

Result:

Statement failed, SQLSTATE = HY001 unable to allocate memory from operating system

[firebird-automations]

Modified by: Holger Klemt (klemmo)

description: just tried to use a udf replacement using the stored function and received very strange out of memory errors from firebird 3\.0\.2 can be easy reproduced when creating this function in an empty fb302 database CREATE OR ALTER FUNCTION RTRIM \( S1 VARCHAR\(32765\)\) RETURNS VARCHAR\(10889\) AS DECLARE VARIABLE S VARCHAR\(32765\); begin s=trim\(S1\); return S; end when executing the following statement, the error "out of memory" is shown SELECT \(R\.RDB$RELATION\_NAME \|\| ' \(' \|\| RTRIM\(CASE WHEN \(R\.RDB$VIEW\_SOURCE IS NULL\) THEN 'VIEW ' ELSE 'No view' END\) \|\| '\)'\) FROM RDB$RELATIONS R workaround: change return declaration to VARCHAR\(10888\) or any other length smaller than 10888 and all works fine\. \(the exact number is always different when doing different statements I think this is an error in the handling of expressions in stored functions params, since when doing the same statement with trim instead of rtrim, all works fine =\> create the database on fb3 and use a isql version from firebird 2\.5 with fb3 client lib and execute SELECT cast\('any text' as varchar\(32765\)\) \|\| 'abcd' FROM rdb$database Result: Statement failed, SQLSTATE = HY001 unable to allocate memory from operating system environment: Test on FB302w64 =\> Test on FB302w64 and FB302lx64 summary: very strange limit in stored function return param =\> wrong negative XSQLVAR\.sqllen returned when using fb25 isql with fb30

[firebird-automations]

Commented by: @asfernandes

Maybe we should change sqllen to USHORT?

[firebird-automations]

Commented by: @dyemanov

MAX_COLUMN_SIZE is 32767, so SSHORT should be enough. No string value should exceed this limit during concatenation, so there should be a bug somewhere.

[firebird-automations]

Commented by: @dyemanov

FB 2.5 operates with MAX_COLUMN_SIZE = 32767 as string length limit, while FB 3.0 defines MAX_STR_SIZE = 65535 for the same goal. This explains the issue.

[firebird-automations]

Commented by: @asfernandes

It's CORE4881 and there is discussion about it in devel.

Main reason to extend it was 32K / 4 was too limited for UTF-8 AFAIR.