A number of errors when database name is longer than 255 symbols [CORE6248]

[firebird-automations]

Submitted by: @pavel-zotov

Is related to CORE6419

Following script illustrates athe problem (of course, most probably it can occur with too long PATH that contains from dozen of nested folders rather than single file name):

set list on;

shell del C:\temp\123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123.fdb 2>nul; shell del C:\temp\1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234.fdb 2>nul; shell del C:\temp\12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678.fdb 2>nul;

--set echo on;

-- Length of full path + file name = 255 bytes is maximal limit for DB referencing using mon$database_name and/or rdb$get_context('SYSTEM', 'DB_NAME'): create database 'C:\temp\123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123.fdb';

show database; -- passed OK

select mon$database_name, rdb$get_context('SYSTEM', 'DB_NAME') from mon$database; -- passed OK

commit; drop database;

------------------------------------------------------------------------ this needs to be fixed: -------------------------------------------------------------

-- When length of full path+file name is 256 then DB is created but can not be referred using RDB$GET_CONTEXT() or MON$DATABASE: create database 'localhost:C:\temp\1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234.fdb';

show database; -- passed OK. WHY ??

select mon$database_name from mon$database; -- Statement failed, SQLSTATE = 22001 / ... -string right truncation / -expected length 255, actual 256

select rdb$get_context('SYSTEM', 'DB_NAME') from mon$database; -- Statement failed, SQLSTATE = 22001 / ... -string right truncation / -expected length 255, actual 256

commit;

drop database;

PS.

-- When length of full path+file name is 260 then "the system cannot find the path specified" raises. -- Statement failed, SQLSTATE = 08001 / I/O error during "CreateFile (create)" operation for file ... / -Error while trying to create file create database 'localhost:C:\temp\12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678.fdb'; drop database; quit;

Commits: FirebirdSQL/firebird@74f7b30b770b4d7b42b7a1db435e346f252c8080 FirebirdSQL/firebird@684e17d0e4ae26163185288222f82658e4821189

====== Test Details ======

STDOUT-logs of backup, restore and gstat currently (09-mar-2020) have only truncated name of DB with length about 230...240 chars. Test verifies presense of such truncated strings rather then expected full path + name of DB.

[firebird-automations]

Commented by: @pavel-zotov

NBACKUP can work with .FDB file which name is less or equal 253 (ascii ?) character (at least on Winows).

After this following error raises: [ PROBLEM ON "begin backup: commit". I/O error during "CreateFile (create)" operation for file "<...>.FDB.delta" -Error while trying to create file -<the system cannot find the path specified> // localized message here SQLCODE:-902 ]

Adding of suffix "..delta" to DB name can not be neither avoided nor replaced with some other (shorter) text.

So, for practice usage max length of path + DB filename must be limited with 253 bytes rather than 255 as this ticket title states. Even if some other OS allows to create files with bigger length because such file will not be restored on Windows.

[firebird-automations]

Commented by: @mrotteveel

I'd suggest that instead, the columns of these tables should be changed to allow for longer filenames. And one Windows, checks should be made for API usage, because recent versions of Windows do support longer file paths that 255 characters, though I'm not exactly sure if that requires a different API or specific settings.

[firebird-automations]

Commented by: @pavel-zotov

> instead, the columns of these tables should be changed to allow for longer filenames.

Sorry, this i did not understand. Do you mean to increase size of RDB$ / MON$ tables text fields ?

> recent versions of Windows do support longer file paths that 255 characters

Such API can provoke to create database with path+file name length beyond this limit and, consequently, this makes such DB not usable on OS which still has limit about 255 chars. I doubt that this is good opportunity...

[firebird-automations]

Commented by: @pavel-zotov

One more limitation was encountered in record-level backup (i.e. gbak or fbsvcmgr action_backup): maximum length of path + name of DB can be 245 characters.

This is the only action that has such limit; all other have greater than this (nbackup = 253, others = 255). Tested utilities / modes:

isql \-i <script with "create database <longname\_db\>" statement\>
isql \-x <localhost:longname\_db\>; 
gstat \-r <localhost:longname\_db\>; 
gfix \-w async \| \-sweep \| \-shut full \-force 0 \| \-online <localhost:longname\_db\>; 
nbackup \-L \| \-N ; 
fb\_lock\_print \-c \-d <longname\_db\>; 
fbvsvcmgr action\_trace \| action\_validate  \| action\_repair \| action\_nbak \| action\_nrest \| action\_backup \| action\_restore

====

For path+name length = 246 characters action backup / restore FAILED with following result:

1) gbak -b -v localhost:<drive:\path\to\name>.fdb <drive:\path\to\name>.fbk -- shows:

gbak:readied database <CUTED-OFF name on 261th character, ellipsis instead of tailing text> gbak:creating file <CUTED-OFF name on 261th character, ellipsis instead of tailing text> gbak:starting transaction gbak:database <CUTED-OFF name on 261th character, ellipsis instead of tailing text> gbak:writing domains gbak: writing domain RDB$1 gbak:writing shadow files gbak:writing character sets . . . gbak:writing SQL roles gbak:writing names mapping gbak:closing file, committing, and finishing. 18432 bytes written gbak:text for attribute 1 is too large in put_asciz(), truncating to 255 bytes

So, .fbk file *WILL* be created and it seems that we can restore from it. But restore will fail:

2) gbak -rep -v <drive:\path\to\name>.fbk localhost:<drive:\path\to\name>.fdb -- shows:

gbak:opened file <CUTED-OFF name on 261th character, ellipsis instead of tailing text> gbak:transportable backup -- data in XDR format gbak: backup file is compressed gbak:backup version is 11 gbak:do not recognize backup attribute 108 -- continuing gbak:do not recognize backup attribute 53 -- continuing gbak:do not recognize backup attribute 49 -- continuing gbak:do not recognize backup attribute 51 -- continuing gbak:do not recognize backup attribute 52 -- continuing gbak:do not recognize backup attribute 73 -- continuing gbak:do not recognize backup attribute 49 -- continuing gbak:do not recognize backup attribute 51 -- continuing gbak:do not recognize backup attribute 55 -- continuing gbak:do not recognize backup attribute 51 -- continuing gbak: ERROR:Expected database description record gbak:Exiting before completion due to errors

[firebird-automations]

Commented by: @hvlad

It have no sence as you can't force user to not move\rename existing database file to make full path longer than 255 chars.

[firebird-automations]

Commented by: @pavel-zotov

What exactly 'have no sence' ? Path consisted of dozen other sub-dirs (e.g. in UUID-form) can easy be near this limit. IMO, limitation must be: 1) the same for every utility / mode (gbak / nbackup / isql: create database statement etc) 2) less than 250 characters for any OS; currently one may create database with len = 259 chars (at least on Windows 8.1 x64), but such DB can not be used normally.

[firebird-automations]

Commented by: @pavel-zotov

> as you can't force user to not move\rename

Sorry, could not understand this properly when first read. If user does move such database from <path+file> less then 250 to some 'deepest' place - this will be his own problem. I think that after getting errors he will change his mind and return DB to normal place. What I suppose is: FB must prevent *creation* of database using such long path+names.

[firebird-automations]

Modified by: @AlexPeshkoff

assignee: Alexander Peshkov \[ alexpeshkoff \]

[firebird-automations]

Commented by: @hvlad

Alex, I hope you not going to implement it as is ?

[firebird-automations]

Commented by: @AlexPeshkoff

No, I plan to truncate database name in found by Pavel places

[firebird-automations]

Modified by: @AlexPeshkoff

summary: Ability to create database with length more than 255 characters should be prohibited =\> A number of errors when database name is longer than 255 symbols

[firebird-automations]

Commented by: @AlexPeshkoff

Fixed code of gbak, monitoring tables and system function rdb$get_context(). Behavior is consistent with "good old" database info call used in 'show db' in isql, i.e. database name is truncated to 255 chars. Appropriate warnings are issued.

.delta suffix added in nbackup is not an issue at all - ALTER DATABASE ADD DIFFERENCE FILE should be used for databases with long name.

[firebird-automations]

Modified by: @AlexPeshkoff

status: Open \[ 1 \] =\> Resolved \[ 5 \] resolution: Fixed \[ 1 \] Fix Version: 4\.0 Beta 2 \[ 10888 \]

[firebird-automations]

Commented by: @pavel-zotov

> Fixed code of gbak ... database name is truncated to 255 chars

This:

gbak:readied database <CUTED-OFF name on 261th character, ellipsis instead of tailing text> gbak:creating file <CUTED-OFF name on 261th character, ellipsis instead of tailing text> gbak:starting transaction gbak:database <CUTED-OFF name on 261th character, ellipsis instead of tailing text>

- still exists (at least on WI-T4.0.0.1796 ), and visible name of DB file is ** 243 ** chars rather 250.

(example: gbak:database c:\temp\1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345... )

Why can't gbak print FULL name of database file, without truncation and adding ellipsis ?

[firebird-automations]

Commented by: @pavel-zotov

PS. Further, if gbak switches contain '-st tdrw' then name of DB file becomes yet shorter: 235, 243 & 238:

gbak: time delta reads writes
gbak: 0.274 0.274 82 1 readied database c:\temp\12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567... gbak: 0.274 0.000 0 0 creating file c:\temp\12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890... gbak: 0.274 0.000 0 0 starting transaction gbak: 0.276 0.001 10 1 database c:\temp\1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345... gbak: 0.276 0.000 0 0 writing domains . . .

Am i right in guess that some limitation in gbak utility exists for "decoration" purposes to make text width no more than 260 chars (when we run it without '-st tdrw') or 290 chars (when we use '-st tdrw') ?

[firebird-automations]

Commented by: @pavel-zotov

PPS.

Command "gstat -h <dbname_with_length_255_characters>" truncates visible part of DB file name to 241 crahacter.

For example, if name of database is:

C:\FBTESTING\QA\FBT-REPO\TMP\ABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255.FDB (length = 255 chars)

-- then "gstat -h" for this DB will show in the 1st line:

Database "C:\FBTESTING\QA\FBT-REPO\TMP\ABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC255DEFABC25...

[firebird-automations]

Modified by: @pavel-zotov

status: Resolved \[ 5 \] =\> Resolved \[ 5 \] QA Status: No test =\> Done with caveats Test Details: STDOUT\-logs of backup, restore and gstat currently \(09\-mar\-2020\) have only truncated name of DB with length about 230\.\.\.240 chars\. Test verifies presense of such truncated strings rather then expected full path \+ name of DB\.

[firebird-automations]

Modified by: @pavel-zotov

status: Resolved \[ 5 \] =\> Closed \[ 6 \]

[firebird-automations]

Modified by: @asfernandes

Link: This issue is related to [CORE6419](https://github.com/FirebirdSQL/firebird/issues?q=CORE6419+in%3Atitle) \[ [CORE6419](https://github.com/FirebirdSQL/firebird/issues?q=CORE6419+in%3Atitle) \]