ERROR: database file appears corrupt - page x is of wrong type (expected 8, found 0) [CORE3287]

[firebird-automations]

Submitted by: Yiannis Bourkelis (yiannis)

Attachments: CorruptionTest.rar screenshot-1.jpg ErrorMsg.JPG

When I continuously insert new rows to the database, and the rows contain at least one blob field, the database gets corrupted after some time.

I am sending you a project and the database to reproduce this issue.

The corruption happens usually after many days when the project runs in the hard disk, but if you run the attached project from a flash disk drive you will be able to corrupt the database in less than 30 minutes.

The error that I got when I try to back up the database using gbak is: gbak: ERROR:database file appears corrupt (E:\DATABASE.FDB) gbak: ERROR: wrong page type gbak: ERROR: page 56268 is of wrong type (expected 8, found 0) gbak: ERROR:gds_$get_segment failed gbak:Exiting before completion due to errors

The error that I got when I try to validate the database using gfix is: database file appears corrupt () -bad checksum -checksum error on database page 56268

[firebird-automations]

Commented by: Yiannis Bourkelis (yiannis)

Steps to reproduce: 1. Copy the attached project to a flash drive. I was able to reproduce the corruption in less than 30 minutes, in a SanDisk Cruzer Slice 8GB formatted as Fat32, on Windows XP and Windows 7. The flash disk must not be very slow.

2. Run CorruptionTest.exe in the Release folder.

3. Fill the Firebird user name and password and click on the "Insert Images" button. The application will start inserting the images inside the "Images" folder into the Database.fdb.

4. Every 15 minutes check if the database is corrupted using gbak. Example: "gbak.exe" -b -g -ig -user SYSDBA -password masterkey e:\Database.fdb c:\Database.fbk In less than 30 minutes the database will get corrupted.

Actual results: When running the project from a FAT32 formatted flash disk, the database gets corrupted in less than 30 minutes.

Expected results: The database should not get corrupted

[firebird-automations]

Modified by: Yiannis Bourkelis (yiannis)

Attachment: CorruptionTest\.rar \[ 11843 \]

[firebird-automations]

Commented by: Sean Leyne (seanleyne)

The test app does not work on my local HDD, so I am unable to confirm. What folder should the database be in?

[firebird-automations]

Commented by: Yiannis Bourkelis (yiannis)

The database is inside th bin folder, the file "Database.fdb".

If you extract the project and run the file "CorruptionTest.exe" in the "Release" folder it should run. Is your firebird server running?

What error do you get?

[firebird-automations]

Modified by: Yiannis Bourkelis (yiannis)

Attachment: screenshot\-1\.jpg \[ 11844 \]

[firebird-automations]

Commented by: @SENikitin

Hello,

> I was able to reproduce the corruption in less than 30 minutes, in a SanDisk Cruzer Slice 8GB formatted as Fat32, on Windows XP and Windows 7. >The flash disk must not be very slow.

Flash drives (not SSD, but usual file storage dongles) are not recommended for usage as storage for databases.

Our experience shows the high rate of database corruptions on flash drives, so we even put it to the list of possible reasons http://www.ib-aid.com/articles/item70

[firebird-automations]

Commented by: Yiannis Bourkelis (yiannis)

Hello Sergey,

I believe that the high rate of database corruptions on flash drives is due to the fact that the user can remove the flash drive from the computer, when the engine writes to the database.

Another reason of corruption is the low-cost no-name flash disks with low endurance.

A high-speed, high-quality USB 2.0 or USB 3.0 flash disk with high endurance is wonderful for portable applications and embedded storage.

However, I suggested reproducing this issue on a flash disk, because it is faster and easier to reproduce it. On a hard disk it takes days to corrupt the database, but on the flash disk the database gets corrupted in less than 30 minutes and it might be easier for the core team to identify the problem.

[firebird-automations]

Commented by: Yiannis Bourkelis (yiannis)

If you get an "OutOfMemoryException" then remove the file "Thumds.db" from the "Images" folder.

[firebird-automations]

Commented by: Sean Leyne (seanleyne)

1 - See the attached screenshot (ErrorMsg.jpg) for the error message

2 - I am running an 2.5 beta, though that should not matter -- I can connect to the database without issue.

[firebird-automations]

Modified by: Sean Leyne (seanleyne)

Attachment: ErrorMsg\.JPG \[ 11849 \]

[firebird-automations]

Commented by: Yiannis Bourkelis (yiannis)

Have you set the "Firebird User Name" and "Firebird Password" on the form?

When you run the application it takes the default user: SYSDBA and password: masterkey

[firebird-automations]

Commented by: @AlexPeshkoff

Sean, sooner of all you should add ISC_USER and ISC_PASSWORD to your environment. Or temporarily return SYSDBA's password to default one.

[firebird-automations]

Commented by: Sean Leyne (seanleyne)

Guys, I may not be a core engine developer, but I have been using Interbase/Firebird since 1994, so I am not an idiot!!!

Yes, I have reset the SYSDBA password to masterkey and the application still refuses to work.

Even with setting the environment variables ... no joy.

[firebird-automations]

Commented by: @AlexPeshkoff

Sorry, Sean - sometimes we all fail to read obvious thing from the screen, I know it by myself :-) But if it does not work with default SYSDBA's password - all this starts to look like a mystery. Or may be something wrong with .NET client?

[firebird-automations]

Commented by: Sean Leyne (seanleyne)

Alex, you may be right about .Net -- I only have .Net 4 installed on my Win7 system. I wonder if the EXE needs an older .Net version?

Have you been able to run the EXE?

[firebird-automations]

Commented by: @hvlad

I'm able to run exe. However i don't think "testing" IO intensive application using flash drive is good idea.

[firebird-automations]

Commented by: Yiannis Bourkelis (yiannis)

Vlad, in my efforts to isolate and reproduce the issue, I found that the fastest way to make the corruption occur is to put the database in a flash disk.

The problem is that the end users cannot describe to me exactly what they do that causes the corruption, because it happens randomly. The corruption is allways happening in the Documents table and only when there is blob data.

[firebird-automations]

Commented by: Sean Leyne (seanleyne)

1 - I was finally able to run the project, once I had changed to code to use the embedded database connection and by refactoring the code to only open the database connection once, instead of opening a connecting with each loop through the images folder files!

2 - I successfully ran the test, inserted 20,000 images in about 30 minutes (got to love SSDs!) and produced a 14GB database (1,789,000 pages) -- without error!

3 - As far as I am concerned, the test does not show a problem with the Firebird engine but shows a problem with the reliability of flash/thumb drive media! As such this case should be closed.

[firebird-automations]

Commented by: @AlexPeshkoff

Sean, I tend to agree - looks like database gets corrupted on a flash drive sooner of all due to low reliability of that drive.

[firebird-automations]

Commented by: Sean Leyne (seanleyne)

I ran another test and inserted 75,000 images without issue. So, this issue is dead as far as I am concerned.

[firebird-automations]

Modified by: Sean Leyne (seanleyne)

status: Open \[ 1 \] =\> Resolved \[ 5 \] resolution: Cannot Reproduce \[ 5 \]

[firebird-automations]

Modified by: @pcisar

status: Resolved \[ 5 \] =\> Closed \[ 6 \]