Disk read errors when accessing near end of volume

[GoogleCodeExporter]

What steps will reproduce the problem?

Asus EeePC 904HA
Boot from a 16GB USB flash drive formatted at NTFS with 2 partitions. 2nd ptn 
is very small (63 sectors) and at very end of the drive and is unformatted. 

command:
find /noexist.txt

Output:
Fatal! Inconsistent data read from (0x80)30990286+127
Fatal! Inconsistent data read from (0x80)30990286+127
Fatal! Inconsistent data read from (0x80)30990286+127

Also command:
cat --length=05 --hex (0x80)30990286+1
gives errors:
Fatal! Inconsistent data read from (0x80)30990286+127
Fatal! Inconsistent data read from (0x80)30990286+127
Fatal! Inconsistent data read from (0x80)30990286+127

What is the expected output? What do you see instead?

The flash drive only has 30990335 sectors but grub4dos is trying to read 127 
sectors from 30990286 = 30990413  which is past the end of the disk!

Partition 1   SIZE=15130.969MiB   Type: 07 NTFS  *ACTIVE*
START POS   = CYL:0 HD:32 SEC:33       END POS = CYL:1023 HD:254 SEC:63
START (LBA) = 2,048 (00000800) SIZE (LBA) = 30,988,225 (01D8D7C1) 
[End=30,990,272]

Partition 2   SIZE=0.031MiB   Type: 21 Hidden(rsvd)       
START POS   = CYL:1023 HD:254 SEC:63       END POS = CYL:1023 HD:254 SEC:63
START (LBA) = 30,990,273 (01D8DFC1) SIZE (LBA) = 63 (0000003F) [End=30,990,335]

P1   Start=2,048 (1,048,576 bytes) End=30,990,272 (15,867,019,264 bytes)
P2   Start=30,990,273 (15,867,019,776 bytes) End=30,990,335 (15,867,051,520 
bytes)

Reported size 15,867,052,032 bytes (14.7773GiB)  Last LBA 30,990,335

If disk is formatted as FAT32, then you can access the very last sector using 
cat --hex --length=5 (0x80)<last_sector>+1

So problem is when booting from NTFS it always seems to read 127 sectors?

What version of the product are you using? On what operating system?
0.4.5c

Please provide any additional information below.

Original issue reported on code.google.com by Steve6375 on 2 Nov 2013 at 1:24

[GoogleCodeExporter]

I am sorry but I would not like to accept it as a problem.

You see you have encountered differences between FAT32 and NTFS. But you did 
not use the filesystem since you just issued "cat --hex (0x80)...+1".

Yes, if grub4dos is trying to read past the end, it is a problem of grub4dos. 
But, bear in mind, generally grub4dos never knows the exact size of a device. 
It is the user's duty to not let grub4dos access sectors with large sector 
number or sectors near the end of a device. Even worse, it could hangup on some 
buggy mainboards when you try to access sectors of large sector number(the 
int13 itself hangs).

grub4dos dare not probe the size of a device just because it could hangup on 
some machines. Reportedly int13/ah=48h encountered a hangup on a machine. 
Besides, int13/ah=48h does not always give correct values.

Because of the bad BIOSes, you cannot ensure your working USB device for your 
machine also works on another machine.

If a device can be accessed using LBA mode, grub4dos will try to access 127 
sectors at a time. For cdrom no-emulation mode, grub4dos will try to access 64K 
at a time. For CHS only devices, grub4dos will try to access one track at a 
time. So if you don't like to get into trouble, you should avoid using the 
ending 64K of your device. And even though you did so indeed, you still unable 
to ensure absolute success on other machines, because of various BIOS limit 
about device size, as mentioned above.

I will close this report later if there is no objection.

Original comment by tinyb...@gmail.com on 4 Nov 2013 at 10:24

[GoogleCodeExporter]

Maybe you could add code to not do 2 retries if the sector(s) being accessed 
are past the end of the last partition  (request LBA+127 > end of ptn) - then 
only one BIOS timeout would be seen instead of 3 timeouts per access?

Original comment by Steve6375 on 23 Nov 2013 at 3:45

[GoogleCodeExporter]

grub4dos will try twice to read a track, where a "track" for LBA mode contains 
127 sectors(the "track" is virtual).

The first read will use one buffer, and the second read will use another 
buffer. Both buffers will be initialized with different data before reading. 
Then compare the two buffers. If any difference occurs, It will report a read 
failure with message "Fatal! Inconsistent data read from ..." as you have 
noticed.

So grub4dos cannot just read it for once.

Besides, when a failure occurs, grub4dos will try the above step for three 
times in order to gain a success(one failure + two more retries = total three 
times of reading).

If we can avoid the two more retries, we should also avoid the first try - 
totally avoid reading the track.

If you don't insist on using the ending 64K space, the problem will not occur.

Some "bad" BIOSes could only access 8G of the USB devices. Some even could only 
access 720KB. It depends on the BIOS. If you place your data in the "tail", you 
are liable to fail. You should use the "head" of the USB device, neither the 
"body", nor the "tail".

Original comment by tinyb...@gmail.com on 3 Dec 2013 at 3:40

[GoogleCodeExporter]

I tried to format a drive using 100% of the space as NTFS and then fill it with 
small files and then access the last file via grub4dos. If the end of the last 
file was within 127 sectors of the end of the physical drive, I was expecting 
an error.
However, I couldn't get an error to occur when accessing the last file...

I guess this issue is not a big deal as no-one else has found (or realised) 
that it is a problem.

Original comment by Steve6375 on 3 Dec 2013 at 3:54

[GoogleCodeExporter]

Look at the rawdisk_read() code:

rawdisk_read (int drive, unsigned long long sector, unsigned long nsec, int 
segment)
{
    const unsigned long BADDATA1 = FOUR_CHAR('B','A','D','?');
    const unsigned long BADDATA2 = FOUR_CHAR('b','a','d','.');
    unsigned long *plast; /* point to buffer of last sector to be read */
    int r;
    /* Write "BAD?" data to last sector buffer */
    /* No need to fill the whole sector.  Just 16 bytes should be enought to avoid most false-positive case. */
    plast = (unsigned long *)((segment<<4)+(nsec-1)*buf_geom.sector_size);
    plast[3] = plast[2] = plast[1] = plast[0] = BADDATA1;
    r = biosdisk(BIOSDISK_READ, drive, &buf_geom, sector, nsec, segment);
    if (r) // error
    return r;
    /* Check for bad data in last read sector */
    if (plast[0]!=BADDATA1 || plast[1]!=BADDATA1 || plast[2]!=BADDATA1 || plast[3]!=BADDATA1)
    return 0; // not "BAD?", success
    // "BAD?", Suspicious
    // Write different data to buffer.
    plast[0] = BADDATA2;
    // Read last sector again
    r = biosdisk(BIOSDISK_READ, drive, &buf_geom, sector+(nsec-1), 1, ((unsigned long)plast>>4));
    if (r) // error
    return r;
    // Compare with previous read data
    if (plast[0] != BADDATA1) 
    {   // Read data changed, error.
    grub_printf("\nFatal! Inconsistent data read from (0x%X)%ld+%d\n",drive,sector,nsec);
    return -1; // error
    }
    return 0; // success
}

It firstly set up some known bytes(as an indicator) at the end of buffer, then 
call BIOS to read. If the last sector not touched, then it will try once more 
to read the last sector.

Take notice. If the read past the end of device, a problematic BIOS could work 
abnormally. Any abnormalities could occur. It could hang up, or reboot 
unexpectedly, or simply jump to a random address and continue to run. There is 
absolutely no safeguard against it. Working on such a case is completely 
worthless. Get away from it.

Original comment by tinyb...@gmail.com on 5 Dec 2013 at 3:19

[GoogleCodeExporter]

Thanks a lot for the detailed report.

I think it is time to close the issue.

Original comment by tinyb...@gmail.com on 6 Dec 2013 at 3:56

• Changed state: WontFix