Intel 4500MHD VESA mode 1440x900x32 produces garbled screen

[GoogleCodeExporter]

VESA mode 1440x900x32 is listed in the VESA BIOS but produces garbled screen. 
Any suggestions on how to troubleshoot this? 1440x900x16 works essentially as 
normal, but the colors seem slightly off (greens appear blueish, yellows appear 
pale).

Original issue reported on code.google.com by stanley....@gmail.com on 20 Dec 2012 at 3:39

[GoogleCodeExporter]

in general, you cannot debug vesa. there might be a bug in the emulation tho.
you find these by looking at gigabytes of machine code traces and see if
theres anything odd. it might be that these vesa modes just do not work due
to not being used/tested. use the -t flag to realemu and redirect std error
to a file, then run a mode switch (be patient, it might take hours with
tracing enabled).

Original comment by cinap_le...@felloff.net on 29 Dec 2012 at 11:24

[GoogleCodeExporter]

This is the output of changing from VESA mode 1440x900x16 to 1440x900x32 with 
realemu -t:

http://plan9.stanleylieber.com/hardware/thinkpad/x301/2776-p4u/vesa.txt

Original comment by stanley....@gmail.com on 4 Jan 2013 at 3:04

[GoogleCodeExporter]

This is the output of changing from VESA mode 1440x900x16 to 1440x900x32 with 
realemu -p:

http://plan9.stanleylieber.com/hardware/thinkpad/x301/2776-p4u/realemu.p.txt

Original comment by stanley....@gmail.com on 5 Jan 2013 at 6:03

[GoogleCodeExporter]

ok, looking thru the machine code trace. i see it is programming the PIPEB
source image size to 768x480 instead of 1440x900. looking thru the code
traces, it seems it swaps mode 0x16b (1440x900x32) with mode 0x162 (768x480x32)
for some reason.

there are two tables involved in this. a 3 byte per entry (mode alias?) table
at c000:7921 where the format is like (xx yy zz) where xx is the low byte of
the mode it wants to alias, or ff or end of table. yy seems to be the mode to
map into (for paged framebuffer?) and zz is the mode the thing gets replaced.

in our case zz is 0x62, thats why it programms the thing wrong. that mode is
then looked up in another table that contains the actual resolution. that table
starts at c000:0268 with 5 bytes per entry. format sems to be xx ?? ab cd ??
where xx is the mode (0x62) and ab cd is a pointer to some other stucture. that
pointer + 8 is decoded like this:

017a38 20004062 00001628 00000004 00e00008 000012d4 00071423 00007bdc 00007b6c 
0000 c000 0000 c000 cszoPdI 45b1 e8 CALL [3dcb]
017a39 20004062 00001628 00000004 00e00008 000012d4 00071423 00007bdc 00007b6a 
0000 c000 0000 c000 cszoPdI 3dcb 8a MOV AH, ES:[DI+4]
017a3a 20003062 00001628 00000004 00e00008 000012d4 00071423 00007bdc 00007b6a 
0000 c000 0000 c000 cszoPdI 3dcf c0 SHR AH, $04
017a3b 20000362 00001628 00000004 00e00008 000012d4 00071423 00007bdc 00007b6a 
0000 c000 0000 c000 cszoPdI 3dd2 8a MOV AL, ES:[DI+2]
017a3c 20000300 00001628 00000004 00e00008 000012d4 00071423 00007bdc 00007b6a 
0000 c000 0000 c000 cszoPdI 3dd6 8a MOV BH, ES:[DI+7]
017a3d 20000300 00001028 00000004 00e00008 000012d4 00071423 00007bdc 00007b6a 
0000 c000 0000 c000 cszoPdI 3dda c0 SHR BH, $04
017a3e 20000300 00000128 00000004 00e00008 000012d4 00071423 00007bdc 00007b6a 
0000 c000 0000 c000 cszopdI 3ddd 8a MOV BL, ES:[DI+5]
017a3f 20000300 000001e0 00000004 00e00008 000012d4 00071423 00007bdc 00007b6a 
0000 c000 0000 c000 cszopdI 3de1 c3 RET 0

reslting in x and y dimensions in AX and BX that will later be programmed into
the graphics card.

i need your realmodemem file. with that, i can decode these two tables and 
possibly
fix up the entry.

Original comment by cinap_le...@felloff.net on 6 Jan 2013 at 8:14

[GoogleCodeExporter]

ok, theres the mode alias table at c000:7921. i put commas
between the entries and you can see, that mode 6b is the
only one that maps to 0x62 in the 3rd byte (see the *).

0000000  01 ff ff,01 30 30,03 32 32,05 34 34,07 38 38,3a
0000010  3a 3a,3c 3c 3c,11 41 41,14 43 43,17 45 45,1a 49
0000020  49,4b 4b 4b,4d 4d 4d,12 50 50,15 52 52,18 54 54,
0000030  1b 58 58,5a 5a 5a,5c 5c 5c,60 60 60,61 61 61,62
0000040  62 62,63 63 63,64 64 64,65 65 65,66 66 66,67 67
0000050  67,68 68 68,69 69 69,6a 6a 6a,6b 6b*62,6c 6c 6c,
0000060  6d 6d 6d,6e 6e 6e,6f 6f 6f,70 70 70,71 71 71,ff

so, we can try to patch that single byte from 62 to 6b like
in the other modes. and maybe notify intel about this bios
bug.

Original comment by cinap_le...@felloff.net on 6 Jan 2013 at 8:29

[GoogleCodeExporter]

to patch the byte:

dd -if /dev/realmodemem -bs 1 -iseek 817532 -count 1 | dd -of /dev/realmodemem 
-bs 1 -oseek 817533 -count 1 -trunc 0

Original comment by cinap_le...@felloff.net on 6 Jan 2013 at 8:44

[GoogleCodeExporter]

confirmed. this works. i wont say one cannot debug vesa bios anymore :)

Original comment by cinap_le...@felloff.net on 6 Jan 2013 at 9:02

[GoogleCodeExporter]

added work arround in aux/vga for this in r2cd7add619a5, needs testing.

Original comment by cinap_le...@felloff.net on 7 Jan 2013 at 8:13

[GoogleCodeExporter]

confirmed, works. the wrong colors are a display problem as it can be 
reproduced with other operating systems. output on external monitor is 
flawless. so closing this bug now.

Original comment by cinap_le...@felloff.net on 8 Jan 2013 at 2:12

• Changed state: Fixed