Test every non tested opcode

barotto commented 6 years ago

A comment from VOGONS thread https://www.vogons.org/viewtopic.php?f=9&t=60095:

XLAT instructions (both 16-bit and 32-bit versions) were faulty: it was checking memory against a write(for the memory operand) instead of a proper read.

superfury commented 6 years ago

Currently left(ignoring those still pending in the other issue(pushf(d)/popf(d)), as well as protected-mode instructions and related instruction for protected mode-specific functionality(0F beginning range instructions)):

Logical operations in the 00-3F range.
Logical operations in the 80-83 range.
INC reg in the 41-47 range.
DEC reg in the 49-4F range.
String input/output from port in the 6C-6F range, probably testable using e.g. the temporary DMA registers(unused registers which aren't used on the DMA chipset of the DMA Page Registers).
TEST instructions(opcode 84-85).
XCHG instructions(opcode 86-87).
MOV instructions(opcode 88-8B).
XCHG instructions(opcode 90-97).
SAHF/LAHF instructions (opcode 9E-9F).
MOV with immediate offset(opcodes A0-A3).
32-bit address-size string instructions(B/W/D).
TEST AL/AX/EAX,imm(opcodes A8&A9).
Different variants of SHLD/SHRD with operand-sizes.
MOV reg,imm (opcodes B0-BF).
RET near (opcode C2).
MOV r/m,imm(opcodes C6-C7).
ENTER/LEAVE(opcodes C6&C7).
XLAT (opcode D7).
LOOP/J(E)CXZ instructions 32-bit operand size(opcodes E0-E3).
IN/OUT instructions(opcodes E4-E7, EC-EF), see string i/o instructions.
JMP instructions(opcodes E9-EB).
REP/REPE/REPNE (opcodes F0-F3).
Opcode F6/F7 TEST r/m,imm
Basic flag operations (opcodes F8-FD).
INC r/m8/16/32 instructions(opcodes FE-FF).
32-bit relative jumps(0F80-0F8F instruction range).

Side note: opcode 82h is missing from the intel-opcodes list, which is an alias for opcode 80h( http://ref.x86asm.net/coder32.html#x82 ).

superfury commented 6 years ago

Just created a little 'testsuite' that executes many of the basic instructions mentioned in my previous post(although it requires to be logged and verified manually by looking for register/memory reads/writes/changes): https://bitbucket.org/superfury/unipcemu/src/9c54c037466f0e079e76a35987584b0687d42af8/UniPCemu/assembly/?at=master

Although I've coded opcodes 00-3F by using raw binary statements to make sure the nasm assembler doesn't actually create opcodes outside of that range somehow(2-byte versions of them).

The only test that actually verifies itself using assembly instructions is the ret imm testing, much in the way your stack tests run.

Edit: https://bitbucket.org/superfury/unipcemu/src/6c46742934fc8ffba70de4e139671244164270e9/UniPCemu/assembly/?at=master

Fixed some bugs in the basic *D opcodes for the 00-3F range being the wrong opcode byte.

superfury commented 6 years ago

So, combining both testsuite results, that only leaves possible errors in:

Logical operations in the 80-83 range**.
String input/output from port in the 6C-6F range, probably testable using e.g. the temporary DMA registers(unused registers which aren't used on the DMA chipset of the DMA Page Registers).
TEST instructions(opcode 84-85).
MOV instructions(opcode 88-8B).
SAHF/LAHF instructions (opcode 9E-9F).
32-bit address-size string instructions(B/W/D).
Different variants of SHLD/SHRD with operand-sizes.
MOV r/m,imm(opcodes C6-C7).
ENTER/LEAVE(opcodes C8&C9).
XLAT (opcode D7).
LOOP/J(E)CXZ instructions 32-bit operand size(opcodes E0-E3)**.
IN/OUT instructions(opcodes E4-E7, EC-EF), see string i/o instructions.
JMP instructions(opcodes E9-EB).
REP/REPE/REPNE (opcodes F0-F3).
Basic flag operations (opcodes F8-FD).
INC r/m8/16/32 instructions(opcodes FE-FF).
32-bit relative jumps(0F80-0F8F instruction range).

** partially tested and verified in both testsuites

superfury commented 6 years ago

Just found a little 'bug' in my emulator(which seems to be only half documented in the 80386 programmer's reference manual's far return instruction description). (E)SP is increased before popping SS:(E)SP during a stack switch to an outer privilege level(resulting CS.RPL>CPL), which is documented, but also after popping the SS:(E)SP(thus, increasing(popping) the caller's stack variables. The latter isn't documented within the 80386 programmer's reference manual as far as I can see.

superfury commented 6 years ago

Having fixed the far return using the immediate on both stack locations(both on source and destination stacks), the Extended Memory Tester v3.0 now properly detects the video card and extended memory, continuing to test memory:D

This app now runs properly: https://archive.org/details/msdos_TESTEXT3_shareware

superfury commented 6 years ago

Hmmmm..... One basic device that can be used to test the ISNS/OUTS instructions is the IDE/ATA harddisk's buffer(Command E4 to read the buffer, Command E8 to write the buffer). First fill it with a pattern manually(512 bytes), then read it back and verify the pattern. That should be able to verify all in(s)/out(s) instructions.

Edit: Although I know some of them work properly(8-bit plain, 8-bit string and 16-bit string) work already, since they're used during POST(8-bit plain), during disk reads(8-bit or 16-bit reads), disk writes(8-bit or 16-bit writes) and CD-ROM access(16-bit reads and writes). The only one of those still untested are string and normal 32-bit variants and normal 16/32-bit single input/output.

superfury commented 6 years ago

I see you're busy on some more (protected mode) tests. Great!

That now just leaves:

String input/output from port in the 6C-6F range, probably testable using e.g. the temporary DMA registers(unused registers which aren't used on the DMA chipset of the DMA Page Registers) and hard disk buffer(using the read/write buffer commands).
MOV instructions(opcode 88-8B).
SAHF/LAHF instructions (opcode 9E-9F).
MOV r/m,imm(opcodes C6-C7).
ENTER/LEAVE(opcodes C8&C9).
XLAT (opcode D7).
LOOP/J(E)CXZ instructions 32-bit operand size(opcodes E0-E3)**.
IN/OUT instructions(opcodes E4-E7, EC-EF), see string i/o instructions.
JMP instructions(opcodes E9-EB).
Basic flag operations (opcodes F8-FD).
32-bit relative jumps(0F80-0F8F instruction range).

So mostly various kinds of move instructions and miscellaneous instructions.

And of course all remaining protected-mode functionality itself(task switching etc.) is still left to test.

I'm currently wondering if protected-mode functionality of UniPCemu still has bugs(concerning call gates and interrupts etc. pushing data on the stack with(out) stack switch) in various cases.

barotto commented 6 years ago

I've found a pretty big bug on my emulator comparing its memory dump with Bochs' after the execution of the test suite: the Dirty bit of a PTE is not properly set on a write when the same page has been previously accessed by a read. So there are a lot of behavioural tests to be implemented beside the various opcodes...

superfury commented 6 years ago

I've just ran Bochs 2.6.9 and dumped the 640K memory. Then I did the same with UniPCemu's MMU functionality(running the debugger, press and hold circle and then tap square) when reaching the HLT at the end of the POST 0xFF(when reaching the HLT status).

It reveals there's quite a lot wrong, apparently:

00000401 C2 00
00000402 10 00
00000405 8E 00
000020B8 67 07
000023E0 27 07
000023E4 27 07
000023E8 27 07
0000FFFC F5 00
0000FFFD FF 00
0000FFFF F0 00
0001FFB7 05 46
0001FFBB 05 32
0001FFBF 56 00
0001FFC2 00 80
0001FFC3 D3 00
0001FFC4 72 40
0001FFC7 58 DB
0001FFC8 73 FF
0001FFCB 38 00
0001FFCC AD 00
0001FFCF 0F 14
0001FFD0 AB 00
0001FFD1 00 FF
0001FFD2 00 FF
0001FFD3 01 00
0001FFD4 00 C0
0001FFD7 EB 05
0001FFD8 FF 00
0001FFDB 04 0D
0001FFDC AD 6F
0001FFDF 20 32
0001FFE2 80 00
0001FFE3 07 00
0001FFE4 08 00
0001FFE6 00 80
0001FFE7 94 00
0001FFE8 72 40
0001FFED FF 00
0001FFEE FF 00
0001FFEF 07 00
0001FFF0 08 01
0001FFF1 00 01
0001FFF3 08 84
0001FFF4 AD 00
0001FFF5 00 FF
0001FFF6 00 FF
0001FFF7 0D FF
0001FFF8 00 FF
0001FFF9 00 FF
0001FFFA 00 FF
0001FFFB 13 6E
0001FFFC 00 75
0001FFFD 00 57
0002E000 01 00
0002E001 FF 00
0002E002 FF 00
0002E003 FF 00
0009F003 00 50

So that means:

First IDT entry has an issue.
Page table entry issues? Seems to be the Accessed bit not being updated correctly?
Stack overflow? (FFFC-FFFF)
More stack issues? (1FFB7-1FFFD)
Normal test issues? (2E000-2E003 and 9F003)

barotto commented 6 years ago

First IDT entry has an issue.

That's the IDT area, but it's not necessarily a problem with the IDT management, could be something that writes garbage at the wrong address.

Page table entry issues?

Possible.

Stack overflow? (FFFC-FFFF)

Maybe, or again you could be writing garbage to the wrong addresses (see below).

More stack issues? (1FFB7-1FFFD)

Maybe.

Normal test issues? (2E000-2E003 and 9F003)

Can't tell right now what the segment at 0x2e000 is used for, but dword at 0x9F000 should be 0x50465046, for some reason you're missing the last byte 0x50 at 0x9F003, or you have overwritten it with 00.

It seems like you have problems with the MMU.

superfury commented 6 years ago

When debugging the direct memory writes by the Paging unit, I see:

Write to address 0x000023e0 value 0x000f8027.
Write to address 0x000023e4 value 0x000f9027.
Write to address 0x000023e8 value 0x000fa027.
Write to address 0x000020b8 value 0x0002e067 (at step EE).

So the Paging unit requests the memory mapping to actually write the correct values there at some point. Maybe something else is somehow overwriting it? Edit: Or maybe there's an issue in the writeback to memory after said breakpoint... Hmmmm... Edit: Already confirmed that the 0x23E0 memory location receives and stores value 0x27. So it's another bug that's somehow overwriting this value? Edit: Just added those addresses hardcoded to the MMU unit. Let's see if they're overwritten with different values normally(thus not an general emulator bug). Edit: Looking directly in the capture of the physical RAM(with memory gaps having been removed when dumping the RAM), the values written are actually in RAM at their supposed locations?

Edit: Looking again at my own created logs(made using MinGW-w64 using the following script), from UniPCemu's ROM directory(which contains the Bochs dump as well as the ROMs):

cp "..\captures\memory.dat" "fullmemory.dat"
dd if=/dev/null of=fullmemory.dat bs=1 count=1 seek=655360
cmp -l fullmemory.dat bochs-640k-memdump.bin | gawk '{printf "%08X %02X %02X\n", ($1-1), strtonum(0$2), strtonum(0$3)}'>memory.cmp.txt

It copies the memory.dat to the ROM directory for processing, then edits it to become 640K large, finally executing the difference dump as per your documentation.

I thought that the left was the Bochs dump, while the right value was my own. Looking at it again(as well as reading your documentation on the results of the gawk command again), it's reversed.

So those three pages aren't supposed to be accessed(since bits 5(PTE&PDE)&6(PDE) are supposed to be cleared instead of set in the end of logging). But they're accessed anyways, which isn't supposed to happen? So those paged accesses on those locations are instructions addressing a wrong point in memory, or incorrect mapping somehow?

barotto commented 6 years ago

PTE at 0x23e0 is for the page at 0xF8000. Its Accessed bit should be 0, not 1 (byte at 0x23e0 should be 0x07 not 0x27). It seems like you're fetching the NOPs that are present in that area of the test code. Or maybe your MMU is updating the wrong PTE.

I also think the dword at 0x9F000 is interesting. See what is wrinting 00 at 0x9F003, overwriting the 0x50.

superfury commented 6 years ago

About 9F003, I see 0x50 being written at 10:0000C455. Then 23E0 written 0x27 at instruction 0010:7FFB. Then 23E4 written 0x27 at instruction 0010:8FFE. Then 23E8 written 0x27 at instruction 0010:98A8. Then 20B8 written 0x67 at instruction 0010:99F7.

The current testsuite lst used: test386.zip

superfury commented 6 years ago

Just found a bug that caused the highest written byte to memory to remain unlogged(it was storing size-1 in the memory usage variable, instead of size). It still needed to add 1 to the current address for detection of memory usage, which it didn't(used for logging purposes only). Since address 9F003 is the last byte in the memory capture, that one was missing and thus filled with zeroes by the dd-command. Edit: It's gone now:

00000401 C2 00
00000402 10 00
00000405 8E 00
000020B8 67 07
000023E0 27 07
000023E4 27 07
000023E8 27 07
0000FFFC F5 00
0000FFFD FF 00
0000FFFF F0 00
0001FFB7 05 46
0001FFBB 05 32
0001FFBF 56 00
0001FFC2 00 80
0001FFC3 D3 00
0001FFC4 72 40
0001FFC7 58 DB
0001FFC8 73 FF
0001FFCB 38 00
0001FFCC AD 00
0001FFCF 0F 14
0001FFD0 AB 00
0001FFD1 00 FF
0001FFD2 00 FF
0001FFD3 01 00
0001FFD4 00 C0
0001FFD7 EB 05
0001FFD8 FF 00
0001FFDB 04 0D
0001FFDC AD 6F
0001FFDF 20 32
0001FFE2 80 00
0001FFE3 07 00
0001FFE4 08 00
0001FFE6 00 80
0001FFE7 94 00
0001FFE8 72 40
0001FFED FF 00
0001FFEE FF 00
0001FFEF 07 00
0001FFF0 08 01
0001FFF1 00 01
0001FFF3 08 84
0001FFF4 AD 00
0001FFF5 00 FF
0001FFF6 00 FF
0001FFF7 0D FF
0001FFF8 00 FF
0001FFF9 00 FF
0001FFFA 00 FF
0001FFFB 13 6E
0001FFFC 00 75
0001FFFD 00 57
0002E000 01 00
0002E001 FF 00
0002E002 FF 00
0002E003 FF 00

superfury commented 6 years ago

The first two are instruction fetches triggering the accessed being set during writeback of the Page tables. 0x23E8 is the memory operand of the MOV at said address, offset AD7C.

superfury commented 6 years ago

It's checking an instruction fetch operand there at 10:7FFB. It's checking a DWORD at that location. A part of said operand is at F8000. It's the byte after 0F85. The PDE is 000023E0. The PTE to write back is 0x000f8027. Edit: The test386.lst confirms the 0x8000 byte is the final byte of the instruction(the jne error instruction, which is a 0F85 imm32 instruction).

It's the final byte of line 15393 in my test386.lst file.

barotto commented 6 years ago

About 9F003, I see 0x50 being written at 10:0000C455.

~~That's ok, the problem is that your dump at POST FFh shows a 0x00.~~ resolved

Then 23E0 written 0x27 at instruction 0010:7FFB.

~~Pages 0xF8000-0xFA000 shouldn't be accessed.~~ they should

Then 20B8 written 0x67 at instruction 0010:99F7.

This seems correct? IT IS

superfury commented 6 years ago

You say that Paged memory at 0xF8000-FA000 shouldn't be accessed, but there's executable code there that's executing in my case(as well as being present in the test386.lst).

The test386.lst says this about said instruction:

  1055 00007FFB 0F85FF3F0000        <2>  jne error

It seems to be one of the arithmetic tests that's executed.

The entire block of nasm code:

  1055                              <1>  testBittestWFlags btr, %1, %2, %3, %4
  1055                              <2> 
  1055 00007FBC 66B80100            <2>  mov ax, %4
  1055 00007FC0 6650                <2>  push ax
  1055 00007FC2 9D                  <2>  popf
  1055 00007FC3 66B80100            <2>  mov ax, %2
  1055 00007FC7 660FBAF001          <2>  o16 %1 ax, %3
  1055 00007FCC 9C                  <2>  pushf
  1055 00007FCD 6658                <2>  pop ax
  1055 00007FCF 6625D508            <2>  and ax, PS_ARITH
  1055 00007FD3 663D0008            <2>  cmp ax, %5
  1055 00007FD7 0F8523400000        <2>  jne error
  1055                              <2> 
  1055                              <2> 
  1055 00007FDD 66B80100            <2>  mov ax, %4
  1055 00007FE1 6650                <2>  push ax
  1055 00007FE3 9D                  <2>  popf
  1055 00007FE4 66B80100            <2>  mov ax, %2
  1055 00007FE8 66B90100            <2>  mov cx, %3
  1055 00007FEC 660FB3C8            <2>  o16 %1 ax, cx
  1055 00007FF0 9C                  <2>  pushf
  1055 00007FF1 6658                <2>  pop ax
  1055 00007FF3 6625D508            <2>  and ax, PS_ARITH
  1055 00007FF7 663D0008            <2>  cmp ax, %5
  1055 00007FFB 0F85FF3F0000        <2>  jne error

That's bit_m.asm, row 85 being assembled. Edit: Looking further up the inclusion list, it's test386.asm row 1052. In test386.asm, I see it's part of the E0 undefined instruction tests.

barotto commented 6 years ago

I'm getting confused by all the edits. PLEASE verify that 2nd column are the values of your emu, 3rd column are Bochs' Because I'm starting to think your post shows the opposite...

superfury commented 6 years ago

I've recompiled the test386.asm source code(it uses the BOCHS version and 386-specific tests(POST E0) enabled).

As I said in my last post, the strange page (which has a PTE/PDE located at paged address F8000 and onwards) is part of the E0 tests that are executing and are present in the test386.lst generated by the nasm compiler.

This is the result of UniPCemu's memory dump(which is a direct dump of physical RAM, which is zero-padded to 640K) compared against the Bochs memory dump, as in your instructions at the test386.asm main code page(Readme).

00000401 C2 00
00000402 10 00
00000405 8E 00
000020B8 67 07
000023E0 27 07
000023E4 27 07
000023E8 27 07
0000FFFC F5 00
0000FFFD FF 00
0000FFFF F0 00
0001FFB7 05 46
0001FFBB 05 32
0001FFBF 56 00
0001FFC2 00 80
0001FFC3 D3 00
0001FFC4 72 40
0001FFC7 58 DB
0001FFC8 73 FF
0001FFCB 38 00
0001FFCC AD 00
0001FFCF 0F 14
0001FFD0 AB 00
0001FFD1 00 FF
0001FFD2 00 FF
0001FFD3 01 00
0001FFD4 00 C0
0001FFD7 EB 05
0001FFD8 FF 00
0001FFDB 04 0D
0001FFDC AD 6F
0001FFDF 20 32
0001FFE2 80 00
0001FFE3 07 00
0001FFE4 08 00
0001FFE6 00 80
0001FFE7 94 00
0001FFE8 72 40
0001FFED FF 00
0001FFEE FF 00
0001FFEF 07 00
0001FFF0 08 01
0001FFF1 00 01
0001FFF3 08 84
0001FFF4 AD 00
0001FFF5 00 FF
0001FFF6 00 FF
0001FFF7 0D FF
0001FFF8 00 FF
0001FFF9 00 FF
0001FFFA 00 FF
0001FFFB 13 6E
0001FFFC 00 75
0001FFFD 00 57
0002E000 01 00
0002E001 FF 00
0002E002 FF 00
0002E003 FF 00

The second column IS confirmed to be UniPCemu's data in memory. The third column is Bochs' dump.

test386.lst.zip

barotto commented 6 years ago

BTW dword at 0x2E000 is currently used as scratch memory for arith-logic tests 0xEE. Memory is accessed as DS:[0], where DS=0x14, with base=0x2E000

superfury commented 6 years ago

Just ran the testsuite on Bochs again. Ran the continue command to let it run until the permanent HLT. Then used Ctrl-C to break into the debugger. It's not a problem with UniPCemu there! Bochs errors out at the processor-specific arithmetic logic tests! It's at 0010:0000C002(the error jumped location) when it's processing those. So Bochs is having a problem with those tests(the E0 tests)!

00001382369e[CPU0  ] write_virtual_checks(): no write access to seg
00001382421e[CPU0  ] read_virtual_checks(): read beyond limit
00001382471e[CPU0  ] write_virtual_checks(): write beyond limit, r/w
00001382521e[CPU0  ] read_virtual_checks(): read beyond limit
00001382571e[CPU0  ] write_virtual_checks(): write beyond limit, r/w
00001385741i[CPU0  ] WARNING: HLT instruction with IF=0!
64799855992i[      ] Ctrl-C detected in signal handler.
Next at t=64813965152
(0) [0x0000000fc002] 0010:000000000000c002 (unk. ctxt): jmp .-4 (0x000fc000)      ; ebfc
<bochs:2>

Edit: Having disabled those tests using the flag, I now see Bochs dumping all EE test results into it's debugger window.

barotto commented 6 years ago

The second column IS confirmed to be UniPCemu's data in memory. The third column is Bochs' dump.

Are you absolutely 100% sure? Because if I compare your post with a Bochs dump I've done, the 2nd middle column seems exactly like Bochs' data. For example the correct value for dword 2E000 is 0xFFFFFF01.

superfury commented 6 years ago

Well, I know for sure that the dumps of UniPCemu's Paging table locations contain those values in the dump and memory variable (0x27 and 0x67). So the second column is the UniPCemu memory and the third column is the Bochs memory. The first column is the address, the second column is UniPCemu, the third column is the crashing Bochs output.

barotto commented 6 years ago

So Bochs is having a problem with those tests(the E0 tests)!

Yes, 0xE0 are the undefined behaviours tests. Bochs is not a faithful replica of the i80386. Disable TEST_UNDEF and rerun the tests. Thanks for pointing out that test E0h should be disabled before comparing memory dumps. I'll add a note in the readme.

superfury commented 6 years ago

This is my new dump of the comparison of UniPCemu and Bochs after having disabled said test:

0000FFFC F5 00
0000FFFD FF 00
0000FFFF F0 00
0001FFBF 56 46
0001FFE4 08 00
0001FFF0 08 00

Only a few bugs left! :D

barotto commented 6 years ago

:D Now seems the stack has still some issues.... Keep me posted.

superfury commented 6 years ago

OK.

First it writes value F0 to address FFFF at F000:FFF0. Also, FFFC=F5. Also FFFD=FF.

Thinking about that, it's the internal BIOS of UniPCemu performing it's startup routine(the CPU actually starts before the internal BIOS is handled, performing a CPU escaped instruction to make the emulator start it's yellow text boot, as well as providing internal interrupt handling(for when using the internal BIOS)).

So I'll need to not only reset the CPU(which already happens when starting the test386.asm ROM), but also clear any memory present(as it's been temporarily been modified to let the emulator's ROM run it's startup routine).

Said routine also loads the ROMs into memory, then resumes execution while resetting the CPU.

That explains the first half of said data. Luckily an easy to fix problem. Edit: Should be fixed now. Applied a full emulator reset before the emulator has loads the ROMs into memory. Then, the CPU reset afterwards has been removed, as the emulator reset has already performed this.

superfury commented 6 years ago

Having fixed said CPU reset to become an emulator reset, the first 64KB of errors disappears.

0001FFBF 56 46
0001FFE4 08 00
0001FFF0 08 00

Those are still left.

superfury commented 6 years ago

Added a little log of accesses to that location. The final writes(during the EE tests) show them being written:

00:04:17:35.05040: RealMMU 0001FFBF=56 at 0010:00007206
00:04:17:35.05088: RealMMU 0001FFF0=98 at 0010:00009859
00:04:17:35.05104: RealMMU 0001FFE4=08 at 0010:00007206
00:04:17:35.05440: RealMMU 0001FFF0=08 at 0010:0000985E

superfury commented 6 years ago

The first is PUSHFD at row 13851 of the test386.lst. It's the start of the printChar function. The second is CALL(opcode E8) at row 17791 of test386.lst. (call printEOL). The third is PUSHFD at row 13851 of test386.lst. It's the start of the printChar function. The fourth is PUSHFD at row 17792 of test386.lst. It's the pushfd directly after the above call printEOL.

Looking at a diff of the port EE logs, now 0FAC SHRD W/D and 0FAD SHRD W/D are both giving errors compared to port-EE-reference!

superfury commented 6 years ago

OK. The general instruction handlers of those two SHRD instructions(0FAC and 0FAD) are the same as SHLD(0FA4 and 0FA5), except running two different functions, depending on said instruction(shift left or shift right).

Looking at those instructions, I already see some Overflow flag problems right there.

superfury commented 6 years ago

After fixing both the overflow flag problems and invalid variable used problems(word vs dword variants intermixing variables incorrectly, both SHLD and SHRD now correctly pass the tests. The diff of the port E9/EE logs gives no differences anymore! :D

superfury commented 6 years ago

Oddly enough, the memory logs still differ on those final EFLAGS being written to RAM(and function return address)?

superfury commented 6 years ago

How can the function return address be different when it's correct(the highest ESP of those addresses)? Looking at what the function pushes on the stack it matches said byte of the return address(of the call instruction)? Shouldn't the return address be pushed at an empty stack(SP being zeroed) during the call with each of those functions that print the flags, register numbers and line break to the log?

superfury commented 6 years ago

Looking at the code again, at the point it reaches those call instructions, SP should be written at FFF4 during the CALL(FFFC has EBP, FFF8 has EDI, FFF4 the return address of the call. Hmmmm.... So that 1FFF0 should be FFF4? Assuming the initial ESP for the EE test is 20000?

Edit: Correction: 1FFF0 IS correct, since before EBP and EDI is also ECX!

So, at the end, the stack contains as follows: 1FFFC=Original ECX(Length of instruction) 1FFF8=Original EBP(Count of values for src) 1FFF4=Original EDI(Values for src) 1FFF0=Last CALL return address.

So why are those values incorrect, according to the memory dump?

superfury commented 6 years ago

Hmmm.. 00009859 doesn't exist, but 98E9 does. Same for 0000985E instead of 98EE? So said values pushed should be EE at 1FFF0? So EE,98,00,00 in the dword range?

Edit: But, directly after those calls is a pushfd, popfd, which overwrites that 1FFF0 byte with the lower byte of the FLAGS register. So it isn't the return address, but the lowest byte of EFLAGS being incorrect? Athough the PS= part of the log is correct?

Edit: This shouldn't be possible. Bit 1 should always be set and bits 3&5 always be cleared in EFLAGS? So 08 in that position should be 02?

superfury commented 6 years ago

Now trying installing Windows 95 on UniPCemu latest commit. So far scandisk ran, setup files copied, entered graphical interface(32-bit protected mode w/ paging?) and clicked the next button. Now setup is loading the wizard. Serial mouse is working w/o drivers:D

Edit: Now running the part where it's checking installed parts. Edit: Checking installed parts 100%. (PC entering serial key). Edit: Checking available disk space 75% (PC analyzing system 69%) Edit: Preparing to copy the files.

Continuing this on the vogons forums thread (https://www.vogons.org/viewtopic.php?f=9&t=56440&p=694173#p694173)

superfury commented 6 years ago

Well, after the reboot(for setup continuation), it eventually crashes afrer updating the registry on executing segment 1E in real mode?

superfury commented 6 years ago

Just a little question: what is the value of SS:ESP supposed to be during the various parts of the EE-tests? That way I can see if ESP is incorrect at that point to begin with or that the issue is within the EE-tests.

Edit: Placed a breakpoint at 0010:000097EA and POST EE to find out if SS:ESP is correct...

Edit: Both have SS:ESP set to 0050:FFFF? Is that even correct? Edit: Looking at the source code, 0050:FFFF is an initialized SS:ESP value that's loaded through the descriptor in memory(using LSS). So that should be correct.

The GDT describes SS as having a base of 0x10000.

So: 0050:FFFB=ECX(physical 1FFFB) = Length of instruction sequence 0050:FFF7=EBP(physical 1FFF7) = Count of values for src(typeValues final four lines, the SHIFTS_BYTES_R(+SHIFT_WORDS_R(+SHIFT_DWORDS_R))) value 0050:FFF3=EDI(physical 1FFF3) = Values for src(shiftsValuesR according to the table) 0050:FFEF=Last EFLAGS result(after CALLs to printing functions, within the printing functions and execution handler, it's actually the return point of the CALL) 0050:FFEB=??? 0050:FFE7=??? 0050:FFE3=??? 0050:FFDF=??? 0050:FFDB=??? 0050:FFD7=??? 0050:FFD3=??? 0050:FFCF=??? 0050:FFCB=??? 0050:FFC7=??? 0050:FFC3=??? 0050:FFBF=???

So the errors are in:

SS:FFBF: 0001FFBF. 56 instead of 46. This is actually 4 levels within a printing handler or execution handler? SS:FFE4: 0001FFE4. 08 instead of 00. byte 1 of the Count of values for src SS:FFF0: 0001FFF0. 08 instead of 00. Byte 1 of the last EFLAGS result. So an overflow flag instead of no overflow flag?

So is the instruction itself going wrong, or is the processing before or after it going wrong?

Edit: Probably the only way to find out what's going wrong is to place a breakpoint at 0010:0097EA and let it run with full debugging(common log format) to the end. Then look at what it was doing during the last instruction and why it's aborting?

superfury commented 6 years ago

It looks like logging the entire EE test is asking for too much. After 16.9GB(1288KB E9 logged) debugger log, the hard disk I was running it on ran full.

Guess I'll change the breakpoint to start at 0010:0000AA7F(the first instruction of the final SHRD loop tested instruction). Maybe that'll give something interesting.

This is the result of the SHRD instruction test up till it reaches POST FF: debugger_finaltestedinstruction.zip

I've moved the 7-zip file inside a zip archive for uploading it here(it doesn't support 7z files).

superfury commented 6 years ago

Looking at the code, it seems to properly terminate correctly. Can you see what's going wrong? I suggest starting at the bottom (the CLI of POST FF) of the file and working your way upwards.

I noticed I can keep searching for the same block by searching upwards for the first 6 hexadecimal characters(the final two being the instruction within the range of the main loop, 0010:000098XX mostly and 0010:000097XX for the top of the loop).

I see the EFLAGS being popped from SS:FFBF at line 5389696, earlier being pushed at line 5389649? EFLAGS=00000056 at that point?

Edit: Those EFLAGS have been set at line 5389605, opcode 84C9 TEST cl,cl at 0010:00007238. That's the test cl,cl within the printVal function? Edit: It's called by printPS.

superfury commented 6 years ago

Looking at your IBMulator code, I see TEST clears the auxiliary flag? But UniPCemu leaves it alone, giving it thr result of the last arithmetic instruction documented to set/clear it(in this case CMP AL,39 with AL being 0x30?

Edit: So all logical instructions, according to Bochs, clear the Auxiliary flag(and, test, or & xor)? Edit: According to Bochs, MUL too. Edit: UniPCemu luckily has a general handler(flag_log8/16/32) for all of those cases. Thus easily implemented. That's at least one bug out of the window.

superfury commented 6 years ago

Great, that just fixes that one case.

Remaining cases:

0001FFE4 08 00
0001FFF0 08 00

Hmmmm... debugger_remainingOverflowFlag.zip

Row 5162676 doesn't show the Overflow flag being set. But the next instance and onwards DOES have the overflow flag set, which is seemingly incorrect?

Edit: Hmmmmm... The SHRD at line 5189674 seems to set the Overflow flag?

0010:0000aa88 0F AD D0 shrd eax,edx,cl  Physical(p):000FAA96=00( ); Paged(p):000FAA96=00( ); Normal(p):0000AA96=00( ); Physical(p):000FAA97=00( ); Paged(p):000FAA97=00( ); Normal(p):0000AA97=00( ); Physical(p):000FAA98=D5(Õ); Paged(p):000FAA98=D5(Õ); Normal(p):0000AA98=D5(Õ); Physical(p):000FAA99=08(); Paged(p):000FAA99=08(); Normal(p):0000AA99=08(); Physical(p):000FAA9A=00( ); Paged(p):000FAA9A=00( ); Normal(p):0000AA9A=00( )
Registers:
EAX: 80000000 EBX: 0000ac68 ECX: 00000001 EDX: 00000001
ESP: 0000ffef EBP: 0000000c ESI: 0000aa7f EDI: 0000ac7c
CS: 0010 DS: 0014 ES: 001c FS: 0048 GS: 0014 SS: 0050 TR: 0000 LDTR: 0030
EIP: 0000aa88 EFLAGS: 00000003
CR0: 80000001 CR1: 00000000 CR2: 0009f000 CR3: 00001000
DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
DR6: 00000000 DR7: 00000000
GDTR: 00000000050002ff IDTR: 00000000040000ff
FLAGSINFO: 00000000000000vr0n00oditsz0a0p1C 
    RealRAM(r):0002E000=04(); RAM(r):0002E000=04(); Physical(r):0002E000=04(); Paged(r):0002E000=04(); Physical(p):000FAA9B=00( ); Paged(p):000FAA9B=00( ); Normal(p):0000AA9B=00( ); Physical(p):000FAA9C=D5(Õ); Paged(p):000FAA9C=D5(Õ); Normal(p):0000AA9C=D5(Õ); Physical(p):000FAA9D=08(); Paged(p):000FAA9D=08(); Normal(p):0000AA9D=08(); Physical(p):000FAA9E=00( ); Paged(p):000FAA9E=00( ); Normal(p):0000AA9E=00( ); Physical(p):000FAA9F=00( ); Paged(p):000FAA9F=00( ); Normal(p):0000AA9F=00( ); Physical(p):000FAAA0=C5(Å); Paged(p):000FAAA0=C5(Å); Normal(p):0000AAA0=C5(Å)
0010:0000aa8b 8A 0D 00 00 00 00 mov cl,byte ds:[00000000]
Registers:
EAX: c0000000 EBX: 0000ac68 ECX: 00000001 EDX: 00000001
ESP: 0000ffef EBP: 0000000c ESI: 0000aa7f EDI: 0000ac7c
CS: 0010 DS: 0014 ES: 001c FS: 0048 GS: 0014 SS: 0050 TR: 0000 LDTR: 0030
EIP: 0000aa8b EFLAGS: 00000886
CR0: 80000001 CR1: 00000000 CR2: 0009f000 CR3: 00001000
DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
DR6: 00000000 DR7: 00000000
GDTR: 00000000050002ff IDTR: 00000000040000ff
FLAGSINFO: 00000000000000vr0n00OditSz0a0P1c

superfury commented 6 years ago

After fixing the overflow flag to (on 1-bit shifts) be the XOR of the MSB and SMSB, like in IBMulator and Bochs, those final two memory errors also disappear! :D

Now the entire software runs without any known memory oddities at the end and with 100% identical port E9 logs! :D

superfury commented 6 years ago

Gotten strange issues with Windows 95, which faults #UD faults(opcode FEFF) somewhere after the CPU detection(using the 0FAF opcode) which eventually result in SS in real mode being in the unavailable RAM area(C000-FFFF segments), thus eternally faulting to FFFF:FFFF.

superfury commented 6 years ago

Looking at the disassembly, I see it popping FFFF into SS at FFFF:0205? Thus producing an invalid stack? It was ECD0(SS=0ECD) before that?

superfury commented 6 years ago

Well, so far one good thing came of the latest bugfixes: Day of the Tentacle is now finally running! (Although it complains about only having 20K free memory when starting it). So far the opening sequence runs at least! :D

superfury commented 6 years ago

I've been trying to get Debian running on UniPCemu to actually run the sandsifter testsuit in that, but it already fails with running loadlin.exe undef MS-DOS(no matter what the size of the memory installed. Tried 4MB and 12MB(12288KB according to the Compaq BIOS) RAM installed, both error out with a message says it doesn't have enough RAM).

Guide used: https://hackaday.com/2011/08/12/installing-linux-on-a-386-laptop/

Windows 95 still crashes with a #UD at 4EE8:92B1 executing opcode 0x63,0x20. That raises a #UD always in non-protected(real/V86) mode?

superfury commented 6 years ago

Just tried again with the relative jumps and absolute jumps being typecasted to int_32(relative offsets) and uint_32(absolute offsets). Now I don't get the ARPL fault, but instead receive an incorrect opcode 8F /3(instruction 8F5A3B), which is an incorrect instruction, at 0146:0004?

Edit: Hmmm.... BOOTLOG.TXT contains some interesting information:

[0009CA95] Loading Device = C:\WINDOWS\HIMEM.SYS
[0009CA97] LoadSuccess    = C:\WINDOWS\HIMEM.SYS
[0009CA97] Loading Device = MSDOS\CD1.SYS
[0009CAB0] LoadSuccess    = MSDOS\CD1.SYS
[0009CAB0] Loading Device = C:\WINDOWS\COMMAND\DISPLAY.SYS
[0009CAB1] LoadSuccess    = C:\WINDOWS\COMMAND\DISPLAY.SYS
[0009CAB2] Loading Device = C:\WINDOWS\IFSHLP.SYS
[0009CAB2] LoadSuccess    = C:\WINDOWS\IFSHLP.SYS
[0009CAB3] Loading Device = C:\WINDOWS\SETVER.EXE
[0009CAB4] LoadSuccess    = C:\WINDOWS\SETVER.EXE
[0009CABF] C:\WINDOWS\COMMAND\MSCDEX.EXE[0009CABF]  starting
[0009CAC8] C:\WINDOWS\COMMAND\MODE.COM[0009CAC8]  starting
[0009CACF] C:\WINDOWS\COMMAND\MODE.COM(Logo disabled)
[0009CACF]  starting
[0009CD74] Loading Vxd = VMM
[0009CD8F] LoadSuccess = VMM
[0009CD90] Loading Vxd = C:\WINDOWS\SMARTDRV.EXE
[0009CD90] LoadSuccess = C:\WINDOWS\SMARTDRV.EXE
[0009CD95] Loading Vxd = CONFIGMG
[0009CD9F] LoadSuccess = CONFIGMG
[0009CDA0] Loading Vxd = VSHARE
[0009CDA3] LoadSuccess = VSHARE
[0009CDA3] Loading Vxd = VWIN32
[0009CDA8] LoadSuccess = VWIN32
[0009CDA8] Loading Vxd = VFBACKUP
[0009CDAA] LoadSuccess = VFBACKUP
[0009CDAA] Loading Vxd = VCOMM
[0009CDAC] LoadSuccess = VCOMM
[0009CDAC] Loading Vxd = COMBUFF
[0009CDAE] LoadSuccess = COMBUFF
[0009CDAE] Loading Vxd = C:\WINDOWS\system\VMM32\IFSMGR.VXD
[0009CDB7] LoadSuccess = C:\WINDOWS\system\VMM32\IFSMGR.VXD
[0009CDB7] Loading Vxd = C:\WINDOWS\system\VMM32\IOS.VXD

Oddly enough, it's running in real mode?

barotto / test386.asm

Test every non tested opcode #5