search

cebix / macemu

Basilisk II and SheepShaver Macintosh emulators
1.38k stars 289 forks source link

Map framebuffer into accessible memory in 24-bit mode #131

Closed simon-frankau closed 6 years ago

simon-frankau commented 7 years ago

The framebuffer wasn't working in 24-bit mode for me. This patch rejigs the 24-bit address space so that the framebuffer is mapped in where the ROM expects it to be.

rickyzhang82 commented 7 years ago

Out of curiosity, I wonder:

  1. What ROM do you refer to in your context? Can you provide md5 hash as well.
  2. Assuming you build BII with virtual addressing, a.k.a mem bank, how can you be so sure that frame buffer address in guest OS starts at the place you specified: 
    map_banks(&fram24_bank, ((RAMBaseMac + ram_size) >> 16) - 1, 1);

    The reason I'm asking is that in some ROM like 64k, Plus and Classic it placed ROM at 4MB in guest OS. ram_size is specified in BII emulation. So how do you prevent frame buffer overwriting ROM?

0080     RAMBaseMac = 0;
0081     switch (ROMVersion) {
0082         case ROM_VERSION_64K:
0083         case ROM_VERSION_PLUS:
0084         case ROM_VERSION_CLASSIC:
0085             ROMBaseMac = 0x00400000;
0086             break;
0087         case ROM_VERSION_II:
0088             ROMBaseMac = 0x00a00000;
0089             break;
0090         case ROM_VERSION_32:
0091             ROMBaseMac = 0x40800000;
0092             break;
0093         default:
0094             return false;
0095     }
0096     memory_init();
simon-frankau commented 7 years ago

I tried the following ROMs:

These worked successfully with my patch (and didn't work for me before).

I also tried the following ROMs, but they didn't appear to be supported by the existing code ('STR_UNSUPPORTED_ROM_TYPE_ERR'):

So how do you prevent frame buffer overwriting ROM?

The key point is the ' - 1' in the second parameter to map_banks - we're making the last 64K of RAM into the framebuffer. It'll never overwrite the ROM as it always coincides with RAM (I believe this series of machines were limited to 4MB, so that ROM and RAM wouldn't conflict).

Why do I think the last 64K of RAM holds the framebuffer? I must admit, I'm not very scientific here. http://www.1000bit.it/support/articoli/apple/mac128.pdf puts the framebuffer at the end of RAM for that model. Observing the emulated 4MB Classic shows the framebuffer at the upper end of the 4MB. It's basically my assumption that it's at the upper end of RAM.

I haven't tried changing the memory size parameter and seeing if the framebuffer location moves too. I probably should. :) Any other tests you'd like me to try while I'm doing that?

rickyzhang82 commented 7 years ago

Thanks for posting MD5 hash. I can test it on mine with System 6.

Regarding to overwritten issue, I have the following concern: there are two perspecitves in addressing - host OS and guest OS.

For sure, there should be no potential conflict in host OS. Because ROM is placed above RAMSIZE and your frambuffer is fixed in the last page of RAM. See my notes here.

However, addressing in guest OS is a different story. In virtual addressing a.k.a mem bank, RamBaseMac is always 0. RamSize is a dynamic number from user preference. But ROMBaseMac is fixed in 4MB for 64k, Plus and classic ROM.

Note that for 32bit clean ROM, it is fixed at 1032MB in guest OS. The RAMSize is limited to 1024MB in GTK preference UI. There is no way to have conflict.

To address this issue, we'd better enforce size check in RamSize when emulating 24 bit mode.

PS: One more thing, can you tell me your host OS? Recently I found that there is a strange yellowish screen when using SDL with virtual addressing under Linux. But it looks normal in Mac OS X. This is not related to your 24bit addressing. But I just want to know more about frame buffer and drawing.

simon-frankau commented 7 years ago

Ah, I see. I believe RAM size being limited to 4MB due to ROM location has always been an issue for the 68000-based Macs, that's only now being highlighted by this PR, so I guess it makes sense to fix it in a separate commit before adding this one?

Another thing that's probably worth bearing in mind is that only certain RAM sizes were supported by the hardware - e.g. for the Classic 1MB, 2.5MB, 4MB (I think), due to the SIMM configuration possibilities. Other memory sizes may not emulate nicely, so that may be worth constraining.

My host OS is MacOS 10.11.4, and the display looks normal.

rickyzhang82 commented 7 years ago

I have a physical Mac SE with 68030 accelerator card and 4MB on card at home. SE has 24 bit ROM. In theory, it can address up to 16MB. But SE ROM only supports up to 4MB even 68030 supports 32 bit addressing.

However, after installing compact virtual, I can create up to 16MB RAM disk from my SCSI2SD hard drive, which is still faster than physical memory. I'd believe that compact virtual may patch SE ROM and take care of original memory map. TBH, I don't know the technical details. Above is my educated guess.

I'm not the maintainer. But I know he has a high standard to accept PR. I'd recommend you that enforce memory size limit check in 24 bit mode based on ROM type. Throw an error and stop when RamSize reach limit.

Here is a list of memory size restriction per ROM. For Plus, SE and Classic, 4MB is the upper limit.

rickyzhang82 commented 7 years ago

Also, you may need to restrict screen resolution as well. HighRes even 1080p will overflow 64k frame buffer.

In any case, I tried to test MacSE ROM (MD5: 9fb38bdcc0d53d9d380897ee53dc1322) and System 6.0.8 with your PR under Mac OS X 10.12. It crashed.

Can you share yours so that I can replicate 24bit modes?

Build

make clean
NO_CONFIGURE=1 ./autogen.sh
./configure --enable-sdl-video --enable-sdl-audio --disable-vosf --disable-jit-compiler --with-x --with-gtk --with-mon
make -j 32

Here is my pref

displaycolordepth 8
disk /Users/Ricky/repo/68K.Mac/BasilikII/DiskImage/MacImage.dsk
extfs /Users/Ricky
screen win/512/384
seriala
serialb
udptunnel false
udpport 6066
rom /Users/Ricky/repo/68K.Mac/BasilikII/ROM/Mac.ROM
bootdrive 0
bootdriver 0
ramsize 4194304
frameskip 0
modelid 5
cpu 4
fpu true
nocdrom false
nosound false
noclipconversion false
nogui false
jit false
jitfpu true
jitdebug false
jitcachesize 8192
jitlazyflush true
jitinline true
keyboardtype 5
keycodes true
keycodefile /Users/Ricky/repo/68K.Mac/BasilikII/BasiliskII_keycodes
mousewheelmode 1
mousewheellines 3
dsp /dev/dsp
mixer /dev/mixer
ignoresegv false
idlewait true
simon-frankau commented 7 years ago

I can see it being worth putting in extra validation to make sure the configuration is sane, but I think it's worth trying to have you being able to reproduce a successfully-running 24-bit Mac first!

I have run my patch successfully with the 9fb38bdcc0d53d9d380897ee53dc1322 SE ROM you mention, in conjunction with a Finder 6.1.8/System 6.0.8 "System tools.dsk" boot floppy image (MD5 de65a7180408b86c0ca31d35982b1f85) from "System608.zip". I don't have a System 6 hard disk image to use. I used your pref file with the disk/rom/floppy lines changed. It worked.

To build, I ran:

make clean
NO_CONFIGURE=1 ./autogen.sh
./configure --enable-addressing=banks --with-gtk=no --with-esd=no
make

Can you give me any more details of how it crashed, please?

rickyzhang82 commented 7 years ago

It seems that it is SDL issue. See

Output

./BasiliskII
Basilisk II V1.0 by Christian Bauer et al.
Reading ROM file...
Using SDL/coreaudio audio output
Abort trap: 6

Crash report

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib          0x00007fff9cb0ed42 __pthread_kill + 10
1   libsystem_pthread.dylib         0x00007fff9cbfc457 pthread_kill + 90
2   libsystem_c.dylib               0x00007fff9ca74420 abort + 129
3   BasiliskII                      0x000000010bac2ee2 driver_base::init() + 354
4   BasiliskII                      0x000000010bac3240 SDL_monitor_desc::video_open() + 96 (video_sdl.cpp:925)
5   BasiliskII                      0x000000010bac3e8f VideoInit(bool) + 2815 (video_sdl.cpp:1150)
6   BasiliskII                      0x000000010ba9c209 InitAll(char const*) + 553 (main.cpp:173)
7   BasiliskII                      0x000000010baea016 SDL_main + 1814 (main_unix.cpp:678)
8   BasiliskII                      0x000000010bbc4151 -[SDLMain applicationDidFinishLaunching:] + 52
9   com.apple.CoreFoundation        0x00007fff8727645c __CFNOTIFICATIONCENTER_IS_CALLING_OUT_TO_AN_OBSERVER__ + 12
10  com.apple.CoreFoundation        0x00007fff8727635b _CFXRegistrationPost + 427
11  com.apple.CoreFoundation        0x00007fff872760c2 ___CFXNotificationPost_block_invoke + 50
12  com.apple.CoreFoundation        0x00007fff87233523 -[_CFXNotificationRegistrar find:object:observer:enumerator:] + 1827
13  com.apple.CoreFoundation        0x00007fff8723255c _CFXNotificationPost + 604
14  com.apple.Foundation            0x00007fff88c57677 -[NSNotificationCenter postNotificationName:object:userInfo:] + 66
15  com.apple.AppKit                0x00007fff84e9b74f -[NSApplication _postDidFinishNotification] + 297
16  com.apple.AppKit                0x00007fff84e9b4b4 -[NSApplication _sendFinishLaunchingNotification] + 208
17  com.apple.AppKit                0x00007fff84d5e819 -[NSApplication(NSAppleEventHandling) _handleAEOpenEvent:] + 552
18  com.apple.AppKit                0x00007fff84d5e46b -[NSApplication(NSAppleEventHandling) _handleCoreEvent:withReplyEvent:] + 661
19  com.apple.Foundation            0x00007fff88ca2afd -[NSAppleEventManager dispatchRawAppleEvent:withRawReply:handlerRefCon:] + 290
20  com.apple.Foundation            0x00007fff88ca2977 _NSAppleEventManagerGenericHandler + 102
21  com.apple.AE                    0x00007fff880a6f26 aeDispatchAppleEvent(AEDesc const*, AEDesc*, unsigned int, unsigned char*) + 544
22  com.apple.AE                    0x00007fff880a6c9d dispatchEventAndSendReply(AEDesc const*, AEDesc*) + 39
23  com.apple.AE                    0x00007fff880a6ba9 aeProcessAppleEvent + 312
24  com.apple.HIToolbox             0x00007fff867d1ddf AEProcessAppleEvent + 55
25  com.apple.AppKit                0x00007fff84d59d1d _DPSNextEvent + 1833
26  com.apple.AppKit                0x00007fff854d57ee -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 2796
27  com.apple.AppKit                0x00007fff84d4e3db -[NSApplication run] + 926
28  BasiliskII                      0x000000010bbc4736 main + 1241
29  libdyld.dylib                   0x00007fff9c9e0235 start + 1

On the other hand, if I use your configure options. I ends up in illegal instructions loop:

Illegal instruction: 4e7b at 00400402
^CD0: 00001bfe D1: 80000000 D2: 80000000 D3: ffffff00
D4: 6db6db6d D5: 00400000 D6: 00000000 D7: 00000000
A0: 0040136c A1: 80000000 A2: 00401384 A3: 00402694
A4: 00000000 A5: 00402620 A6: 004000b0 A7: 0003b164
USP=00000000 ISP=0003b164 MSP=00000000 VBR=00000000
T=00 S=1 M=0 X=1 N=0 Z=0 V=0 C=0 IMASK=7
FP0: nan FP1: nan FP2: nan FP3: nan
FP4: nan FP5: nan FP6: nan FP7: nan
N=0 Z=0 I=0 NAN=0
8000009c: 0040 1380 0040 1380 0040 OR.W #$1380,D0
next PC: 800000a0

So I wonder where does your System608.zip comes from?

simon-frankau commented 7 years ago

My build hits the illegal instruction, carries on, and boots successfully:

$ ./BasiliskII --rom ROM.se --rominfo --floppy System\ Tools.dsk 
Basilisk II V1.0 by Christian Bauer et al.
Reading ROM file...

ROM Info:
Checksum    : b2e362a8
Version     : 0276
Sub Version : 0000
Resource Map: 0001af1c
Trap Tables : 0001aa9c

Illegal instruction: 4e7b at 00400402
WARNING: No FSM present, disabling ExtFS

4e7b looks like "MOVE2C", according to uae_cpu/table68k. That same file suggests it's a privileged 68010 instruction. My random guess is that perhaps it's using an instruction not present on the 68000 to work out what CPU it has? From a 2-minute glance at the assembly, this looks plausible to me.

In any case, looking at an execution trace, my Basilisk II just carries on executing the following instructions and continues to boot successfully.

What exactly happens on your copy after hitting the illegal instruction?

I got the zip from https://sites.google.com/site/minivmacapplicationsv6/systems-os.

rickyzhang82 commented 7 years ago

What exactly happens on your copy after hitting the illegal instruction? It is an infinite loop of illegal instruction 4e7b.

Note that address 0x00400402 is above RAM SIZE, which is address of ROM. It seems come from Mac SE ROM. It throw illegal instruction exception. It may execute this instruction from user privilege.

See below I run into cxmon:

Ricky@imac:~/repo/68K.Mac/BasilikII/ROM$ cxmon
[0000000000000000]-> [ 0 "MacSE.ROM"
00040000 bytes read from 0000000000000000 to 000000000003ffff
[0000000000040000]-> m 402
0000000000000402: 4e7b0002 600c7201 0c6f0010 00066702  'N{..`.r..o....g.'
[0000000000000502]-> d68 402
0000000000000402: 4e7b 0002             movec   d0,cacr
0000000000000406: 600c                  bra.s   $00000414

I can see two major differences:

  1. I enabled SDL in my configure. If I use yours, SDL is not used.
  2. I used emulated hard drive, while you used emulated floppy disk.

It may be a deal breaker if there is no SDL audio.

simon-frankau commented 7 years ago

The more I look, the more I am confident that the 4e7b is a red herring, and expected during boot-up. The disassembly looks like this:

; Save illegal instruction vector.                                              
0x004003ee      43f80010       lea.l 0x10.w, a1                                 
0x004003f2      2f11           move.l (a1), -(a7)                               
; Overwrite illegal instruction vector with 0x408                               
0x004003f4      41fa0012       lea.l 0x12(pc), a0                               
0x004003f8      2288           move.l a0, (a1)                                  
; Get ready...                                                                  
0x004003fa      204f           movea.l a7, a0                                   
0x004003fc      4267           clr.w -(a7)                                      
0x004003fe      7202           moveq 0x2, d1                                    
0x00400400      7001           moveq 0x1, d0                                    
; Do the illegal instruction                                                    
0x00400402      4e7b           invalid                                          
; The following bytes form part of the legal 4e7bXXXX instruction on 68020+, and then it          
; jumps out the way - 0x00400406 is bra.b 0x00400414                                 
0x00400404      0002600c       ori.b 0xc, d2                                    
; illegal instruction vector.                                                   
0x00400408      7201           moveq 0x1, d1

The code is clearly setting up a temporary illegal instruction vector, in order to test an instruction that is illegal on (according to the UAE emulation) 680[01]0, and ok on 68020 and beyond. It's some kind of CPU generation check.

If you want to rule it out as the source of your problem, you could just put a "return 0" at the start of "movec_illg" in newcpu.cpp, to force the instruction to not be illegal. This is incredibly hacky, but doing this allowed me to have a booting system that didn't display any illegal instruction warnings.

Failure to boot from HD sounds quite plausible. I have only tested booting from floppy, not HD, so if there are any latent HD bugs in the 24-bit mode, I wouldn't have stumbled across them. Can you try booting from a floppy, please? If that works for you, it looks like we'll have to work on fixing HDs!

rickyzhang82 commented 7 years ago

This is Mac SE ROM which uses 68000 CPU. According to 68K programmer reference manual, 68000 CPU didn't implement MOVEC instruction. I'd believe physical 68000 CPU enter into illegal instruction exception process.

I found someone from mini vMac project annotate this part assembly and state the it check CPU:

                      P_mWhichCPU:
3EE    43F8 0010                 Lea.L     (IllegalInstructionVector), A1
3F2    2F11                      Move.L    (A1), -(A7)
3F4    41FA 0012                 Lea.L     E_2, A0
3F8    2288                      Move.L    A0, (A1)
3FA    204F                      Move.L    A7, A0
3FC    4267                      Clr       -(A7)
3FE    7202                      MoveQ.L   $2, D1
400    7001                      MoveQ.L   $1, D0
402    4E7B 0002                 MOVEC     $2
406    600C                      Bra       L13

                      E_2:
408    7201                      MoveQ.L   $1, D1
40A    0C6F 0010 0006            Cmp       $10, $6(A7)
410    6702                      BEQ       L13
412    7200                      MoveQ.L   $0, D1
414    2E48           L13:       Move.L    A0, A7
416    229F                      Move.L    (A7)+, (A1)
418    1E01                      Move.B    D1, D7
41A    4E75                      Rts

Yes, your patch works on floppy! We are very close to make BII runs like mini vMac. I doubt if BII rom patch has done anything special to SCSI hard drive. I will dig it deeper later. In any case, your frame buffer patch work.

IMHO, it is no realistic to patch any ROM. If you are willing to do any further work together, let's pick a base line 24bit ROM to patch it for BII.

simon-frankau commented 7 years ago

Woo! Progress! :)

What now? I don't know what the normal approach in the project is, but my preference is to commit small, incremental improvements, rather than create a huge, huge PR. My suggestion would therefore be to start by getting the smallest change in to get a 68K Mac booting from a floppy (including the memory size check you mentioned, and perhaps a warning if you attempt to boot from an HD that "Hard disk support in 24-bit mode is untested").

After that, we can work on 24bit support for HDs with a specific ROM and System, and build out from there. I'm confident that we could support a good number of ROMs and Systems, given enough time, but we should start small and build up.

How does this sound to you?

Cheers, Simon.

rickyzhang82 commented 7 years ago

I totally agree with you: no huge PR. An error message "Hard disk support in 24-bit mode is untested" should be good enough for now.

One more thing you may consider: add check on maximum physical memory size and maximum screen resolution. It is better cap them up to 4MB and fix the resolution to 512x342 BW. See list

I know that David is working on adding SDL2 which supports scale up/down. It is not bad to a fixed low res once we have scaling from SDL2.

rickyzhang82 commented 7 years ago

I improved break point in ROM. This should help us a little bit easier to poke around the 24bit ROM during start up. See my PR #133

rickyzhang82 commented 7 years ago

The illegal instruction 4E7B is to test 68000 CPU illegal exception handling. In address 3F4, it overwrites illegal instruction vector 0x10 = 4 * 4 with new address in 408. By the time it returns, it restore original illegal instruction exception handler and set D1 as 1 if this is 68000 CPU.

So the intention is to check CPU type.

See my comment below:

                      P_mWhichCPU:
; assign $10 -> A1
3EE    43F8 0010                 Lea.L     (IllegalInstructionVector), A1
; backup original illegal instruction handler to stack
3F2    2F11                      Move.L    (A1), -(A7) 
; override illegal instruction handler with new address E_2
3F4    41FA 0012                 Lea.L     E_2, A0
3F8    2288                      Move.L    A0, (A1)
3FA    204F                      Move.L    A7, A0
3FC    4267                      Clr       -(A7)
; Setup flag
3FE    7202                      MoveQ.L   $2, D1
400    7001                      MoveQ.L   $1, D0
402    4E7B 0002                 MOVEC     $2    
406    600C                      Bra       L13

                      E_2:
408    7201                      MoveQ.L   $1, D1
40A    0C6F 0010 0006            Cmp       $10, $6(A7)
410    6702                      BEQ       L13
412    7200                      MoveQ.L   $0, D1
; restore original illegal instruction handler
414    2E48           L13:       Move.L    A0, A7
416    229F                      Move.L    (A7)+, (A1)
418    1E01                      Move.B    D1, D7
; return back to ROM address 5C
41A    4E75                      Rts
simon-frankau commented 6 years ago

BTW, sorry for no updates in the last few days - I've been travelling on business and haven't had access to my dev machine. I should start again in a couple of days. Cheers.

rickyzhang82 commented 6 years ago

No problem. I'm looking into why disk driver doesn't work in 24bit ROM. To be more precise, it is Mac SE ROM. That ROM has been annotated by mini vMac.

If you are interested, let's discuss it in a separate thread #134

Regarding to the illegal instruction MOVEC, there is something weird in emulation. Sometimes it has an infinite illegal instruction loop, while Sometimes just boot in with just one illegal instruction message.

rickyzhang82 commented 6 years ago

I also fixed SDL for one bit per pixel case. It turns out that black&white is broken in bank addressing mode.

Give a try in my experimental branch https://github.com/rickyzhang82/macemu/tree/exp-24bit-sdl

Run Basilisk/src/Unix/build.sh to build. Of course, you need to install SDL v1 library.

rickyzhang82 commented 6 years ago

@asvitkine What's your concern to accept this PR? I have several additional patch that relate to this 24bit ROM fix.