NES MMC3 Config Breaks Pointers and Stack (?)

[seanwiththebeard]

TL/DR: I almost figured out why the bankswitching demo doesn't work on other emulators or real hardware even though things look fine in the IDE

Something between the nesbanked.cfg linker config and the custom neslib implementation is messing up any use of pointers when working in a mapper-4 MMC3 configuration. I've set up three repos to show the problem and a partial solution. I suggest using Mesen to reproduce the issues

The Problem Illustrated - If you try the bankswitching demo from the NES examples, it works fine in the IDE's JSNES emulator but you get a completely blank screen in Mesen or on an Everdrive. FCEUX seems to work fine, but since Mesen and my real hardware exhibit the same behavior in all the relevant cases I'm going to refer to those as "outside the IDE" for the purposes of this issue.

I'm not sure why these behave differently and I was hitting my head against the wall for a few days, but today I looked at some debugging viewers in Mesen and noticed that code is definitely running. Then I looked at the nametable data and noticed there was a line of character 0x7F where it should have said "Bank 0 @ 8000." So code is definitely running, PPU writes are happening, and the character set is being copied, but the string pointer isn't being passed right. The next three lines don't draw at all because they're passed as parameters to the draw_text function, which I assume should be put on the stack. Are we getting a stack? I looked at the nesbanked.cfg config (which I assume is used when you #define NES_MAPPER 4) and there isn't a stack location defined. Same issues with the palette set function, so the screen remains black. I've also wondered if it's a header issue, since the 8kb PRG-RAM doesn't seem to be enabled in the iNES header. Another shot in the dark is the condes table which ends up in the 0x6000 block where that RAM would be.

The problem circumvented (sort of): https://github.com/seanwiththebeard/NES_MMC3_Example

So after a lot of experimentation, I used a different method for setting the palette and setting text strings and made a little demo. This is where I found that strlen doesn't work either. This demo shows two ways of drawing strings, one direct way works in both the IDE and on real hardware, but passing a parameter to a function only works in the IDE. Also notice the local variable x used in the while loop at the end used to set the background color, outside the IDE it only increments before the loop. I tried compiling locally with CC65 (your neslib version's source copied to the source folder) using and surprisingly got identical output. Here is where I'm starting to think it's a linker configuration issue, and pointers/stack things don't get handled to the right places.

The (almost) solution: https://github.com/seanwiththebeard/ConfigTest_MMC3_NESDougConfig

This example from NESDoug is a sample project that compiles locally, one click compile and you get an MMC3 bankswitched demo. I took this template and shoehorned my demo into it, adapting for some different memory segment layout from it's included linker config. Using the included neslib it compiled fine and worked on everything. Once I tried your custom neslib, something with the startup code wasn't working. But, since the template worked for sure, I tried uploading the whole project into 8bitworkshop to see if I can override the custom neslib. It's a hack because you need to link crt0.s and then comment it out to get it to run, but then you can save the ROM and it works on everything. Inside 8bitworkshop there's a weird graphical glitch that doesn't happen in the exported ROM. It crashes after a few seconds and I'm not sure why, but the exported ROM is stable as expected, music even works from whatever bank it's configured to. What if anything is different about the linker config (aside from the memory segments) isn't apparent to me but I haven't been able to find where the condes table goes to in this one... At any rate, things like pointers and global variables work fine.

So here's what I think might be ideal: https://github.com/seanwiththebeard/33_MMC3_8bitworkshop This is NESDoug's demo shoehorned back into the last project, exact same behavior. Same issue where you have to uncomment the crt0.s and recomment it. So if this proves that it's possible to get a fully compatible MMC3 ROM exported with the right neslib, it would be nice to have an option in 8bitworkshop to use the regular neslib instead of the custom one. Or, ideally, it would be great if the custom neslib (or the startup code?) was fixed for this use scenario.

Sorry if this is a longwinded for a small problem mostly related to real hardware and certain emulators, but I had an interesting time getting here. Technically now I have a template for a larger MMC3 project, but hopefully a more elegant fix can be implemented.

[sehugg]

Wow! Thanks for chasing this down the rabbit hole. I am often regretful that I hacked up the custom NESLib, it's caused several problems. The IDE also uses an older build of the CC65 toolchain and passes some flags by default (like local -> static variables) so it may not match the command line behavior.

SP = $8000 on startup, so I wonder if the JSNES emulator has secret RAM that it keeps there regardless of the mapper setting?

[seanwiththebeard]

A rabbit hole for sure! I kept getting the feeling this came down to one wrong value somewhere. I should note that in regard to mapper setting, the JSNES emulator seems to want your #defines in order to start if using a specified config, having them in the cfg file as symbols isn't enough to put the emulator into mapper mode. To actually use a specific config file the declaration has to be after the mapper and prg_banks definitions otherwise it seems to be using nesbanked.cfg from internally.

I checked the memory viewer and in 8bitworkshop the stack pointer value sp shows 0x8000 stored at 0x0024. In Mesen, the value alternates each frame between 0x8000 and 0x7FFF. I tried putting pal_col(0, 0x02) in the while(1) loop, then after manually editing the memory to 0x6000/0x6001 saw the palette pop.

So here's a partial fix for getting off the ground using the default samples and config - Put this right at the start of main(): int sp = (int)0x0024; sp[0] = 0x500; //This is where the default linker config seems to keep the stack?

Now more code actually runs as expected, but it's still not perfect. The new issue I'm running into is the VRAM buffer sample, I've tried putting the buffer array all over in usable memory but it doesn't seem to work, sometimes it passes values but not the right ones. My guess is there's more to the stack config than just moving the stack pointer.

[seanwiththebeard]

More progress - I found where the 0x8000 stack pointer assignment comes from.

The stack pointer variable sp at 0x0024 gets loaded in crt0.s, and the code looks the same in your custom neslib as the original

Line 184 of crt0.s

    lda #<(__RAM_START__+__RAM_SIZE__)
    sta sp
    lda #>(__RAM_START__+__RAM_SIZE__)
    sta sp+1            ; Set argument stack ptr

In the normal neslib2.cfg, the RAM segment is in the lower area of 2K onboard memory and there's no 0x6000 block since a mapper 0 game has no RAM. DATA loads to PRG and runs from RAM, BSS, and HEAP load to RAM. The RAM starts at 0x0300 and has a length of 0x0500, so the stack pointer ends up at 0x0800.

In the bankswitched.cfg, the lower area is given a segment name of SRAM, and the cartridge memory at 0x6000 is given a segment name of... just RAM. It's 8K long so the stack pointer ends up at 0x8000 in read-only cartridge space. DATA goes into the PRG6 (fixed block) and run from the cartridge memory, BSS gets loaded to the lower memory, and there's no definition for HEAP.

Using a custom nesbanked.cfg we can swap the names for SRAM and RAM to be sure that symbol RAM points to the lower area, because that's what the code in crt0.s expects. BSS loads to RAM (lower), and DATA runs from RAM (lower) as opposed to the cartridge memory (not ideal). VRAM Buffer sample works with this config. This is as far as I've gotten, it doesn't work if you run DATA from the cartridge memory so my next step is to play around with that segment in code.

[seanwiththebeard]

For the cartridge memory, you have to manually enable it with the MMC3 register. Some emulators leave it enabled by default, or at least behave that way? I'll have to read more on headers to understand if the WRAM needs to be added there but it seems for now that using the registers is enough in most cases.

#define MMC3_WRAM_DISABLE() POKE(0xA001, 0x40)
#define MMC3_WRAM_ENABLE() POKE(0xA001, 0x80)
#define MMC3_WRAM_READ_ONLY() POKE(0xA001, 0xC0)

Is enabling this register something that could be added to the crt0.s startup code for maximum compatibility without breaking mapper 0 config? If not, I'm hoping that at least the segment name swap in bankswitched.cfg makes it into the next update. Just swap the RAM and SRAM segments names and be sure DATA is set to run RAM and BSS loads RAM. The WRAM segment can be there but optional to use.

For simple things I'm getting mostly identical behavior between the IDE and Mesen now, there's still a few things to play around with and test in regard to moving the whole DATA segment to 0x6000, the snake game sample almost works but I think the libraries and includes need DATA at the low address, I got it working with everything but the condes table and joystick driver moved up. My goal is to build a template/demo for MMC3 with battery RAM, or NES-TKROM for an official board reference with an available PCB and replacement mapper chip. After some more testing and memory segment management I'll post my "big project" linker config.

[seanwiththebeard]

Proof of Concept Sample

[sehugg]

Thanks for this! I updated the nesbanked.cfg file, the two sample files seem to work, you can try it out on the dev site: https://sehugg.github.io/8bitworkshop/?platform=nes&file=bankswitch.c

I didn't mess with the crt0 because I seem to have, er, misplaced the source .. I assume I hoarked the .o file directly from NESLib. Is setting the cartridge RAM at startup just for alignment with Mesen?

[seanwiththebeard]

Awesome, as far as I can tell, that works on Mesen and real hardware now

I may need to understand a bit more about MMC3 but what I've gathered is that access to the cartridge RAM is going to be different for each case:

• No battery and no SRAM - Keep it disabled because nothing will be there - the emulator pretends that it is
• SRAM, no battery (ex: SMB3) - Full write access is fine since it clears on power-off anyway, but it has to be enabled manually on the real hardware
• SRAM and Battery (TKROM ex: Kirby) - Since the CPU can glitch on power-off and write to random addresses, you'll want SRAM set to read-only when you aren't using it, handling this correctly solves the "hold reset before power-off" issue

In most cases you'll probably be in the second scenario, so if it's not in the startup code as enabled on boot the easy solution is to add the macro to the bankswitch example and call it first thing in main(). Designing for saved games is such a different use case that I'd build a different sample around it if at all. Same with CHR-RAM.

What I don't really understand is the header information, whether SRAM needs to be enabled there or not. As far as I can tell it doesn't matter but I haven't gotten to saved data yet, I'd assume that most emulators create a file if that byte of the header indicates a battery. How many things for the header can be defined in the code like #define mapper 4 or as symbols in the cfg file?

A small thing that was throwing me off is the references to 16k prg banks in the linker config, as far as I can tell they're all 8k

This might be getting away from the original issue but the thing that's throwing me off now is when I use malloc(), the pointer goes to 0x0000 and writes over ZP data. Might need a heap segment? C64 and Apple II seem to work without one from what I've seen so maybe I'm doing something funny or just don't see it in the configs. It seems to do this in the default Hello World sample too without the mapper4 config. At any rate, this has been a fun excuse to play around with the different segments and learn what they're for

[seanwiththebeard]

Now I think I figured out the malloc thing, the heap ends up with a size of 0 because the RAM and stack segment stops where it should start. If you extend the RAM segment to start as it does at 0x0500 and end at 0x2000 (size is $1B00), it seems to work. If I malloc something it ends up at the address in heaporg ascending and incrementing heapptr. So it's sort of related, being a linker config thing. Now if there's a way to move the heap into the SRAM segment...

[seanwiththebeard]

I think I've got the heap figured out and here's a small demo to show how I moved it into SRAM. I'm sure there's a more elegant method for setting the variables from <_heap.h> but I'm not really understanding how they're exposed, I just wrote new values to a pointer to the address of each symbol name. This works fine on real hardware. The demo fills the heap and it gets faster as it goes because the amount to count keeps getting smaller. The heap goes from $7000 to $8000 so there's room for putting data in the first 4k of the SRAM. Maybe eventually I can move the stack too. Running a heap within 2k of base RAM is kind of pushing it, but 8k SRAM definitely opens up some possibilities for data-driven games.

I've played around a lot with the memory segments but can't seem to run DATA from SRAM, it breaks on real hardware and Mesen, I think the best solution is just to #pragma data-name (push, "XRAM") to manually store things there. BSS seems fine to load to SRAM if that's just uninitialized data anway (?). The stack can be up to about $500 big if you start RAM at $300. Possibly more, I haven't moved the VRAM buffer area yet (but I'm using a smaller buffer size of 84 now for NTSC, anything more has issues with big writes). I think that's the only edit to the nesbanked.cfg on this demo.

At this point I want to say everything related to real hardware compatibility is resolved with some kind of workaround, and a full-featured MMC3 game is definitely possible entirely in 8bitworkshop. I haven't gotten to the sound and music demos yet but I'm guessing there's going to be some memory locations to move there. The heap was probably unrelated to the original issue, after I've looked at the default cc65 NES config I have more questions than answers on the intended mapper target and whether the heap would be functional with that config anyway.

This adventure reminded me that the memory map in 8bitworkshop is my favorite feature because it makes this sort of thing easy to work with on-the-fly building out your segments and seeing the sizes, and I can't imagine working on a big project without it.

[seanwiththebeard]

It's been a minute but I'm still working on this in regard to real hardware vs emulators. It's late and I might still be missing something but I think I've hit the next milestone and this is a good point to document what I've got. I think the "issue" is probably solved at this point, but there's a lot about the startup code that I still don't understand well enough to say for sure.

My summer project was building a Muramasa MMC3 Famicom board to test on real real hardware. Doing that means expanding the memory config to 512k/256k PRG/CHR, splitting the output ROM into PRG and CHR files to write to EPROMS, and plugging them into DIP sockets on the board (I'm working on a neat bracket with ZIF slots). It took a little troubleshooting (removing the optional resistor fixed it) then I confirmed full operation with Mario 3, Kirby, battery save, etc before I got to my own code. My plan is to prove you can theoretically produce a new-cartridge RPG. Then things at work picked up and it's taken me a few months to get back to playing with my code again, finally I got some time today to sit down and solve a few things.

This is going to be referencing the same repository as the post above, I'm continuing to use that as my repo for an 8bitworkshop-compatible full-featured MMC3 template.

First challenge was modifying the startup code, I could get my demo to start up only sporadically after pressing reset a few times. I don't entirely understand how the startup vectors work but something doesn't point to the initialization code on boot and I think it has to do with the MMC3 chip not having banks selected. I went through your neslib's crt0.s and compared the differences with nesdoug's MMC3 example. There's some MMC3 initialization code to copy over from that example to this one in the clearRAM function, and the clearPalette function needs the value loaded into 'a' to be $00 otherwise the tiles corrupt (only in 8bitworkshop's emulator, not real hardware). Not sure how I figured that one out, I was trying to change the background just to be sure I was using the uploaded crt0.o. The files needed to produce the new working crt0.o are in the zip file of the repository, it basically copies just the MMC3 stuff from that example into your neslib to maintain compatibility with 8bitworkshop.

The other change was putting the startup code in a new segment at the very last page just before the vectors ($E000-FFFA), after noticing how nesdoug's config was set up that way. It has to be exact or it doesn't work on both the emulator and hardware. The emulator doesn't glitch the graphics if you load startup at $C000 like I had it before, but that doesn't reliably boot on hardware.

So after all of that I finally get a 512k PRG, 256k CHR ROM that works on a real reproduction board and boots every time, as far as I can tell. That's a stopping point that I'm happy with for now and I wanted to document and publish what I've got. Next milestone is probably importing the music demo and copying these changes over to the NES version of my game repo.

[seanwiththebeard]

This was the issue that came up once I had my startup code where real hardware expected it in the last bank. I think this issue has to do with the JS NES emulator permitting graphics writes when other emulators and real hardware don't, and it comes from the clearPalette function in crt0.s using a value of $0F (sets the palette to blacks). The fix for this was to set it to $00 instead. Different values make different patterns because something overwrites the bytes of tile index $00 with the value loaded into 'a' in that function.

What I don't understand is why this emulator glitch happens when the startup code is moved to $E000 (makes it work on real hardware) but not when it runs from $C000.