3E bankswitch >16K RAM bug?

[andrew-davie]

I'm working on my chess program, which switches RAM banks as it goes deeper in the search. When I get to bank 16, things go haywire.

Looking at the debugger, I see it tells me bank 0 is switched in RAM (it should be 16). When I go to the "cartridge 3E" tab and select any RAM bank up to 15, then the disassembly shows that bank. When I select "16" as the RAM bank, the disassembly shows "bank 0". In other words, it looks to me both in code debugging and visually in the debugger that 3E is somehow hardwired to 16K RAM. This should not be the case. Is this a bug, or am I misunderstanding something? Thanks.

[thrust26]

The % operation in Cartridge3EWidget::bankState() uses numRomBanks for RAM banks. Should be an easy fix.

[DirtyHairy]

Be careful, there definitely is at least one ROM out there that will break with support for >16k RAM. That‘s why I put an artificial 16k limit in both 6502.ts and on the Uno. Don‘t know the exact ROM from the top of my head, though, I‘ll have t check later.

[thrust26]

AFAIK only BD and Andrew's latest games (Sokoboo and Chess) are using 3E. Do you remember one of them being the game which breaks?

[DirtyHairy]

Sorry, ignore what I wrote, I doublechecked, the limit I referred to is 32k, and that‘s what‘s in the code.

[DirtyHairy]

I think the problematic ROM was the bad apple demo. It relies on the bank index wrapping at 32.

[andrew-davie]

Since the original 3E bankswitch scheme was implemented on the Krokodile Cartridge (KK) and designed by Armin and myself, the "formal" definition of that scheme would be the one as implemented on the KK. That is currently specified as "3E (up to 512K ROM and 32K RAM)" So, I guess to be truly correct as far as 3E goes - yes, bank index wrapping for RAM would be at 32.

[andrew-davie]

One addition to that; I recall an informal agreement to "redefine" 3E as being up to 480K to allow for the CC2 to be able to implement the scheme (I recall that the 32K RAM came out of the 512K total that the CC2 provided, and hence ROM was a maximum of 512-32. Something like that).

[thrust26]

Never heard of that agreement. And it is also not reflected in the Stella code. There the RAM is fixed at 32K.

[DirtyHairy]

I don't know the historic background, but technically, the scheme can support up to 256k RAM and 512k ROM. To my understanding, the only hard limit here comes from hardware implementations that can't support the full amount of RAM and / or ROM. When I implemented the current Uno version, I originally supported more than 32k RAM, but this failed to work with bad apple as it implicitly assumed that the RAM wraps at 32.

So, I set the the limit to 32k there, too, which is also what is encoded in Stella. On the Uno, the limit can be raised to 96k if the extension ".3EX" is used. Noone has been using that so far, though.

I didn't want to start a discussion on that, though, I just misread the issue on my mobile and thought it was about extending the limit beyond the 32k 😏 Sorry for the noise.

[andrew-davie]

Not sure if you understood what I was saying there. The RAM is fixed at 32K, but the ROM for the demos I did were limited to a maximum of 480K for CC2 compatibility. That was the informal agreement so that CC could run any 3E demos that I made back in the early part of this century.

It brings up a discussion point, really. Since 3E was implemented on Krokodile Cart, the limitations of that (512K ROM, 32K RAM) in my view effectively defined the bankswitch scheme. Yes, the "design" easily allows for 256K RAM, and 512K ROM, but is it still 3E, or is it another bankswitch scheme? It's another, in my view.

I'd argue since the "bad apple demo" in good faith relied on the definition of 3E RAM being limited to 32K, and banks wrapping around, and the KK hardware (and CC2 hardware) restrict it to 32K as well, this is in fact the definition of what the 3E bankswitch scheme actually is.

[thrust26]

I agree, 32K RAM should be the limit for 3E. But @DirtyHairy effectively named the larger scheme 3EX, so that seems fair to me.

[andrew-davie]

AFAIK only BD and Andrew's latest games (Sokoboo and Chess) are using 3E. Do you remember one of them being the game which breaks?

There were also early demos, such as Dancing Baby and I seem to recall the beginnings of a 2 player karate-type game. They're probably still "out there" somwhere. Edit: might have been 3F only

[andrew-davie]

I agree, 32K RAM should be the limit for 3E. But @DirtyHairy effectively named the larger scheme 3EX, so that seems fair to me.

So, would be nice to implement the 3EX scheme so that someone (ahem!) can start using it!

[andrew-davie]

If UNO capability defines 3EX then that's a 96K limit. But the concept of the scheme can have 256x1K banks. SO, do we keep changing schemes for each new bit of hardware, or do we just say "3E scheme can handle varying amounts of RAM, depending on your hardware/cartridge". And change Stella to 256K? My point being, I don't really want a half-dozen different "3E-like" bankswitching schemes and you have to figure which one your binary is using/expecting. If Stella just allowed 256K, and we either special-cased taht binary that wraps at 32K, or requested a rewrite/fix? Also, should be be allowing cartridge limitations to drive stella's implementation of a bankswitching scheme. Particularly when different cartridges implement "that" scheme differently? Personally I wouldn't mind if KK compatibility was dropped. Also, CC2. Just throwing these comments up to see what others think.

[thrust26]

It would be very easy to change Stella that way, the problem are the ROMs which rely on RAM wrapping at 32K. These would break.

I suggest that we add one more scheme (e.g. 3EX) which supports 256K RAM. And each ROM using less RAM must not rely on certain RAM limits.

[DirtyHairy]

👍 Yes, imo that's the way to go: 3EX support 256k of RAM max, and it is up to hardware implementations to support less RAM --- they just won't be able to run particular games that require more.

I wonder: could we abuse the NMI vector to store a magic byte together with the number of RAM required? Then we could autodetect 3EX, and hardware could validate that it is capable to run it.

[andrew-davie]

I thought @DirtyHairy was saying 3EX was limited to 96K (UNO cart). In any case, I agree/think that a new scheme, whatever it is called, should have the full 256k RAM and 512K ROM capability. In other words, no development cart/hardware limitations.

[DirtyHairy]

thought @DirtyHairy was saying 3EX was limited to 96K (UNO cart)

Correct. 512k ROM and 96k RAM on the UNO.

[sa666666]

I agree with the last two comments:

• I would want only one implementation of 3EX, that accommodates the max amount of ROM/RAM; I don't want to be doing new releases for every little tweak that the scheme can do. So we need to properly design it now, and have it done.
• Putting a 'magic byte/string' somewhere in the code would be nice too. That's how many other schemes are autodetected, and it also allows the Harmony/UNO/other emulators to properly deal with variations.

I guess the real question is, who has the authority to define a new scheme? I mean we (Stella team) can obviously do it, but how do we spread that to other carts/emulators?

[thrust26]

IMO the definition has to come from us. Hardware designers will usually tailor the schemes to their needs and limitations.

[sa666666]

Also, as we're commenting a closed issue, perhaps a new issue should be created for this?

[andrew-davie]

AFAIK Thomas and I and the bad apple author are the only ones to use 3E? Then there are the hardware implementors for CC2, KK, and UNO. But, any "definitive" "3EX" will not be compatible, for all binaries, with the above hardware. So,conceptually, the definition has to come from programmers in my view. And given that Thomas and I are the most familiar with it, we get top billing :P

[thrust26]

@sa666666 A new issue makes perfect sense.

[andrew-davie]

I have one other comment - one of the major major issues with 3E was the inability to do indirect Y addressing "lda (addr),y" across page boundaries. I would dearly love to fix that. I never really understood the reasons why, but I thought it was a hardware/space issue in the original KK. Can we please NOT have that limitation, if at all possible, in any "new" implementation? This would, of course, make "3EX" incompatible with "3E" but IMHO that's not an issue anyway because we already have issues with 3E RAM bank wrapping at 32K

[thrust26]

I don't think we have implemented this with 3E in Stella.

[andrew-davie]

I don't think we have implemented this with 3E in Stella.

Mmh. Well, that means that ROMS will behave differently on KK and Stella!

[thrust26]

I only very vaguely remember this. Do you have some notes or any communication with Kroko where this was discussed?

Nevermind, found a comment in BD source code:

; Note: This relies on DrawTheScreen starting on page boundary so that the (),y addressing will not violate the page-crossing restriction of 3E.

[andrew-davie]

I only very vaguely remember this. Do you have some notes or any communication with Kroko where this was discussed?

We could always ask Armin. But yes, the issue is that KK did not have enough free memory to fully implement that addressing mode, so the high byte is not incremented during address calculation.

Edit: This makes BD all the more incredible ;)

[DirtyHairy]

I'm pretty sure that this restriction has not found its way into any emulator.

[andrew-davie]

You either "invalidate" the Krokodile Cart (KK) and say it doesn't support 3E, OR you stick to the original implementation on KK with the 32K RAM restriction AND the "lda (addr),y" restriction. Purity says to me that "true" 3E is the latter case, and KK supports it. Anything written in 3E that assumes "lda (addr),y" behaves normally (i.e., crosses pages) is actually incorrect - and any emulator that runs it is also incorrect. Any new bankswitch scheme should not have that restriction. And, of course, that scheme would not be 3E. Boulder Dash, Sokoboo, Chess are (at the moment) "true" 3E. IMHO.

The other course of action, as noted, is to just abandon the KK 3E "compatibility" and redefine 3E as 32K RAM with no restriction on "lda (addr),y" addressing.

I'm pretty sure there are practically zero KK in use right now.

[thrust26]

I am also quite sure that the flash carts also do not implement that restriction. And the hardware which was used for the Boulder Dash release cards too.

So by that definition KK supports something like 3E-. :smile:

BTW: New issue created (#619).

[andrew-davie]

Furthermore, to complicate things... that ROM that does the bank wrapping at 32K... IF that ROM didn't get the "lda (addr),y" restriction right (and it probably didn't), then that's yet another version. That's "3Ex(wrapping)".

[andrew-davie]

I am also quite sure that the flash carts also do not implement that restriction. And the hardware which was used for the Boulder Dash release cards too.

So by that definition KK supports something like 3E-. 😄

BTW: New issue created (#619).

This is because well written 3E code that does not cross pages (as per spec) will also run perfectly on poorly written emulator bankswitch code that doesn't implement that spec properly :)

[thrust26]

Isn't that 3E as implemented in Stella?

[thrust26]

I am also quite sure that the flash carts also do not implement that restriction. And the hardware which was used for the Boulder Dash release cards too. So by that definition KK supports something like 3E-. 😄 BTW: New issue created (#619).

This is because well written 3E code that does not cross pages (as per spec) will also run perfectly on poorly written emulator bankswitch code that doesn't implement that spec properly :)

Did we use z26 back then? I vaguely remember a special version just for us. Else, how were we detecting the problem? Only by hardware tests?

Update: Found it! z263E.zip

[andrew-davie]

I am also quite sure that the flash carts also do not implement that restriction. And the hardware which was used for the Boulder Dash release cards too. So by that definition KK supports something like 3E-. 😄 BTW: New issue created (#619).

This is because well written 3E code that does not cross pages (as per spec) will also run perfectly on poorly written emulator bankswitch code that doesn't implement that spec properly :)

Did we use z26 back then? How were we detecting the problem? Only by hardware tests?

I started programming 3E and particularly the early BD systems before we had any emulator support. So it was purely run on KK/hardware in those early days. By the time we got emulator support, I suspect that it became ingrained never to cross page boundaries - and a lot of the systems were designed exactly that way. It's possible, I bet, that BD might in fact crash if the restriction was enforced in the emulator because it's been so long since KK was used to test. On the other hand, I seem to recall posts of people running BD demos on KK. So, I don't know. I suspect BD is just very careful about adhering to the restriction. I know I was very very very careful about it.

[andrew-davie]

So, the question -- does Z26 implement 3E with the restriction, or not?

[DirtyHairy]

At least I haven't found it by skimming the code.

[andrew-davie]

One could modify the 3E scheme on a test version of Stella to implement the restriction, and see how BD runs :). I say it's 50/50.

[thrust26]

I wouldn't even know how to implement that. The cart classes do not know about the opcodes.

[sa666666]

There is precedent for it. CartDPC+ and I believe CartCDF react to an immediate mode LDA, so they're effectively watching the opcodes.

[thrust26]

Whoever is interested, feel free to code as you like. :smile:

[DirtyHairy]

I don't see any real value in modifying 3E in Stella to fully emulate the KK implementation of 3E, but I am really curious what the exact nature of that glitch is. My best guess is be that the KK lacks the necessary resources to properly deal with consecutive cycles of cartridge RAM access, so it silently assumes that any such sequence will access consecutive addresses. If a page N is crossed to page N+1 during an indirect indexed access, the 6507 will do a ghost read from the first byte of page N (while performing the carry) before actually reading from the correct address, and that would mess up the counting. However, that would also break code running from cartridge RAM. @andrew-davie do you remember any details?

[sa666666]

I suggest to leave 3E alone as-is, unless there is some bug to be fixed. Otherwise, the scheme has been around for a long time, and this was never an issue before, so maybe lets not make it one. Any new and improved features can go in 3EX.