New chapter: Memory

[dansanderson]

This is a tutorial-style introduction to MEGA65 memory topics with a focus on programming in "MEGA65 mode." It is not meant to comprehensively describe every feature of the memory-related hardware, and defers to appendices for rigor.

PDF of the draft chapter for review convenience: DRAFT-memory.pdf

I'm interested in review from novices for clarity, and from experts for accuracy. Please point out typos, code sample errors, and typesetting issues. Feel free to leave feedback as comments on the PR if you don't want to go hunting through the memory.tex file for lines of source, though line comments are also fine. If you'd rather provide feedback privately, you can DM me on Discord (dddaaannn#7325).

Portions of this chapter serve as rewrites for narrative material in the appendices. I may consider revising the appendices accordingly at a later time. For now I intend to leave the appendices as is, in case having two takes on the material is useful.

I'm less interested in style feedback for now, except where it impacts clarity. This chapter makes some intentional style decisions that may be inconsistent with the rest of the chapters, but the books themselves are highly inconsistent in these ways for now. For example, this chapter proposes a mental model around 16-bit address translation, and revises terminology in accordance with that model. It is intentional about when to use $FFFF vs. FFF.FFFF notation. Once we determine whether that mental model is useful, we can revise text accordingly in a subsequent PR.

[dansanderson]

Thanks @rscottfree for the thorough review!

[dansanderson]

Updated draft PDF of the chapter: DRAFT-memory-20220131.pdf

[ki-bo]

Some remarks (up to page 12-15 in the version 20220131):

12-8: Typo "The MEGA65 has an adderss space of 28 bits"

12-9: "the bank number is the leftmost digits of the address,..." This is only correct for addresses with 5 digits. As there are longer addresses possible, I'd rather point to the 5th digit than the left-most one.

12-10: "For example, to store the 28-bit address $003FD08..." This can be a perfecty fine address in the BANK/MSB/LSB notation, as well (using only three bytes with upper 4 bits zero). I'd prefer to use a 28-bit address in this example that can't be represented with the 3-byte format (which is used by DMA for example)

12-10 "BASIC 65 includes commands for reading and writing" Seems to be an unfinished paragraph?

12-11 "I/O registers and specialty RAM, in range FF7.E000 – FFF.FFFF" The range F00.0000 - FF7.DFFF is a reserved one for future I/O expansion. I would already include it into the I/O registers space. Then the easy rule applies: "Any address Fxx.xxxx is an I/O or device related address"

12-11 "Attic RAM cannot be used directly by the VIC chip for graphics data, nor can it be used directly for audio sample playback (”audio DMA”)." I am not sure whether audio DMA is really not possible from attic ram. Maybe double-check with audio dma experts on this.

12-12 "3.C800-3CFFF C65 Kernel" The manual states this region is RESERVED. Where is it used and how is it banked in when made available? Btw: You use the term Kernel here, although in Commodore history, the term Kernal was typically used. The MEGA65 Book also uses the term Kernal, so you may want to keep that consistent.

12-14 "The jump table is not yet described in this book. As of this writing, the MEGA65 jump table is identical to that of the Commodore 128." (see footnote) I'd add that the C65 did not change that table so it also applies there. This is to avoid the question why the MEGA65 adopted the C128 table instead of the C65 one.

[dansanderson]

All excellent notes, incorporated, with thanks!

12-9: "the bank number is the leftmost digits of the address,..." This is only correct for addresses with 5 digits. As there are longer addresses possible, I'd rather point to the 5th digit than the left-most one.

I agree that "bank is only nibble 5 of a 28-bit address" is reinforced by how 28-bit addresses are sometimes stored in the C65-compatible areas of the architecture (with the MEGA65 extension being a separate "megabyte byte"). This is less applicable in 32-bit addresses, but the "bank" concept isn't really useful in those cases anyway. I have added some text in the "how addresses are stored" section to this effect, but have otherwise retained "bank is all upper nibbles of the address." I'm not too picky and could be further convinced that "bank" must refer to a 64KB region of a megabyte, but I think this is sufficient.

12-10 "BASIC 65 includes commands for reading and writing" Seems to be an unfinished paragraph?

Finished. This was meant to introduce WPOKE and WPEEK.

12-11 "Attic RAM cannot be used directly by the VIC chip for graphics data, nor can it be used directly for audio sample playback (”audio DMA”)." I am not sure whether audio DMA is really not possible from attic ram. Maybe double-check with audio dma experts on this.

Confirmed with M3wP, who is doing extensive work with this feature.

12-12 "3.C800-3CFFF C65 Kernel" The manual states this region is RESERVED. Where is it used and how is it banked in when made available?

Good note, I have updated this to match the "C65 compatibility ROM layout" that's already documented. The details are unimportant in this context. I only wanted the ROM layout to be mentioned here to suggest that some ROM regions are unused in certain modes, and it's safe for a program to reclaim unused ROM as RAM.

Internal detail: 2.C800-2.CFFF and 3.C800-3.CFFF are currently identical, apparently as a way of sharing "interface" code between C65 and C64 modes. They contain start-up code, banking routines, and code for switching between modes. 2.C000-2.C7FF and 3.C000-3.C7FF are not identical: the lower region contains DOS, and the upper region is reserved. My draft was calling out 3.C800-3.CFFF as C65-specific, but I don't think that's useful in retrospect.

Btw: You use the term Kernel here, although in Commodore history, the term Kernal was typically used. The MEGA65 Book also uses the term Kernal, so you may want to keep that consistent.

Hot topic! :) You are correct that the rest of the book uses either "kernal" or "KERNAL." I'm strongly considering revising the style guide for this book-wide to use the standard spelling for the general concept. It's charming that Commodore misspelled it, but I would rather upgrade to the standard spelling than treat "kernal" as an alternate spelling of the regular noun. As I think you're suggesting, I'd want to at least use "Kernal" or "KERNAL" as a proper noun (and never "kernal") in all cases. I will follow up on this with SteerCo and the community along with a few other style guide revisions I want to consider. (Maybe we keep "Kernal" as a matter of historical preservation. :) )

[ki-bo]

Internal detail: 2.C800-2.CFFF and 3.C800-3.CFFF are currently identical, apparently as a way of sharing "interface" code between C65 and C64 modes. They contain start-up code, banking routines, and code for switching between modes. 2.C000-2.C7FF and 3.C000-3.C7FF are not identical: the lower region contains DOS, and the upper region is reserved. My draft was calling out 3.C800-3.CFFF as C65-specific, but I don't think that's useful in retrospect.

This is interesting. As far as I know, the $C000 area was always mapped via $D030. This is a hard-wired mapping from $2C000 to $C000. In the original Commodore ROM, the area $3C000-$3CFFF was really reserved and "empty" (all $FF). Do you have any reference that explains why that mirror was done with the MEGA65, as it was not the case originally?

Another interesting fact: Commodore used the area $C000-$C7FF for DOS ROM, since the original DOS seemed to take more space than available at $8000-$BFFF. By restricting the interface area to $C800-$CFFF, they had additional 2KB for the DOS ROM to use. It was always mapped via $D030, and only if DOS was requested, the remaining parts at $8000-$BFFF were banked in via MAP, as well. I think I read in some older conversations that the MEGA65 team was able to reduce the footprint of the DOS ROM so the extension into $C000 was not needed anymore. Therefore, the area at $C000-$C7FF is all $FF in the latest stable MEGA65 ROM.

I wonder whether we want to capture such historical details somewhere in a consolidated fashion, because currently they are spread over old Commodore code, the old C65 spec, MEGA65 ROM code, and Discord chat history.

Edit: adding to my comment about the MEGA65 stable ROM not using $C000-$C7FF for DOS anymore. I found a Discord conversation where it was suggested to remove REL file support from the internal drive's DOS, as it was broken/incomplete and nobody saw a value in fixing it. It seems to be removed and allowed to shrink the DOS below the $BFFF mark.

https://discord.com/channels/719326990221574164/782757495180361778/841567192524914708

[nobruinfo]

@dansanderson I took me some days to not only swallow memory management but to actually properly (as you would say) digest the wonderful story of the Mega's memory concept. Wow, this is an easy read but hard to absorb - fascinating. Thank you a thousand times, I would not be able to do such thing.

On little thing hit me hard, being me again of course. I wanted to follow some examples in BASIC. I started by using MAP, sometimes even with the startup MAP 128 situation. Then wanted to make my life simple and went into the MONITOR. Slap, mapping is not set there. Yes, I gave you a screenshot of some JMP table in discord. But that was the only coincidence because of error I used 3xxxx adress space instead of BANK. - Could "BASIC map will not be considered in MONITOR" be a later edit or am I missing something else? Also, map doesn't seem to be a MONITOR command. One would have to do it by entering a short assembly language programme.

Of course all being no urgent matter. Thank you for your opinion.

[dansanderson]

This is interesting. As far as I know, the $C000 area was always mapped via $D030. This is a hard-wired mapping from $2C000 to $C000. In the original Commodore ROM, the area $3C000-$3CFFF was really reserved and "empty" (all $FF). Do you have any reference that explains why that mirror was done with the MEGA65, as it was not the case originally?

I don't have a reference and am still figuring it out, so your info is useful and we can keep digging. The ROM build fills 2C000-2CFFF and 3C000-3CFFF with the same data. The first half is mostly empty (FF). The ROM source labels 2C000-2C7FF as "DOS," 2C800-2CFFF as "KERNEL" (with an E ;) ), 3C000-3C7FF as "VOID" and 3C8FF-3CFFF as "KERNEL."

There's key map data starting at C000, so DOS/VOID isn't completely empty. The system initialization routine starts at C800, and there are comments—probably vintage—describing C000-CFFF as "common ROM" and emphasizes its importance for system routines ("this MUST be here!"). The area also contains serial, banking, and some monitor entry point stuff.

I'm very interested to know if C000-C800 is usable space. We're short on places to add things. I'm not eager to add features but we still have a long bug list and I want to be able to triage. I wrote a ROM visualization tool, currently in the development branch ("make report").

I wonder whether we want to capture such historical details somewhere in a consolidated fashion, because currently they are spread over old Commodore code, the old C65 spec, MEGA65 ROM code, and Discord chat history.

I want to add internals documentation to the ROM repo for current and future maintainers. I'm keeping private notes for now.

Edit: adding to my comment about the MEGA65 stable ROM not using $C000-$C7FF for DOS anymore. I found a Discord conversation where it was suggested to remove REL file support from the internal drive's DOS, as it was broken/incomplete and nobody saw a value in fixing it. It seems to be removed and allowed to shrink the DOS below the $BFFF mark.

https://discord.com/channels/719326990221574164/782757495180361778/841567192524914708

I can confirm that there's no DOS code up to C357. I don't quite see which commits took it out but I believe it.

[dansanderson]

On little thing hit me hard, being me again of course. I wanted to follow some examples in BASIC. I started by using MAP, sometimes even with the startup MAP 128 situation. Then wanted to make my life simple and went into the MONITOR. Slap, mapping is not set there. Yes, I gave you a screenshot of some JMP table in discord. But that was the only coincidence because of error I used 3xxxx adress space instead of BANK. - Could "BASIC map will not be considered in MONITOR" be a later edit or am I missing something else? Also, map doesn't seem to be a MONITOR command. One would have to do it by entering a short assembly language programme.

I agree it's worth mentioning, I'll look for a place. I considered having a dedicated section on using the monitor to play with memory in this chapter, to encourage people to come up with their own experiments. In the machine code section I describe experiments that use the monitor as a way to assemble short routines and see their effects. The best I could come up with was to assemble a routine that sets MAP, reads a 16-bit address, stores it in $1900, then returns, so you can examine it in the monitor afterward.

It would be challenging to add a feature to the monitor to view data through a MAP lens. The monitor sets MAP for its own purposes, and MAP is a write-only register so the monitor can't preserve custom values between subroutine calls.

[nobruinfo]

@dansanderson So thank you for having this considered. And no, I don't need to have things changed, never. There are always good reasons. And your explanation showed once again that also here this is the case.