Add an assembler of some sort

[scotws]

We already have a disassembler, so we should go all the way.

[SamCoVT]

Does it make sense to try to port your tasm65c02 to assemble directly on the hardware? The nextname stuff looks complicated, but if you drop the labels and use Forth's control structures directly (which we can get away with in an STC Forth), the rest should be able to port over fairly easily. We may want to make a plan for how to handle multiple vocabularies, as adding both an editor and an assembler will add a lot of words to the dictionary.

[scotws]

I had thought about that, but the assembler is pretty large, and now that we have the compare instruction for strings, we should be able to make it really small. I also wonder if the traditional assembler notation might be easier to process (it would certainly be easier for people who are not me to use) on this level, because all mnemonics are three characters long. Honest, I'm not sure about this.

However, I agree that we should solve the vocabulary stuff first. I'll open an issue for that.

[scotws]

The basic idea is to be able to use the the same mnemonics data for the disassembler and the assembler, just for space reasons, even if this means that the assembler and disassembler have to be swapped out at the same time. Currently, the disassembler alone is using about 2 KByte, which is just too much for what it does.

Also, we note that the assembler should be fast, because it will be part of various words, but the disassembler can be slow, because it's going to be for slow humans.

[scotws]

I'll be adding a version of https://github.com/scotws/tasm65c02 as a first assembler.

[scotws]

@SamCoVT My thoughts so far: We keep the actual assembler routines in a separate file with the most very original name assembler.asm so that native_words.asm doesn't overflow even more than it is now. I'd like to label the instruction routines xt_asm_<OPC>, following your example of nt_asm_nop in headers.asm.

Since we want to keep the Forth words for the individual instructions in a normal Dictionary format so we can use all that new vocabulary magic, my current idea is to have normal header entries where NOP is now, and then have the actual routine look something like this:

xt_asm_nop:          
                jsr asm_common
                .byte $EA, 1
z_asm_nop:

The subroutine jump to a common routine pushes the address on the Return Stack where we can then find the opcode and the length in bytes. asm_common takes these and, if appropriate, takes the operand from the stack where it was pushed before.

This means that we use five bytes for each instruction here and at least 11 bytes per header entry for a total of at least 16 bytes per instruction. For 178 opcodes (I think) on the 65c02 that would mean we use at least 2.8 Kb this way. My goal would be to keep the whole assembler under 4 Kb.

The first iteration would be just the instructions till we are sure that works. The directives would follow tasm65c02 (https://github.com/scotws/tasm65c02, now released into the public domain). We should have one from the beginning that pushes the Accumulator to the Forth data stack, maybe, uh, push-a. Don't think we need push-x etc because we can just use TXA. We need to figure out what to do with BRK, either as NOP or abort?

For obvious reasons, the first step will be some Python program that takes the instruction data and creates the assembler listing for assembler.asm and header.asm. I'll try to get a basic version of the assembler working before Christmas.

[SamCoVT]

That all sounds pretty good to me. Once we have a working assembler, I can translate the hand-assembled code in the cycle tests. I really only need LDA and JSR for that. I also wasn't sure if the assembly words were supposed to be immediate - I guessed yes, so that's how NOP was put in.

[scotws]

Design question: Do we want underflow checking for the instructions that require an operand? Since we only have to check for TOS, it could be handled with a subroutine jump to a local version, but it would add an awful lot of bytes and slow stuff down considerably. My suggestion would be to leave it out for now, document the fact, and add it later if the need presents itself.

[SamCoVT]

If all you are passing to asm_common is the opcode and the length, does it make more sense to pass them in A and Y, rather than all the gyrations to get things after the JSR? It would be nice to have underflow checking as it will help newbies (and me)... can't asm_common do that? The length tells asm_common whether or not there is supposed to be an argument, and it's zero or one argument for each opcode, right? (I'm assuming you have the addressing mode in the opcode name)

[scotws]

After reading the very unhelpful ANS Forth page on CODE and ;CODE and the far more useful page from Gforth (http://www.complang.tuwien.ac.at/forth/gforth/Docs-html/Code-and-_003bcode.html#Code-and-_003bcode), I'm going to focus on CODE and END-CODE first. I'm not quite sure I understand the ;CODE example there yet, but that is the normal state of affairs for me with new Forth words at this stage :-).

[scotws]

@SamCoVT - I had originally thought that

lda #$EA
ldy #$01
jmp asm_common

would be inferior to my above solution because it uses one byte and one cycle less per instruction. However, now that I have actually written a version of asm_common, I think you are right because this way, we can immediately call jsr cmpl_a for the operand. Will change, thanks. You are probably right that underflow checking would be a big help in debugging, so I'll include that in asm_common.

[SamCoVT]

Because Tali is an STC forth, you don't need to do anything at all to switch between forth words (which are encoded as JSRs) and assembly. In all of the other types of Forths, you do need a method to switch, and that's what the CODE, ;CODE and END-CODE words are for.

The word CODE is just like : except that it's supposed to start running the assembly for that word. The word END-CODE is supposed to stop assembling, and it's actually not standardized by ANS - it's just suggested. The word ;CODE is interesting because it allows you to switch from Forth into assembly in the middle of a word definition. ANS does not seem to provide a word to switch back, but hints at the END-CODE word being in common use. It wasn't clear in my reading of the standard if END-CODE was the equivalent of ; or if a semicolon would still be required after using END-CODE. Edit: It looks like GForth uses END-CODE in place of ;, which make sense because it has to stop running assembly and go back to running DTC (or whatever) codes.

If you want these words in Tali, it's super easy. Have CODE use xt_colon because for us, they have identical behavior. ;CODE has no effect, as Tali doesn't have to jump anywhere to start running assembly code right after a Forth word. We also don't need END-CODE, but if you want it then it can be mapped to xt_semicolon.

The beauty of an STC Forth is that you can switch back and forth between assembly and Forth words any time you want.

It's up to you whether or not you want to add CODE and ;CODE to Tali, but they aren't required at all for your assembler to work. I don't really think there's an argument for portability here, as there isn't going to be much in the way of pre-existing code that already has 65C02 assembly in the format Tali will need. Unless you want to start adding the TOOLS words, I'd say just leave them out.

I also had a thought regarding the lda thing: If the very first instructions are always

    LDA #opcode
    LDY #size 

then you can make a disassembler by traversing the ASSEMBLER-WORDLIST and looking at the second byte (because the first one is the LDA opcode and we want the operand). Once you find a match, you have identified the name in the dictionary, and you can look at the LDY operand for the size of the argument if there is one. I know you already have a table driven disassembler, but if we're already going to put all of the words in the dictionary, it might make sense to reuse that data structure.

[scotws]

Right, STC is really cool in that way as well. In fact, my proof of concept code looks like this at the moment:

forth-wordlist assembler-wordlist 2 set-order
here .          \ Remember where we are, for example 4799
1 lda.#         \ LDA #1
push-a          \ Pseudo-instruction, pushes A on the Forth data stack
rts             \ End subroutine. Don't use BRK!
4799 execute    \ Run our code
.s              \ Will show 1 as TOS!

The first line with the word lists is a PITA, though, and forces the user to know about wordlists. What we could do is use code as a shorthand for that line and end-code to reverse the wordlist changes. That way, the user can just throw in assembler mnemonics without having to think about wordlists.

Really cool idea with the disassembler, nice meta thinking there. Yes, we should certainly reuse the data structure in some way. Let me get the assembler working and then I'll come back to the disassembler.

[scotws]

So the first version of the assembler is working, though very primitive. There are no labels yet and branch instructions must be calculated by hand, but we're getting there, and I wanted a basic version up so we can see where all we can use this. Once we have >order the use will be easier because we can "hop on" with assembler-wordlist >order and "hop off" with previous without having to create the whole order.

(One problem with testing is that for some reason <true> isn't being recognized anymore. I'm not sure if this has to do with the wordlist -- do we want to move the testing words to their own wordlist? For the moment, I'm just using -1 instead).

[SamCoVT]

I double-checked the <true> issue and your problem is that you wanted to use true, which is the built-in ANS word, rather than <true>, which is declared in some of the test files (notably core.fs in order to test the word true), but which is forgotten because we are using MARKER to keep the memory usage down in the tests.