Support the ZX basic compiler directly

[kosarev]

It might be a good idea to add a direct support for the https://github.com/boriel/zxbasic compiler in the emulator.

Benefits

• The users of both the compiler and the emulator do not need to bother with intermediate steps such as generating .tzx files.
• Speed-up. The compiled code can be executed instantly; no need to wait for the emulator to load the tape file. Sounds like a huge potential win in terms of development and testing time?
• Further extension of the variety of possible unattended scenarios such as converting the compiler .bas programs to .z80 snapshots and .scr screenshots.

What needs to be done

• The compiler infrastructure has to be available as a library (if not already). This includes handling of translation diagnostics (errors and warnings).
• The emulator would need a mean to get the compiled program in a form of memory blocks and an entry point rather than a tape file.
• We need a way to distinct the compiler sources from the regular ZX Basic files, which too use the .bas extension. Possible options may range from giving compiler sources some more special extension (which is probably the simplest way) to scanning the source in order to guess its type (perhaps not very reliable and relatively hard to implement).

Jose @boriel, what do you think?

[boriel]

Not yet done, but it can be done.

• If no format is provided, a default raw binary with .bin extension is created.
• The compiler can be invoked from another source file with zxb.main(argv) where argv is the command line args, like in C (that is, even the zxb.py filename should be the first element of the array. The format is a list of strings. I.e: zxb.main('zxb', 'program.bas', '-o', '/tmp/testbinary.bin') will compile the program and return 0 on success, 1 on failure. If success the file tmp/testbinary.bin can be direcly written into ram starting at the ORG position (by default 32768).

Anyway, I'll improve the API so a more direct approach can be used.

[kosarev]

@boriel Yep, that at least may work as a prototype solution. Will give it a try. Thanks!

[boriel]

the origin of the compile code (ORG) is, by default, 32768, but can be set to any other location with ['--org', '25000'] for example.

Also, bear in mind that tzx and tap files contains a Sinclair BASIC loader which sets the RAMTOP below that ORG (i.e. for 32768 you will see CLEAR 32767 in the BASIC listing). This will signal the ROM not to use the RAM above that location. Otherwise writing the binary data directly might corrupt the ROM BASIC interpreter Stack and hang the machine.

A solution for this could be starting with a machine state where the RAMTOP is already set to, for example, 32767 and then load the .bin raw binary above that location.

What do you think?

[kosarev]

@boriel Yes, I think we could start with using the default ORG and set RAMTOP up in the resulting snapshot respectively (and thanks for reminding about it, by the way :-) ). But doesn't that .bin file has the RAMTOP value set as necessary already?

As to how to support other ORG values, maybe for now it would make sense to rely purely on #pragma org? I believe I saw something like this line in some of your .bas files. In that case on the emulator side we would need a way to pull the ORG value from the compiler somehow. Or, can I just look into RAMTOP in the .bin file?

[boriel]

#pragma org will also work. However this just means the assembled code is set to run in that location (with the jumps resolved, etc). But before that a empty ram file with the ramtop set for 32767 should be preloaded. For example: executing CLEAR 32767 in an empty RAM emulation will leave the state ready.

Then to that RAM we should overwrite with the contents of the .bin file starting a 32768.

[kosarev]

@boriel Looks like with https://github.com/boriel/zxbasic/commit/8821becf3cdba9f3ef2393a50ec1b7da34cf8184 and https://github.com/kosarev/zx/commit/c837fbdc36cc8c2096a13e00e832afe41e912fc7 in place the emulator has all the necessary info.

One more question, though. Can you please confirm it explicitly that the way the emulator uses the https://github.com/boriel/zxbasic compiler does not violate its license? The emulator is available under the MIT license and the compiler seems to be GPL'ed, so it might be worth some clarification. :-) Thanks!

[boriel]

GPLv3 basically allows you to do everything, except that the sources should always be available and that if you make a modification on the compiler you should publish it to the community (that is, the compiler .py itself; the .asm / bas library are under MIT / BSD licenses)

If you need to distribute the compiler or a variation in a closed source manner, I can relicense it under MIT. anyway.

[kosarev]

GPLv3 basically allows you to do everything, except that the sources should always be available and that if you make a modification on the compiler you should publish it to the community (that is, the compiler .py itself; the .asm / bas library are under MIT / BSD licenses)

If you need to distribute the compiler or a variation in a closed source manner, I can relicense it under MIT. anyway.

Perfect, thanks for the clarification!

The emulator doesn't currently specify the compiler as its dependency, and checks it dynamically if it is installed. And if it's not, exits gracefully with a message explaining the situation. (By the way, any improvements to the message, such ones that provide more details on how to install the compiler, etc. are very welcome. :-) ) So looks like the license matters are settled for now.

[kosarev]

Supported via: 9f33d4b [#11] Advertise the support for ZX Basic compiler sources. c837fbd [#11] Translate ZX Basic compiler sources on the fly.