Add/mix/graft STM8 eForth into a C project

[TG9541]

While looking at ebike displays, I wonder about the process to add/mix/graft e4th into a primarily C project.

Originally posted by @sbridger in https://github.com/TG9541/stm8ef/issues/234#issuecomment-445432100

[TG9541]

Interesting problem - I had that on my list for a very long time :-)

The TSDZ2 has a Medium Density (STM8S105x4) device with 2K RAM and 16K/32K Flash. As long as the C application can live with, I'd say, 1K RAM the 4-5K Flash needed for the Forth shouldn't be a big problem.

Low Density devices with 1K RAM and 8K Flash, like STM8S003F3P6, would only allow a very frugal C application.

It's easiest if the Forth application is in the foreground, and whatever C there is lives in interrupts. Obvious use cases are:

• setting parameters, flags or state variables for interrupt code in C
• running "interrupt Forth code" from interrupts in C

What needs to be done to getting C code into the foreground depends on the use case:

• C in the IDLE loop (akin to a context switch)
• C code as words from the command line with a Forth to C interface (see Sockpuppet)

Can you think of other use-cases or applications?

[TG9541]

In the case of the ebike display I can't see why the main loop shouldn't run in a low-priority interrupt with 10ms cadence.

In the use cases above I forgot to mention the trivial case where C functions are of the type void (*benign_side_effect)(void) :-)

[TG9541]

SDCC passes parameters on the return stack (or in registers) and leaves the stack set-up and the clean-up to the caller. Good for the Forth interface.

There is an example of how to implement C functions in assembly here but the STM8 parameter passing scheme in SDCC is largely in the dark).

At first glance it looked like it's not so simple but with the help of some experiments it turns out that it's not complicated at all:

The simple function long ints2long(int a, int b) is called like this:

;   led.c: 44: result =ints2long(200, 400);
    push    #0x90
    push    #0x01
    push    #0xc8
    push    #0x00
    call    _ints2long
    addw    sp, #4
    ldw _result+2, x
    ldw _result+0, y

Hypothesis:

• parameters are copied to the return stack in the relevant size, ready for access with "SP indirect addressing", and in the reverse order of the definition
• 8bit return values are in A
• 16bit return values are in X
• 32bit return values are in MSW X - LSW Y

Note that returning structs ("aggregates") isn't implemented in SDCC Version 3.6.0 #9615 which I used for tests (error 54: Function cannot return aggregate. Func body ignored).

Based on the hypothesis above writing wrappers with DOXCODE for C functions that take and produce int parameters should be easy. It should also be possible to write a "wrapper generator" IMMEDIATE Forth word.

Passing C symbols as addresses into Forth is a different question. Maybe it's easier to write a script that creates a wrapper at C-compile-time ;-)

[TG9541]

I made a new tg951/docker-sdcc:latest using SDCC 3.8.4 #10751.

So far make all works but there is an issue with uCsim SSTM8. Hopefully there will be a solution soon. Otherwise I'll fall back to the old snapshot of SDCC 3.6.0.

[sbridger]

Can you think of other use-cases or applications?

• In the case of a display, everyone wants something different, so 4th offers a non-compiler customisation route.

• The system has a controller and a display (which eventually is of multiple kinds) with a protocol (i.e. binary packets) connecting them. The protocol evolves, and if you have different people maintaining different displays (LCD, OLED, Android), then each change at the controller requires manual changes to all the displays. So having a common C file that acts as a schema for the protocol, but which can compile for the different displays would be good. e.g You could put the maths that turns TemperatureADC->TempValue into the schema. I have thought that using a 4th/RPN style for that would make it very easy to use different languages, as it is so easy to knock up some rpn primitives.

• I quite like to use rpn in (say) C, i.e. once you have done the basic stack maths primitives, then it is just a chain of calls, and very efficient. If e4th was in the program, then C programmer can use the e4th stack primitives for maths operations.

[TG9541]

@sbridger sorry for the late reply - the flu still has me.

I'd like rephrase your proposals it (for fun and profit):

1st: use Forth as some kind of a DSL for a highly variable system concern (the display that everyone wants different.)

2nd: use Forth as some kind of a DSL for customized system features that have a different life cycle than the underlying implementation

3rd: use Forth as a library for improved code density inside C as a size/engineering-effort trade-off

Can you think of an existing C showcase application? The Open-Source-EBike-Firmware is very interesting but it's a bit expensive to build a test set-up.

[sbridger]

Not off the top of my head for STM8. On ARM or 8051, I would say BLE modules. The bluetooth libaries are all C, but I think that a forth programmable BLE/bluetooth module would be rather nice. They have heaps of either flash or ram code space for 4th. They also tend to have a bit of a big ugly C codebase, and for CC2541 a commercial compiler, which is all an overkill for making a simple BLE gadget.

[TG9541]

I agree that complex SoC with an opaque code base have the most benefit from a lightweight system-integration oriented language. A good example is Punyforth for the ESP8266.

Perhaps I'll try something from STM8_templates by @gicking. It's not yet clear to me how well it's integrated with SDCC, and I guess it would be best to pull everything into a Docker container.

Edit: after initial tests and discussions with @gicking sduino appears to be a better choice.

[sbridger]

As a matter of interest, how big is the assembler overhead of using the (4th) data stack for params of a C function call, instead of the return stack? i.e. how compact is the datastack push/pull instructions on STM8?

[TG9541]

There is quite some overhead: Data Stack push/pull on the STM8 is quite expensive: there is no index register auto-increment. In STM8 eForth the inefficient push/pull is hidden in the Forth primitives.

Apparently there are Forth systems that use DTC (i.e. no CALL/RET) that use the return stack as the data stack. Such a system would be easier to integrate with SDCC but it would have other trade-offs.

On the other hand, C has a self inflicted penalty: the parameters placed on the return stack before the call must be removed after the call!

Data passing between functions is therefor much more expensive than in Forth. Some µC C compilers (e.g. Keil MCS51) try hard to do all parameter passing through registers, which sometimes leads to efficient chaining of functions. That works quite well for flat call hierarchies but it has obvious limitations (don't even think about recursion). The main problem is that data has to be copied because there is no "contract" in C that a function are allowed to alter data on stack of he calling function. Forth is much more akin to languages with nested functions. Functional languages with closures are on a next level. In Forth, this would require to pass stacks around.

[TG9541]

Merge d2c29ea makes STM8 eForth compatible with SDCC 3.8.4.

Daniel Drotos, the author of uCsim, of which sstm8 is used in simload.sh and codeload.py, implemented a new "lockstep mode" for the telnet interface of the STM8 UART simulation. Thanks a lot!

There is now a docker-sdcc:3.8.4 and .travis.ymlnow refers to it.

Apparently the register and stack interface to SDCC functions didn't change from SDCC 3.6.0 to SDCC 3.8.4, however, it's not certain if my understanding is right, or if there are any plans to change the interface. I raised that question in the SDCC Forum.

[TG9541]

There is an answer from @spth, SDCC maintainer, in the forum above: my guess based on observations was right but the interface is still considered an implementation detail even if it has been stable for a long time (I proposed to give the interface some kind of a formal status).

There is also 64 bit parameter and results passing, which is maybe not the most interesting for a C-Forth interface. I updated the Wiki here.