MV word to the XH-M188 board?

[hexagon5un]

Hi all,

I've been using a simple word to output a calibrated PWM value to set the power supply to a given voltage. The combination of that and a blocking MS function (based on the BG task) covers most of the simple use cases pretty easily.

Basically, it's

: ms 5 / tim + begin dup tim = until drop ;
: millivolts 1024 swap 5000 */ ;  
: mv millivolts pwm! ;    

Lets you write stuff like voltage schedules and ramps and whatever pretty easily, and it's fairly hardware-specific. Is this is a good candidate function to be a built-in to boardcore.inc?

Or should we go with a code example on the board's page? If that were prominently displayed, it might lead the beginner both into a little programming and some simple use cases. But then it's not always there on startup.

Built-in, or typed in?

I'm leaning toward built-in, just b/c I've appreciated having the BKEY and ADC functions on the other boards.

I'm always torn about giving people blocking-waits, especially when you look at what it's done to the Arduino community. :) But they'll just write their own anyway, and it makes simple things very simple even if it's a pitfall later on.

BTW: The 1024 is just the PWM value at which my unit (could be adjusted to) output 5.00 V. It's rather arbitrary, but it's a power of two and it's in the middle of the range, so why not? :)

[TG9541]

You're right, board support should provide what's required for writing application code in the intended functional domain of a board. Of course, writing code in assembly should be limited to where it's really worth the trouble (speed, size, core). What do you think of a board.fs file that's part of the build process as a third alternative? Of course, words provided that way wouldn't be affected by RESET.

The use cases would be like this:

• assembly code in boardcore.inc for were assembly is best
• board.fs for high level board support words
• example applications in the Wiki (or other places)

About timer code: blocking timers are a stable of "imperative single-issue programming". Next level up the ladder is "imperative loop-test programming". This is much better, but it's clumsy (code tends to be all over the place). In order to work around the limitations of the first "multitasking" came into existence... a maintenance nightmare was born!

Signals/functional programming is much better, but for efficient data flow something like a closure is required. Closures in Forth? I have to think about it.

For now making the 2nd options (loop/test) better usable looks like a good option. What do you think about "one-shot" timers for triggering actions? Generating a new timer can be done with CREATE...DOES>. The CREATE part would link timer cells to a loop/test routine in the background. The DOES> part would trigger the timer. The action part would be a normal Forth word.

Let's say our CREATE...DOES> action is called TIMER. Here would be a way to use it:

: on 1 out! ;
: off 0 out! ;
' off TIMER offdelay
on 100 offdelay

The advantage over simple blocking imperative code is that it's non-blocking while it still looks like imperative code.

Another option might be a scheduler word with a configurable number of actions (like inter here except for discrete thresholds, and with the ticker as the input parameter).

EDIT: I did a quick test with loading board.fs through the Makefile using @RigTig's loadserial.py. The problem is, of course, that hardware must be connected to right type of (simulated) serial interface. Maybe that's not the worst idea since this could be used for an integration test (HiL style ;-) ), and maybe all that "homebrew test equimpent" can be put to good use after all. Of course, there are other possible solutions, e.g. address translation, tokenization, hardware simulation, metacompilation ...

[hexagon5un]

On the main topic (boardcore.inc vs board.fs) I would lean toward board.fs for the straight-Forth stuff like what I've written above. It would be more instructive to newcomers, and my "assembly" version was going to lean embarrasingly heavily on the CALLR instruction anyway. :)

How would a board.fs solution avoid getting reset by reset? (I haven't looked into reset yet.) If this is easy to incorporate into the binary-building chain, I'm all for it!

I've been using ascii-xfr to move code over to the boards, and have an entry like this in my Makefile

%: %.fs   # generic upload target
    ascii-xfr -s -c10 -l500 $^ > /dev/ttyACM0

I define a set of BASE files that go into NVM, and then let make all point to the Forth file that I'm currently working on. Alternatively, one could just type make foo to send foo.fs to the connected board.

Downside? ascii-xfr is very slow. I'll have to look into loadserial.py.

And finally, on timers: you're probably right that a more general purpose timer solution would be better. I started writing something like that for making voltage schedules and then bailed out b/c I realized that I didn't care that it was blocking anyway, and that I was building a lot of scaffolding to solve a very simple problem.

But I do think that a more flexible timing system would be awesome.

[RigTig]

In eForth, the word to accept a whole series of characters into the input buffer is ACCEPT. ACCEPT begins with an address and maximum count, and returns with address and actual count. Now, (on linux) the program stm8flash can write to a given address, say one line at a time. To keep things synchronised, eForth could send out the buffer address and maximum length (using read ability of stm8flash), and then the buffer be written. Because this is all done via the SWIM interface on STM8, whatever baseline eForth code you might need for a particular device can be loaded from the one physical interface. The limitation of speed of loading is then almost entirely due to the eForth processing capability, and it is easy to lock in the endpoint for RESET too. Does anyone know of a better API into ST-LINK? If not, just a bit of bash code could set up read (just waiting for address and max length), parse out the results and write from given file(s) back to the eForth. The easiest way to ensure no splitting of words is just to send one line at a time from the source file.

Not at all able to contribute much on timers. I've used Forths with round-robin multitaskers, but I did not have any need for interrupts that I remember. Besides, I've never even heard of Closures. What fun have I missed out on?

[TG9541]

Accept is a good entry point. I'd like to use the STM8 simulator sstm8, which is included in the SDCC tool chain, as a testing aid, and, why not, as a compilation platform. Unfortunately, the sstm8 doesn`t support all features of the underlying ucsim (e.g. there appears to be no serial interface simulation), and thus I'd like to look into extending it.

The problem was that I couldn't make the build working, and my assumption was that this was due to a problem with my build environment. Today I found out that there is a bug in the Small Device C Compiler 3.6.0 build, which was fixed some time after the release, and SDCC revision 9922 builds fine.

The uCsim docs on using the serial interface look promising!

[TG9541]

There is a new version of uCsim under development with which I managed to load, and start a STM8EF binary, but unfortunately it stops after 5000ish cycles.

EDIT1: exactly 5984 ticks, and the HALT address was 0x008004 (the TRAP vector). The uCsim author found a bug, and I received a bug fix within hours. I'm still trying to get a Forth console.

EDIT2: a bit of progress (less unknown unknowns) - the ticket is here: https://sourceforge.net/p/sdcc/bugs/2621/

EDIT3: after some more help from SSTM8 author: it's now possible to connect any character-I/O tool through a TCP port, execute Forth, compile to (Flash) memory, dump the memory contents.

Issues:

• SSTM8 is slow (about 13% execution speed on my 1.6GHz i7 box)
• for now, the FLASH_IAPSR spin-locks need to be removed, and SSTM8 must be started with the -w option (mostly solved, see ticket)
• a solution for automatically extracting (dumping) code is needed
• binaries for SSTM8 require UART serial interface code. For many devices (e.g. W1209) some hex file patching is needed.

However, automatically generating and testing object code looks feasible.

EDIT4: I can control a simulated STM8S through a bash script (load code, run, connect to console, compile Forth code, and dump µC memory to a file) ... BUT for entering Forth code a handshake is needed. It looks like I need @RigTig 's loadserial.py to use a TCP port instead of a serial interface.

#!/bin/bash
# requires SSTM8 >= https://svn.code.sf.net/p/sdcc/code/trunk/?p=9923  
cat <<EOF > /tmp/sstm8-start.txt
set mem io_chip 0x5f 0x0a

EOF

./sstm8 -w -g -C/tmp/sstm8-start.txt -Z10001 -tS103 -Suart=1,port=10000 out/MINDEV/MINDEV.ihx &
sleep 1

nc localhost 10000 <f/test.fs
sleep 1

nc localhost 10001 <<EOF >out.txt
dch 0x8000 0x9FFF 16

EOF

nc localhost 10001 <<EOF
kill

EOF

[RigTig]

I started to write a TCP equivalent of loadserial.py, then decide to check with the gurus. Here's the link: https://docs.python.org/2/howto/sockets.html. The really hard part of network programming is that you are dealing with a fully buffered system, and that any request may not complete in a single call either. Usually a TCP connection is bidirectional, but it seems from the above code that one port is used for sending and another for receiving. (Mmm...I have no idea what dchdoes.) Anyway I've done a first cut for you Thomas, which may or may not suit your purpose and may or may not work, either. See https://github.com/RigTig/stm8ef/blob/RigTig-barebones/codeloadTCP.py and let me know what I have misunderstood. Edit: Moved the .py files into /tools/ so see https://github.com/RigTig/stm8ef/blob/RigTig-barebones/tools/codeloadTCP.py instead.

[RigTig]

Whilst I like the idea of using a loaded eForth to process more eForth code, the loading process is then only as fast as the eForth machine can process. Just dumping assembled (and compiled) data is so fast by comparison, it is very tempting to somehow process eForth code to ready-to-execute binary data. And that pre-processing can be done on a far more powerful machine. For performance, definitely I go for the pre-processing route. But there is still a part of me that likes the idea of using a little CPU with just enough code to be able to bootstrap itself into doing even more magical things. It is just not that blindingly fast, but then maybe fast enough. I might just continue my diversion into the depths of eForth to have ago at implementing my ACCEPT proposal, without prejudice on the offering by @TG9541. Thanks team for the challenges and I do keep learning!

[TG9541]

Thanks a lot. @RigTig , I'm still figuring out the details of automating SSTM8. The ICP/ACCCEPT way looks promising, too. Anyway, we're moving bit by bit to tethered Forth options (or something akin to it).

[TG9541]

Some things take a bit longer. With the help of Dániel Drotos, the developer of uCsim in the SDCC community, a solution for compiling STM8EF Forth code natively as an integral part of a build/test chain could be found!

A key to the solution was using ucSim socket communication, and the Python telnet library.

Here is my Python telnet PoC (this is my first Python script ever - I nicked some code from @RigTig ):

import sys
import telnetlib

HOST = "localhost"
path = "f/inter.fs"

tn = telnetlib.Telnet(HOST,10000)
tn.read_until("menu\r\n")

with open(path) as source:
    for line in source.readlines():
        line = line.strip()
        if not line:  continue
        tn.write(line + '\r')
        print tn.expect(['(\v|\n)'])[2]

Here is the wrapper shell script PoC:

#!/bin/bash
# requires SSTM8 >= https://svn.code.sf.net/p/sdcc/code/trunk/?p=9933
cat <<EOF > /tmp/sstm8-start.txt
EOF

./sstm8 -w -g -C/tmp/sstm8-start.txt -Z10001 -tS103 -Suart=1,port=10000 out/MINDEV/MINDEV.ihx &
sleep 1

./telnet.py

nc localhost 10001 <<EOF |gawk '/^0x/' >out.txt
dch 0x8000 0x9FFF 16
kill
EOF

f/inter.fs (still hardcoded in telnet.py) defines the words @dif and @inter. They appear nicely in the ucSim memory dump (dch).

0x09350 0x43 0xbe 0x7e 0x89 0xae 0x03 0x30 0x81 0x93 0x4b 0x04 0x49 0x52 0x45 0x54 0x85 C.~...0..K.IRET.
0x09360 0xbf 0x7e 0x80 0x93 0x5a 0x04 0x40 0x64 0x69 0x66 0xcd 0x83 0xa0 0xcd 0x87 0x0e .~..Z.@dif......
0x09370 0xcd 0x87 0x0e 0xcd 0x82 0xff 0xcd 0x83 0xad 0xcd 0x82 0xff 0xcd 0x84 0x2e 0x81 ................
0x09380 0x93 0x65 0x06 0x40 0x69 0x6e 0x74 0x65 0x72 0xcd 0x83 0xa0 0xcd 0x82 0xff 0xcd .e.@inter.......
0x09390 0x87 0x18 0xcd 0x83 0x69 0xcd 0x87 0x0e 0xcd 0x83 0xa0 0x83 0x00 0x03 0xcd 0x87 ....i...........
0x093a0 0x68 0xcd 0x83 0xc2 0xcd 0x82 0xff 0xcd 0x85 0x60 0xcd 0x87 0x37 0xcd 0x82 0x62 h........`..7..b
...

This works :-)

The following remains to be done (no particular order):

• uploader features (serial, telnet, stm8flash?)
  • an include file feature for the uploader (best with a library feature )
  • detection of compile errors
  • injection of ACCEPT or UART serial interface code into software-serial binaries (e.g. W1209)
• code for reformating a uCsim memory dump into an Intel Hex file
• integration of uploaders into a Makefile framework
• a test automation feature

I'm looking forward to your comments!

EDIT: ucSim memory dump to Intel Hex: done

[TG9541]

@RigTig

I started to write a TCP equivalent of loadserial.py, then decide to check with the gurus. Here's the link: https://docs.python.org/2/howto/sockets.html. The really hard part of network programming is that you are dealing with a fully buffered system, and that any request may not complete in a single call either.

I updated your codeloadTCP.py. I don't know if the Python telnetlib also resolves the network buffering issue you mentioned, but I think that the "ping-pong" with a pace character makes any such problem unlikely (just as in serial communication).

[TG9541]

Here is a script that also converts the hex dump into an Intel HEX (ihx) file:

#!/bin/bash
# requires SSTM8 >= https://svn.code.sf.net/p/sdcc/code/trunk/?p=9933

# AWK scripts
read -d '' makeHex << 'EOF'
/^0x/ {
  cs = 0; a=":"
  Append("10",1); Append($1,4); Append($1,6); Append("00",1)
  for (i=2; i<=17; i++) Append($i,3)
  print a sprintf("%02x",and(-cs,0xFF))
}

END { 
  print ":00000001FF" 
}

function Append(x,n) {
  s = substr(x,n,2)
  a = a s; cs += strtonum("0x" s)
}
EOF

# create sstm8-start.txt 
cat <<EOF > /tmp/sstm8-start.txt
EOF

./sstm8 -w -g -C/tmp/sstm8-start.txt -Z10001 -tS103 -Suart=1,port=10000 out/MINDEV/MINDEV.ihx &
sleep 1

#./telnet.py
tools/codeloadTCP.py f/inter.fs f/persist.fs

nc localhost 10001 <<EOF |gawk "$makeHex" >out.ihx
dch 0x8000 0x9FFF 16
kill
EOF

Note that the AWK code makes use of gawk extensions (e.g. and, strtonum). Working without those is possible (but "AWKward").

EDIT: makeHex implemented without awk array EDIT: persist.fs added. @hexagon5un : check comment below.

[TG9541]

How would a board.fs solution avoid getting reset by reset? (I haven't looked into reset yet.) If this is easy to incorporate into the binary-building chain, I'm all for it!

I added transferring the following file f/persist.fs to the uCsim based script build script above

FILE
' 'BOOT 1+ DUP $12 DUP ROT + SWAP 
ULOCKF CMOVE LOCKF
HAND

@hexagon5un , with this the Forth words defined in the build process are the new base for reset.

[TG9541]

For communicating with a simulated W1209 the serial interface must be initialized, and the wordsTX! and ?RX must be replaced. Init, and TX! are rather straightforward. For ?RX I use the following code:

QRXSIM:
        CLR     A               ; A: flag false
        BTJF    UART_SR,#5,1$
        LD      A,UART_DR      ; get char in A
1$:     ; JP      ATOKEY          ; push char or flag false
        TNZ     A
        JREQ    2$
        CALLR   2$              ; push char
        CLR     A               ; flag true
        CPL     A
2$:     ; JRA     ASTOR           ; push char or flag false
        CLRW    Y
        LD      YL,A
        ; JP      YSTOR
        DECW    X               ; SUBW  X,#2
        DECW    X
        LDW     (X),Y           ; push on stack
        RET                    

The (relocatable) serial interface code can be loaded at 0x1000, a "RAM_CHIP" location (Access Line devices have 2KiB RAM, the simulator ucSim offers some more resources).

:1010000090AE680390CF5232350C5235810000000b
:101010005CF65C720F5230FBC75231810000000059
:101020004F720B523003C652314D2704AD024F436d
:10103000905F90975A5AFF81000000000000000066
:00000001FF

To load this code, store it in a file uartpatch.ihx, and on the control console use load "uartpatch.ihx".

The words are in the following order:

Address	Word
0x1000	Serial interface init - can be used instead of HI
0x1010	TX!
0x1020	?RX

[RigTig]

Well done. Magnificent progress. I was trying to think of a way to load code into uCsim, and its done!! Special congrats on your first Python code. And my code was based on code from I know not where. Gotta love reuse of code.

Regarding network buffering, I suspect it is unlikely in a purely local network. The issue might raise its head at some point, but how about we just ignore it until it bites us. We just need to be aware of the possibility.

A philosophical question: Is using a simulator to interpret the code and then downloading code to actual chip an example of tethering or is it still within the general definition of being a standalone interpreter solution? Either way, it is a neat solution and it solves lots of problems about handling particular hardware, including possible ram shortages. I'll have a go in due course.

[TG9541]

Thanks! I'm still not there with injecting UART serial interface code into the W1209 (but the code above works as intended if I activate it manually with the Forth console on a simulated MINDEV board). There are several options, and I'll decide how to tackle it after trying some things.

As for buffering: I think that the telnet protocol handles that fairly well. I've never seen a stuck telnet console (waiting for the next byte to arrive before it sending what it already knows must be sent). That's different with plain socket communication.

The simulator option maybe wasn't practical at the heyday of Forth, and hence there is no word for it. The problem only exists in STC and DTC Forth where "metacompilation", and "tehthering" was seen as the go-to solution (ITC Forth corresponds to UCSD or Java "bytecode", but that wasn't "close enough to the metal" for "lean computing enthusiasts" dealing with slow 8bit CPUs).

[TG9541]

OK, I've got the a W1209.ihx file extended with Forth words using the uCsim method, but I had to work around some uCsim quirks, and the solution isn't fully automated yet:

• locate the serial interface code at 0xC000
• at 'BOOT (e.g. 0x8237), replace the vectors for HI, ?RX, and TX! with C000C010C020
• recreate the Intel HEX checksums
• run the Forth compile script
• replace the C000C010C020 at 'BOOT with the original vectors
• recreate the Intel HEX checksums I flashed the resulting file to a W1209 hardware, and it works!

One of the problems with uCsim is that by issuing commands one risks stopping the simulation.

[RigTig]

A long time ago, I remember writing a intel hex format generator in Forth. Probably when I was playing with cross-compilation and Forth-based assemblers. Anyway, that code is long gone, but I decided to have another go, since it might just better suit our scenario. Besides, it was the easiest problem to have a go at. Anyway, for what it might be worth, here is my first attempt:

VARIABLE LINE_SUM

: .b+ ( b -- )
  \ output hex byte, and adds to line_sum
  DUP <# # # #> TYPE LINE_SUM +! ;

: .LINE_IHX ( addr n -- )
  0 LINE_SUM !
  $3A ( : ) EMIT
  DUP .b+  \ length
  OVER $100 /MOD $FF and .b+ .b+ \ address
  $00 .b+ \ data record
  OVER + SWAP DO I C@ .b+ LOOP 
  LINE_SUM @ NEGATE $FF AND 2 U.R 
  CR ;

: .EOF_IHX ( -- )
  \ :00000001FF
  $3A ( : ) EMIT
  $00 .b+ 
  $00 .b+ $00 .b+ 
  $01 .b+  \ EOF
  $FF .b+  \ pre-calculated
  CR ;

: .IHX ( end-addr+1 start-addr -- )
  \ output 32 bytes per line 
  DO I $20 .LINE_IHX $20 +LOOP
  .EOF_IHX ;

\ usage is HEX NVM CP @ $8000 .IHX \ needs WORDS_LINKCOMP = 1
\ else HEX RAM $6E @ $8000 .IHX

[TG9541]

Nice :-) I think that .b+ can be used instead of $FF AND 2 U.R. Depending on the encoding this a bit more compact: : .EOF_IHX ." :00000001FF" CR ;

[RigTig]

Correct on both counts. Two heads are better than one.

I have just had a good read of the ST user manual (document UM0470) on the modules SWIM and DM included in STM8 chips. I am amazed that such tools exist in the hardware. The STM8 chips are just low-density, inexpensive 8-bittters and I have not worked with anything vaguely as capable.

Of course, I've had an idea whilst reading it. Basically, my idea is to use the debug module to stall the CPU just before the first instruction of ACCEPT, fill the input buffer with next line of source code (uses SWIM) and adjust last data stack entry for actual line length, then restart the CPU at the end of ACCEPT (well, any RET will do). Repeat until end of source code. Note, that no changes are required to the assembled module!! The two addresses (break and restart) need to be known, but forth.rst has those.

Accessing the ST-LINKV2 requires using a USB library (same way as stm8flash does). I am not a C programmer at all, but I reckon I can hack my way through a Python program. What do you think of this for an idea?

The bad news is that this won't work for uCsim simulator. Is there a hook we can get onto to do something similar for the simulator?

[TG9541]

Hi @RigTig

I also had a long look at the debug module description (UM0470), and I found the features quite amazing (in the 90s a professional $3000 emulator would have been only marginally more powerful). I agree, STM8FLASH can be used as a starting point but Python is better for incremental coding, and I think that it wouldn't be too hard to also do programming with the kind of tool you're proposing.

The simulator ucSim has many of the features for doing something similar to your ACCEPT approach. The problem is that the run-state of the simulator isn't fully transparent (something between a missing feature, and a bug). On the other hand, the serial interface approach works fine when I use the same kind of information that's also needed for the debugger approach (I'm working on a solution where no hex file patching is necessary).

If you'd like to take the lead I'd love to experiment with the debugger features. In extension to your approach, a debugger-tethered Forth would be the leanest solution possible!

[TG9541]

Now I have a solution for Forth build automation with uCsim for board code that uses a simulated COM port. The trick is to set a breakpoint at the end of the initialization (i.e. HI) , and to change the I/O vectors to routines in RAM:

#!/bin/bash
# requires SSTM8 >= https://svn.code.sf.net/p/sdcc/code/trunk/?p=9933

# AWK scripts
read -d '' makeHex << 'EOF'
/^0x/ {
  cs = 0; a=":"
  App("10",1); App($1,4); App($1,6); App("00",1)
  for (i=2; i<=17; i++) App($i,3)
  print a sprintf("%02x",and(-cs,0xFF))
}
END { print ":00000001FF" }

function App(x,n) {
  s = substr(x,n,2)
  a = a s; cs += strtonum("0x" s)
}
EOF

# create sstm8-start.txt 
cat <<EOF > /tmp/sstm8-start.txt
load "out/W1209/W1209.ihx"
EOF

# add a breakpoint at `HI` to the startup file
gawk '/^ +([0-9A-F]){6}.* HI:/ {print "break 0x" $1}' out/W1209/forth.rst >> /tmp/sstm8-start.txt

./sstm8 -w -C/tmp/sstm8-start.txt -g -Z10001 -tS103 -Suart=1,port=10000  &
sleep 1

# at breakpoint: inject UART code, patch I/O vectors, and start over
nc localhost 10001 <<EOF 
load "tools/uartcode.ihx"
run
EOF

sleep 1 
tools/codeloadTCP.py f/inter.fs f/persist.fs

nc localhost 10001 <<EOF |gawk "$makeHex" >out.ihx
dch 0x8000 0x9FFF 16
kill
EOF

The file uartcode.ihx contains the UART code, and does the I/O vector patching:

:04006000100010205C
:1010000090AE680390CF5232350C52359B9B9B9B20
:101010005CF65C720F5230FBC75231810000000059
:101020004F720B523003C652314D2704AD024F436d
:10103000905F90975A5AFF81000000000000000066
:00000001FF

The code above can be used in a makefile (just "W1209" needs to be replaced with a parameter). I still have some issues with the handshake (character VT, "pace") in a telnet session, but why I haven't fully understood.

[RigTig]

Ok, I'll have a go at the DM SWIM stuff. I'll leave you with the uCsim to work on. I've had a few goes at the telnet interface, but I really miss being able to scroll back to an earlier command. I am a lazy typist! However, using the simulator is really good on my little netbook when I have a few minutes to just have an explore around STM8EF. No dangling hardware!!

[TG9541]

Yes, the DM/SWIM interface is fascinating :-)

Regarding lazy typing: here is something that might catch your interest e4thcom, a development oriented serial terminal for untethered embedded Forth systems.

I'm contact with the author, member of the (very active) German "Forth-Gesellschaft e.V.". It might be a good idea to align somehow (e.g. support #include and #require in the uploader). e4thcom supports half-duplex communication, but some fine tuning for single-wire com interfaces will be necessary (the target configuration files consist of Forth code).

[TG9541]

@hexagon5un @RigTig , as mentioned before there is some discussion about support for the "Forth terminal" e4tcom going on, and today I sent a STM8S board to the author. I learned that the FILE/HAND feature is rather unusual (even if it's supported by the original eForth). I did some experiments: the uploaders work just as well without it (the uCsim/TELNET variant works better).

What do you think dropping the feature? Do you have other preferences?

[TG9541]

I solved the sync issue: since I wanted to avoid patching the Forth binary, I decided to overwrite the I/O vectors in RAM. In order to avoid patching the COLD, I decided to initialize the (simulated) UART at every TX!. As it turned out, a mistake.

Now the initialization happens only once (during the first run the EMIT vector is modified from 0x1000 to 0x1010 with a mov 0x0061,#0x10 - yes, that's hand coding) :

uartcode.ihx

:04006000100010205C
:1010000090AE680390CF5232350C523535100061e6
:101010005CF65C720F5230FBC75231810000000059
:101020004F720B523003C652314D2704AD024F436d
:10103000905F90975A5AFF81000000000000000066
:00000001FF

I'm now working on improving the uploader code (e.g. timeouts, error messages, #include).

[TG9541]

There is now a basic Forth application build framework. Some of the ideas discussed above have been implemented:

• make BOARD=<boardname> forth makes the <boardname>.ihx binary, and then compiles <boardname>/board.fs
• #include files are resolved (the feature is recursive). The search path is:
  1. cwd
  2. the directory of the current file
  3. cwd/lib
• ucSim works regardless of the type of the serial interface (injected UART code)
• the file out/<boardname>/<boardname>-forth.ihx is created

Left to do/known bugs:

• FILE isn't supported (maybe it's not necessary at all)
• support for devices with > 8KiB Flash
• <boardname>-forth.ihx is padded with 0x00
• the error handling is insufficient not stellar
• "use hardware" option missing (loadserial.py)
• include file handling in codeloadTCP.py and loadserial.pymissing

Include file support, and error handling are next. Code test features will find their way into a new "story".

[RigTig]

I ran into a brick wall: the host side interface of ST-LINK/V2 is not documented by ST (at least I cannot find it). I attempted to add another option to stm8flash, but too little information meant it did not not work. Lots of make then run cycles, so I've got the idea on how to cut-and-paste C code! More research just done: found lots of projects on GitHub regarding ST-LINK, so downloaded a selection for checking out. It seems most information has been very hardwon, including sniffers and reverse engineering. I've just found some fairly well-documented code intended for use with STM32 (https://github.com/pavelrevak/pystlink), so I think I can move forward again. I'll make my hack of stm8flash work first, then share it with you. However, I am thinking that a Python app might be easier to integrate and use into the makefile. Any thoughts on that matter from you?

FILE / HAND have served a purpose, but I do not see them as a necessary part of STM8eForth. If needed in a future project, then the developer can add back using board.fs.

[TG9541]

That's what frequently happens: documentation is "expensive" - especially if an API needs to be maintained later on. Most likely sniffing is the only way to gain any kind of information, although the single wire interface module seems to have a mostly memory mapped interface. Hacking STM8Flash first, and moving to Python later sounds like a good idea.

Did you also look at this code? https://github.com/afaerber/stlink The reference to send_usb_mass_storage_command is interesting (is that the correct functional description of a STLINK protocol element? It sounds "hacky" to me).

As for FILE/HAND: I'll remove it for now - it can come back as user code if needed :-)

[RigTig]

It seems most of the interfaces are rather hacky. The further I dig, the deeper the hole becomes and the murkier becomes the waters. I have discovered that single-stepping doesn't, at least not all the time. It always fails to stall the CPU when a new PC is set, for something like 3 to 5 bytes. And there is a 'side-effect' somewhere that triggers a HALT interrupt, also seemingly when a new PC is set. And, when single-stepping, IRET does not work as specified, leaving the address of the IRET on the stack (or maybe related to one of the earlier bugs).

Now, all of these are complicated by the total non-existence of doco. I suspect there is no doco because not only is the interface buggy, but it changes for each new version of the ST-LINK software. I've got a bit of help from someone in the ST community, who has said that OpenOCD got some code from ST. So that is my next line of enquiry.

Basically, I can get the CPU to run COLD, to stall just before the first instruction in ACCEPT, to accept a line of input into TIB, and even to update the length of line on top of data stack, but I have not ever got to do EVAL on the TIB; the PC just does weird stuff. I'll keep playing with it, mainly because I see it as a challenge. However, it seems unlikely to be a reliable mechanism for code input, mainly because the code sequences are different to those for a straight debug interface. All the command sequences I copied and tried for initialisation cause a restart of CPU, which we don't want since we want to compile into ram. I finally cracked which two commands to leave out of initialisation to NOT restart CPU (obvious in retrospect). I've now got twice as much diagnostic code as the main code for the code injection, and still no closer to getting the QUIT cycle to go around once. At least I can dump memory and registers at will, now.

For example, using stm8flash/stlinkv2.c code, here is how to get registers (with comments showing the byte strings going in and out of ST-Link/V2 - yep, all commands are supposed to be 16 bytes long!):

            stlink2_cmd(pgm, 0xf40b, 6, 0, 11, 0x00, 0, 0x7F, 0x00);
            // f4  b  0  b  0  0 7f  0  0  0  0  0  0  0  0  0
        TRY(128, (stlink2_get_status(pgm) & 0xffff) == 0x0b00);
            // f4  9  0  0  0  0  0  0  0  0  0  0  0  0  0  0
            // response =  0  b  0  0
        stlink2_cmd(pgm, 0xf40c, 0);
            // f4  c  0  0  0  0  0  0  0  0  0  0  0  0  0  0
        msg_recv(pgm,ck,11);
            // example = 90  0 83 41  3 1c  0  6  3 fb 28
        fprintf(stderr, "PC = 0x%02x%02x\n",ck[2], ck[3]);

which takes three command sequences to STLINK/V2 and two responses. The registers match the definition in stm8device.inc, as follows:

; ***** CPU

;.ifne  0
        CPU_A        = 0x7F00   ; Accumulator                           (0x00)
        CPU_PCE      = 0x7F01   ; Program counter extended              (0x00)
        CPU_PCH      = 0x7F02   ; Program counter high                  (0x00)
        CPU_PCL      = 0x7F03   ; Program counter low                   (0x00)
        CPU_XH       = 0x7F04   ; X index register high                 (0x00)
        CPU_XL       = 0x7F05   ; X index register low                  (0x00)
        CPU_YH       = 0x7F06   ; Y index register high                 (0x00)
        CPU_YL       = 0x7F07   ; Y index register low                  (0x00)
        CPU_SPH      = 0x7F08   ; Stack pointer high                    (0x03)
        CPU_SPL      = 0x7F09   ; Stack pointer low                     (0xFF)
        CPU_CCR      = 0x7F0A   ; Condition code register               (0x28)
;.endif

[RigTig]

I've been using SWIM commands, like in stm8flash, but OpenOCD uses the DEBUG commands. I've seen them used in a couple of other places too, but the list in OpenOCD is more extensive than any other. I was going to try OpenOCD, but it does not support STM8 (STM32 is well supported, it seems). Of course, there are commands for options that do not exist on STM8, but I am hoping the ones I need might work better than the SWIM set. Off we go again!

[TG9541]

You have come far! One point I don't understand: you write that the command interface changes for each new version of the ST-LINK software. Does that mean that the ST-LINK software updates the firmware in the STM32 µC inside the dongle? Unless the part of the API necessary for programming a target is very stable stm8flash must be expected to get incompatible some day with new dongles, right? I guess that the easiest way to learn how the API works is using a USB sniffer and the ST tools.

[RigTig]

Once upon a time there was ST-LINK, and it accepted a bunch of commands pretty much as used in stm8flash. Then there was ST-LINKV1 and there was a whole new bunch of commands (as well as the old ones). This version is what is in texane/stlink. Somewhere along the line was ST-LINKV2. Then there was ST-LINKV2-1 and that is what is used in OpenOCD (and all the old stuff is referenced there too, but not used). Of course, what chips are actually supported is another 'moving feast'. OpenOCD does cover STM32 chips but not STM8. To cap it all off, the libraries to support USB interfacing are somewhat kludgy too. (I see comments occasionally like 'Just need to add 1 here to get this to work'), which is what I do when I am hacking something. I just had no idea that so much code is still in what seems to be just-hacked state. The bad news is that the DEBUG commands don't work for STM8; I cannot even get some of them to give me a response. It seems all the effort has gone into making the ST-LINK software work with STM32 (and lots of other chips), but ignored our little STM8. I am going to take a break from this for a while, and design an effector for my big 3D printer. I've decided to use silicone in cartridges (like builders use to fill gaps). Should produce something very wobbly, but soft to walk on!! I'm not disappearing, but I need a change of focus for a while.

[TG9541]

:+1:

[RigTig]

Just heard than someone is writing code for STM8 support in OpenOCD. Haven't followed up yet, but might be useful in its own right.

[TG9541]

I guess this issue has been resolved now. Words like mV can be added to the <target-dir>/board.fs, and uploaded with e4thcom, or codeload.py through the serial interface, and also with ucSim in a Docker container (e.g. in Travis-CI).