stm8gal contains non-free binary blobs

spth commented 6 years ago

stm8gal currently contains some non-free binary blobs for some target devices. They should be replaced by free software.

Philipp

spth commented 6 years ago

Philipp

gicking commented 6 years ago

hi Philipp, I see your point.

To my understanding the RAM code blobs under question interface to some bootloader routines in ROM of the respective STM8. Without API documentation, the only way I see is to reverse engineer the hexfiles, and then re-implement them differently...? Or do you see another option?

Two more questions / thoughts:

the new, more liberal SW license now does allow re-distribution. Besides, even with new RAM routines, the ROM code would remain proprietary. So what exactly is the point to re-implement the RAM routines?
does reverse-engineering code and re-implementing it already make it free in a legal sense? Or, to ask differently, how different does the code need to be to free it from the proprietary license?

Thanks, Georg

gicking commented 6 years ago

hi again,

in the meantime I disassembled the RAM routines using "naken_asm", and uploaded them to "STM8_Routines/ASM". Depending on the respective STM8 device and BL version, the code is VERY simple or quite long. Is your proposal to re-engineer the function of the routines, and re-write them while keeping the same entry addresses? And as I am not savvy with ASM, would you or anybody else be willing to support?

For your feedback (and ideally help) thanks a lot in advance!

Regards, Georg

basilhussain commented 3 years ago

I have been investigating the disassembly of the current ST erase/write routines to see what exactly it is they do. To aid in understanding, I also dumped and disassembled the ROM bootloader from an STM8S208 I have - see attached.

My observations so far about the erase/write routines are as follows. They focus only on the bootloader_write_erase_128k_ver2.4 version.

There are two entry points: one at 0xA0 for the erase function, one at 0x180 for the write function. These locations are fixed and must be maintained for compatibility.
The code does not use the stack. All input and output is done using global variables, whose locations are also fixed. Again, their usage will be necessary for compatibility.
The write routine is only used for block programming. The UM0560 documentation talks about how the bootloader's 'write' command does either word (although it sounds like they mean byte) or block writing as appropriate, but the ROM code must handle the word/byte-wise writing itself, and only call the RAM write routine for block operations.
Both write/erase code call back to a ROM function (at 0x608A) in several places - particularly within loops. I have deduced from the ROM disassembly that this is simply a watchdog refresh function. This will need to be retained in any substitute code. No other function calls are made.
At least a small portion of any substitute erase/write routines will need to be written in assembly due to the absence of 24-bit 'far' pointer support in SDCC, which would be necessary when writing to flash locations >32KB (using ldf instruction).

I'm going to attempt to transcribe the erase/write routines in to C code, such that perhaps we can then have a base upon which we could build our own open-source-licensed version that is embedded with stm8gal.

STM8S208RBT6_bootloader_disasm.txt

basilhussain commented 3 years ago

Some more observations:

The earliest versions of the 32K and 128K routines do not include any nop padding. I guess the ST engineers at one point modified the first version of code but then found they had to adjust all calls to it in the ROM code, so decided to add some future-proofing and pad the code out to give room for any future changes.
The differences between versions are small. Excluding the oldest non-padded version (too difficult to compare, so I didn't bother yet), the changes are almost exclusively just different addresses for global variables and the watchdog function call.
Differences between newest 32K and 128K versions are also small:
- Erase: For 32K, only handles sectors up to 0x20; for 128K, up to 0x81. Code for translating sector numbers to addresses is correspondingly smaller for the former.
- Write: code is essentially identical. Not surprising, as 32K and 128K devices share a flash block size of 128 bytes.

basilhussain commented 3 years ago

I have created a GitHub repository with my work so far at transcribing the erase/write routines to C code.

https://github.com/basilhussain/stm8-bootloader-erase-write

I only worked with the 128K v2.4 so far. I also just focused on replicating the logic and structure of the code. Didn't bother to try and assign names to the global variables yet, as their meaning is not 100% clear yet.

The code at present compiles and everything is located at the correct addresses, but has not been tested in any way whatsoever. There is also one significant problem that needs resolving: that both erase and write code are too large! The erase code spills over 0x180 and the write code past 0x1FF.

gicking commented 3 years ago

great, thanks! This is a base I can start with :-)

basilhussain commented 3 years ago

You've probably seen that I've now adapted it so it covers both 32K and 128K devices of various bootloader versions (but not the earliest versions yet). :)

I'm currently seeing if I can reduce the size of the compiled code, as it doesn't fit as-is - far too large!

gicking commented 3 years ago

:-) great! I will wait for you then. Somehow I have the vague feeling you can do better than me :-)

gicking commented 3 years ago

PS: is calling the WD service routine bigger than setting 2 WD registers?

basilhussain commented 3 years ago

PS: is calling the WD service routine bigger than setting 2 WD registers?

Unfortunately, no. The ROM watchdog refresh routine actually sets 4 registers:

0x608a:  35 55 50 e0    mov $50e0, #$55                          cycles=1
0x608e:  35 05 50 e1    mov $50e1, #$05                          cycles=1
0x6092:  35 aa 50 e0    mov $50e0, #$aa                          cycles=1
0x6096:  35 7f 50 d1    mov $50d1, #$7f                          cycles=1
0x609a:  81             ret                                      cycles=4

17 versus 3 bytes... :)

basilhussain commented 3 years ago

I have created a new branch in my repo, called size-reduce, with efforts made to reduce the size of the code.

Managed to get it to squeeze in, but only just! The erase routine area has 1 byte to spare!

Georg, if you have any further ideas for size optimisation, please make any amendments to that branch.

gicking commented 3 years ago

Basil,

just checked the version you created with a modified stm8gal. Specifically I (only) skip upload of the STM RAM routines and uploaded your RAM code prior to flash upload. For compatibility I use a 128kB STM8S207MB. Unfortunately I get an error.

Here is the command: ./stm8gal -p /dev/ttyUSB0 -w RAM_Routines/erase_write.ihx -w dummy.s19 -v 3

And here the output: stm8gal (v1.4.0) reset STM8 and press open serial port '/dev/ttyUSB0' with 115.2kBaud ... done synchronize ... done (ACK) check UART mode ... done (duplex) get device info ... done (STM8S; 128kB flash; BSL v2.1) load 'erase_write.ihx' ... done (938B) convert Intel HEX file ... done (385B in 0xa0 - 0x220) write 385B / 385B in 0xa0 to 0x22f ... done
read 223B / 223B from 0xa0 to 0x17e ... done
read 114B / 114B from 0x180 to 0x1f1 ... done
read 48B / 48B from 0x200 to 0x22f ... done
verify memory ... done load 'dummy.s19' ... done (590B) convert Motorola S19 file ... done (210B in 0x8000 - 0x80d1) write 210B in 0x8000 to 0x80d1 Error: in 'bsl_memWrite()': at 0x8000 ACK3 failure (expect 0x79, received 0x1f)

As you can see, RAM upload works. The verify is in 3 steps as write, erase and common are not consecutive. But when I try to upload to flash I get a 0x1F = NACK. Any idea what could be wrong here?

By the way, do you have a means of debugging this procedure via ucsim? I understand you have the ROM code as ASM, so in principle that should be possible...!? Because with only a NACK it's hard to dig deeper... ;-)

gicking commented 3 years ago

Basil,

for sector erase the story is a bit different: I get no error message, but the flash is not erased.

Here is the command: ./stm8gal -p /dev/ttyUSB0 -v 3 -w RAM_Routines/erase_write.ihx -e 0x8000

And here the output: stm8gal (v1.4.0) reset STM8 and press open serial port '/dev/ttyUSB0' with 115.2kBaud ... done synchronize ... done (ACK) check UART mode ... done (duplex) get device info ... done (STM8S; 128kB flash; BSL v2.1) load 'erase_write.ihx' ... done (938B) convert Intel HEX file ... done (385B in 0xa0 - 0x220) write 385B / 385B in 0xa0 to 0x22f ... done
read 223B / 223B from 0xa0 to 0x17e ... done
read 114B / 114B from 0x180 to 0x1f1 ... done
read 48B / 48B from 0x200 to 0x22f ... done
verify memory ... done erase flash sector 0 @ 0x8000 ... done, time 16ms jump to address 0x8000 ... done done with program

But if I then read out the flash @ 0x8000, the data is still there. Maybe it's passing the address...?

However, mass erase seems to work :-)

But I have to run now. I will check more thoroughly over the weekend. Sorry!

basilhussain commented 3 years ago

As you can see, RAM upload works. The verify is in 3 steps as write, erase and common are not consecutive. But when I try to upload to flash I get a 0x1F = NACK. Any idea what could be wrong here?

The flowchart figure 22 in UM0560 indicates if the bootloader gives a NACK at that stage (ACK3, after no. of bytes and data sent), then it did the write and got an incorrect checksum. Perhaps the write is not succeeding, or writing incorrect data. I will have another look at the disassembly and C code to see if I got it right.

By the way, do you have a means of debugging this procedure via ucsim? I understand you have the ROM code as ASM, so in principle that should be possible...!? Because with only a NACK it's hard to dig deeper... ;-)

I suppose in principle it should be possible to run the ROM bootloader Hex file in ucSim. Just need to load it and tell simulator to run from 0x6000. Judging from the docs, if it doesn't find special values in first byte of flash at 0x8000 (INT or JPF instruction), it will consider the chip 'virgin' and continue with bootloader, which ties nicely with the fact that flash memory is all zeroes by default in ucSim. However, some way of communicating through the simulated UART will need to be found.

ucSim docs says it can redirect UARTs to a file, but only gives examples for Linux. As I am on Windows I tried a named pipe, but it doesn't seem to work properly. You may have seen my SDCC user mailing list post on the subject. Also, stm8gal doesn't seem to support giving a named pipe as the port argument:

stm8gal (v1.3.0)
  reset STM8 and press <return>
  open serial port '\\.\pipe\ucsim' ... Error: in 'set_port_attribute()': get port attributes failed with code 1

Perhaps this simulator stuff would succeed on Linux. :-)

basilhussain commented 3 years ago

See also: https://community.st.com/thread/44550-the-missing-piece-for-a-fully-free-toolchain-st-link-firmware

By the way, this link seems to be defunct these days. It just leads to a page requesting a login.

I found the thread in question now here: https://community.st.com/s/feed/0D50X00009XkVtYSAV

gicking commented 3 years ago

proprietary STM flash w/e RAM routines replaced by OSS routines from https://github.com/basilhussain/stm8-bootloader-erase-write (thanks Basil!)

gicking / stm8gal

stm8gal contains non-free binary blobs #4