Compilation of w600 port II

[rkompass]

We continue here the discussion of development/debugging the w60x port in branches w60x and w60x_r.

[rkompass]

So this area is written behind the binary, and every time a binary is flashed, it may be recreated after it? I.e. TLS does not have to keep it's variables for longer, unlike the parameter area variables like MAC address and the like?

It seems to be re-created if false. For long term use, it's better at the start or end of flash. Why don't we simply keep it at the end?

Because you may then have a continuous area of flash for the 2M version, i.e. a 1.4 MB file system. O.K. you have that anyway, if it's configured this way (at the end of the second MB). And after you add QSPI flash, you install a new MP binary. So not really necessary to change the old position.

[robert-hh]

I did more of testing for the file corruption events, which are strange. What happens:

• I start vfstest.py using import vfstest
• I run it for a while. It always succeeds. No read error, no lfs inconsistencies.
• BUT: after a while, the source file vfstest.py is modified. The first 16 bytes are overwritten with some binary data.
• The behavior is different for the initial version of the flash driver, even if both version use exactly the same tls_fls_xxxx calls in a similar way, except for the tls_fls_erase() call. See my previous note about tls_fls_erase() effectively not executed. That was introduced by me like 3 years ago, because LFS allocated too much RAM before with 4K progsize.

Analyzing the behavior of the wm_internal_fls.c with the tls_fls_xxx functions gave the following insights:

• The functions are very complex and do much more than required for the lfs support. lfs just needs straight read, write, erase functions and handles partial write internally, tlf_fls_write() does that as well, and erases a sector if required.
• The functions never check, if a flash commend finished, even for slow operations like sector erase or page write. But there is some delay at such operations. So maybe the system is simply stopped during that time by locking flash.
• Both tls_fls_read() and tls_fls_write() each allocate a 4k cache buffer (and release them).
• tls_fls_write() reads the to-be written area before writing, causing 8k to be allocated for a single write.

Given the sector size inconsistencies of the previous implementation, the only difference between the old and new version is, whether tls_fls_erase() is executed or not. The new version does, the old version does not. So I removed that in the new version. Since tls_fls_write() erase too if required, the code still works. And the corruption of vfstest.py did not show up until now. Do not ask me why.

[rkompass]

It's good that you scrutinize on that now.

If the flash has 4K sector size, then my understanding is: For writing it has to read the sector to 4K RAM, modify the RAM and then erase the whole 4K flash area and rewrite it from the 4K RAM again. So for several sequential writes into the same sector, like LFS perhaps does, it would be beneficial (also for reducing of wear), if the subsequent writes to the same sector could be accumulated. Otherwise all the partial writes would accumulate to huge total erase counts.

But surely LFS does it another way. The bits in the flash are and-ed out, iirc, between erases. And erase establishes all bits again (like setting all bytes to 0xff). So in principle one might punch out more bits from a chunk of flash if it's guaranteed that this is onto a part of the 4K sector that has not been written to, after the last erase.

Since tls_fls_write() erase too if required

Would that mean that the tls_fls_write() is compatible with such an approach? And can you see from the code (or tests) whether LFS does use an organization such that subsequential areas are written to, so that there is no frequent write to the same places, if the same small file (say 100 bytes) is changed not once but several times in succession. Which functions does the LFS driver use? Is it tls_fls_read() and tls_fls_write()? Or some MP hal functions? Do we have the sources of tls_fls_read() and tls_fls_write(). In principle, as the write includes a read, one should be able to combine them. But if they are used in sequence, then one function allocates the 4K that the other just freed before, right? So 4K allocation in total anyway and no need to optimize here?

Another question:

Both tls_fls_read() and tls_fls_write() each allocate a 4k cache buffer (and release them).

From what memory area? From the MP heap or from somewhere else?

(There is plenty of RAM on the w600. It would be nice if we knew which parts are used by which functions.)

I just now take some time to understand the crc functions in wm_tool.c.

[robert-hh]

If the flash has 4K sector size, then my understanding is: For writing it has to read the sector to 4K RAM, modify the RAM and then erase the whole 4K flash area and rewrite it from the 4K RAM again.

No. That is not required. Flash can be usually written in any size as long as the area to be written is filled with 0xff. And that is checked by the tls_fls_write(). It caches the sector only in case that the sector has to be erased. Then, the full sector is written. Otherwise only the new data. The only restriction on writing is usually, that the data must not cross a page boundary. That's all fine implemented.

Do we have the sources of tls_fls_read() and tls_fls_write().

Yes. We have them. The buffer allocation for read is strange. For usual flash chips that should not be required, because reading can be done in any way. But maybe the built-in flash has restrictions.

From what memory area? From the MP heap or from somewhere else?

From the generic heap via tls_alloc(). The MP heap is al well allocated from that heap. The tls drivers are not aware of MP.

I just now take some time to understand the crc functions in wm_tool.c.

CRC is always a PITA.

[rkompass]

Do we have the sources of tls_fls_read() and tls_fls_write().

Yes. We have them. The buffer allocation for read is strange. For usual flash chips that should not be required, because reading can be done in any way. But maybe the built-in flash has restrictions.

I'm not feeling I have to advise you (I wouldn't do that). Just want to share my impression: For example: I just see that part of the crc is computed with unsigned long long, without any necessity (checked it). And the code of wm_tool.c does not use any means of grouping code into functions, for another example. So many repetitions of same code. Also there is no focus of keeping things local.

So I now tend to not expect too much quality of the code and feel free to follow my good intuition/understanding and simplify / reorganize if it makes things clearer.

Perhaps the 4 K allocation may be get rid of?

For usual flash chips that should not be required

Probably it's still a usual flash and the programmer didn't know..

[robert-hh]

There is a rule that says: "If it ain't broken don't fix it." The flash driver code works. It is not optimal and could be shortened, but that's it. A rework would not to change some minimal aspects like the - as it seems - unneeded allocation, but to make a simple driver for lfs. But without a detailed data sheet of that flash I will not start that.

Probably it's still a usual flash and the programmer didn't know..

From what I see it's not. It is programmed though a on-chip controller, which must provide the XIP feature. You found the register description of that controller, but it does not include a functional documentation.

[robert-hh]

Below is a log of the boot activities. The question is still, where is the entry point in ROM for the bootloader. if that is known, most of the just requires to call that code once a series of ESC is seen after boot, like 3-4. You can force that even with a terminal emulator.

Boot sequence with Secboot

1. PC sends ESC every 10 ms
2. Board responds with "\nsecboot(1MB) running v3.13...\r\nC"
3. PC continues to send ESC every 10ms
4. Board continues to send character 'C' (0x43) every ~240ms
5. PC sends 0x21 0x06 0x00 0xea 0x2D 0x38 0x00 0x00 0x00
6. Board responds with "MAC:286DCD1C2758\n
7  PC sends 0x21 0x06 0x00 0xc7 0x7c 0x3f 0x00 0x00 0x00
8. Board send character "P" (0x50) every 240 ms
9. PC sends 0x21 0x0a 0x00 0xef 0x2a 0x31 0x00 0x00 0x00 0x80 0x84 0x1e 0x00
10 Board sends 0x06 every 1 ms 10 times
11 Board switches to 2M Baud.
12 Board sends character 'P'
13 PC starts data transfer.

Boot Sequence w/o secboot after pushing Reset with A0 low.

1. After Reset with A0 low, board sends ESC every 10 ms
2. Board responds withc Character "C"
3. PC sends 0x21 0x06 0x00 0xea 0x2D 0x38 0x00 0x00 0x00
4. Board responds with "MAC:286DCD1C2758\n
5  PC sends 0x21 0x06 0x00 0xc7 0x7c 0x3f 0x00 0x00 0x00
6. Board send character "C" (0x50) every 240 ms
7. Board sends 0x06 every 1 ms. 10 times
8. Board switches to 2M Baud.
9. Board sends character 'C'
10. PC starts data transfer.

[robert-hh]

Disassembling the memory @0x0 did not reveal immediately a proper place. It's 16 of code. A brute method to force the device into the ROM bootloader is erasing the first page of the flash, address 0x8002000 - 0x80020ff. Unfortunately, machine.flash_write() calls tls_fls_write(), and that one erase the first 4k as first action, causing the code to stop. After a manual reset, the bootloader comes up. But that is not smooth. The board should directly start the bootloader. And there is no fallback. Once in that state, one must upload a firmware.

[robert-hh]

Based on the flash erase method, I added a machine.bootloader() method. It erases the first 56 bytes at address 0x8002000 (the firmware header) and then it calls tls_sys_reset().

Note that in the other w60x branch there are commits which have not been added here yet. Can be done.

[rkompass]

Can you with A0 = Low trace with Ozone where the code goes to? We should only allow to jump into the ordinary bootloader. Like waiting half a second for several escapes send CCCCCC.. if after the escapes 0x21 0x06 0x00 0xea 0x2D 0x38 0x00 0x00 0x00 is received then jump into the ROM bootloader. Whos address is yet to be found. The secondary bootloader is obsolete imho.

In the Firmware Upgrade Guide there is written, that after the Bootloader sending CCCCCC.. you can put the .FLS file onto the screen (i.e. send it though uart) with XMODEM protocol. Now I'm wondering: The .fls file always starts with 0x9f 0xff 0xff 0xa0.

Where do the 0x21 0x06 0x00 0xea 0x2D 0x38 0x00 0x00 0x00 bytes come from in this case? They don't belong to the XMODEM protocol, do they?

WM_W60X_Firmware_Upgrade_Guide_V1.3.pdf

[rkompass]

Searching through the ROM memory there are no continuous patterns of b'\x21\x06\x00\xea' or the other way round.

from machine import mem8

# ---- find byte pattern in given memory area ----
#
# addr:   memory address to start at
# length: length of memory area (in bytes)
# pat:    pattern to be searched for; list, array or bytearray
# cont:   wheter search is to be continued after first find
#
def find_byte_pattern(addr, length, pat, cont=False):
    for a in range(addr, addr+length):
        succ = True
        for k in range(len(pat)):
            if mem8[a+k] != pat[k]:
                succ = False
                break
        if succ:
            print('pattern found at: 0x{:08x}'.format(a))
            if not cont:
                break

# find_byte_pattern(0, 262144, b'\x21\x06\x00\xea')
find_byte_pattern(0, 262144, b'\xea\x00\x06\x21')

[robert-hh]

These things are are to find in compiled code. At least I found accesses to the GPIOA register. Nevertheless, we have to implement two things_

1. The trigger. Which is either a sequence of ESC after reset or a call to machine.bootloader(), which both call the mechanism from 2. The check for ESC can be earliest be done in UserMain() or at the start of mpy_task().
2. Calling or enforcing the bootloader. Enforcing is possible, calling is still a challenge. Erasing the 256 byte area between 0x8002000 and 0x80020ff is not a problem, since during firmware update the while 4k Sector will be rewritten.

[rkompass]

Calling or enforcing the bootloader. Enforcing is possible, calling is still a challenge. Erasing the 256 byte area between 0x8002000 and 0x80020ff is not a problem, since during firmware update the whole 4k Sector will be rewritten.

I had a bad feeling about this first, but now only still a bit of it. I see that there are no additional erases by that. The only remaining issue being odd is: If you execute machine.bootloader() and then this area is immediately erased, then there is no way back anymore. With the other platforms: If the file presented to the bootloader does not meet the requirements, then the old system is still there. At least I think so.

This therefore requires some caution. So that perhaps not a few ESC's after a RESET delete the MP.

I think we are not in a hurry. We will find the bootloader jump in address in the next days. For now grounding A0 is fine, it's the same on the Pyboard.

I now think that some safeguards are not bad. Like keeping the .fls file format for flashing the chip. This means keeping the update methods in wm_tool and w600tool.py. Having the .fls file generated in the build process seems also nice. I will not change the programs to accept a binary file only. In the .fls are two crc mechanisms built-in, that should prevent uploading wrong binaries. You noted the boot process also requires the image header to be written into the flash and it is checked there always.

3. PC sends 0x21 0x06 0x00 0xea 0x2D 0x38 0x00 0x00 0x00
4. Board responds with "MAC:286DCD1C2758\n

This is done by the following snippet (adapted parts of w600tool.py)

import struct

# CRC-16/CCITT-FALSE
def crc16(data : bytearray):
    crc = 0xFFFF
    for i in range(0, len(data)):
        crc ^= data[i] << 8
        for j in range(0,8):
            if (crc & 0x8000) > 0:
                crc =(crc << 1) ^ 0x1021
            else:
                crc = crc << 1
    return crc & 0xFFFF

CMD_SET_BAUD = 0x31
CMD_ERASE    = 0x32 # ROM boot only
CMD_SET_SEC  = 0x33 # ROM boot only
CMD_GET_SEC  = 0x34 # ROM boot only
CMD_SET_GAIN = 0x35
CMD_GET_GAIN = 0x36
CMD_SET_MAC  = 0x37
CMD_GET_MAC  = 0x38
CMD_GET_QFID = 0x3c # ROM boot only
CMD_ERASE_SECBOOT = 0x3f

def sendCommand(cmd):
    cmd = struct.pack('<BHH', 0x21, len(cmd)+2, crc16(cmd)) + cmd
    for b in cmd:
        print('0x{:02x}'.format(b), end=' ')
    print()

sendCommand(struct.pack('<I', CMD_GET_MAC))

The bootloader accepts commands and does some safeguard-checking with the crc.

[rkompass]

O.k. I ran Ozone to try to find out where the processor is when in the booting state. First set A0 low. Then pressed reset button. Then checked with secureCRT that the CCCCC is sent to the serial. So we are in the bootloader. Then in ozone did a Reset and Halt. Now I'm here:

 00000584   TST            R0, #0x0FC0
 00000588   BNE            0x0000058E
 0000058A   MOVS           R0, #0
 0000058C   BX             LR
 0000058E   MOVS           R0, #1
 00000590   B              0x0000058C
 00000592   NOP
 00000594   LDR            R0, [PC, #0x018C]             ; [0x00000724]
 00000596   LDR            R0, [R0]
 00000598   TST            R0, #0x0FC0
 0000059C   BEQ            0x00000594
 0000059E   LDR            R0, [PC, #0x0188]             ; [0x00000728]
 000005A0   LDR            R0, [R0]
 000005A2   UXTB           R0, R0
 000005A4   BX             LR
 000005A6   LDR            R0, [PC, #0x017C]             ; [0x00000724]
 000005A8   LDR            R0, [R0]
 000005AA   TST            R0, #0x0FC0
 000005AE   BNE            0x000005B6
 000005B0   MOV.W          R0, #0xFFFFFFFF
 000005B4   BX             LR
 000005B6   LDR            R0, [PC, #0x0170]             ; [0x00000728]
 000005B8   LDR            R0, [R0]
 000005BA   B              0x000005B4

These questions occur:

1. With Reset and Halt is the chip now at the address where it goes immediately after the reset?
2. Or is it at the code that sends the CCCCs?

What I wanted was: Connect and immediately halt. To see where the processor is (PC=?), when it writes the CCCC s and waits. Is such a jump in and halt and show me where we are? possible?

But in case of 1: I still could let it run there, as A0 is still low. Perhaps with a trace running...

(Ideally one would like to have a recording of the history of calls up to that. Is that possible?) <--- seems so.

As always I'm learning and in that process seeing that I lack some basics --- many stupid little basics I managed to avoid so far ---.

[rkompass]

The disassembly above was from the reset. When the CCCC is put to the serial:

 00001B36   CMP            R6, #2
 00001B38   BNE            0x00001B3C
 00001B3A   B              0x00001B36
 00001B3C   BL             0x00000580
 00001B40   CBZ            R0, 0x00001BB4
 00001B42   LDR            R0, [PC, #40]                 ; [0x00001B6C]
 00001B44   BL             0x00001670
 00001B48   MOV.W          R8, #1
 00001B4C   MOV            R0, R8
 00001B4E   ADD.W          R8, R8, #1
 00001B52   LDR            R1, [PC, #24]                 ; [0x00001B6C]
 00001B54   LDRB           R5, [R1, R0]
 00001B56   CMP            R5, #35
 00001B58   BEQ            0x00001C3A
 00001B5A   BGT            0x00001B74
 00001B5C   CMP            R5, #1
 00001B5E   BEQ            0x00001B7E
 00001B60   CMP            R5, #2
 00001B62   BEQ            0x00001B80
 00001B64   CMP            R5, #33
 00001B66   BNE            0x00001C3C
 00001B68   B              0x00001BB6
 00001B6A   NOP
 00001B6C   STR            R0, [SP, #0x01D0]
 00001B6E   MOVS           R0, #2
 00001B70   LDR            R0, [SP, #0x01D0]
 00001B72   MOVS           R0, #2
 00001B74   CMP            R5, #38
 00001B76   BEQ            0x00001C5C

...

 00001F26   LSRS           R0, R0, #14
 00001F28   BNE            0x00001F5C
 00001F2A   CMP.W          R9, #0
 00001F2E   BNE            0x00001F38
 00001F30   MOVS           R0, #67
 00001F32   BL             0x0000060A
 00001F36   B              0x00001F5C
 00001F38   LDR            R0, [PC, #124]                ; [0x00001FB8]
 00001F3A   LDR            R1, [R0]
 00001F3C   LDR            R0, [R0]
 00001F3E   ADDS           R0, R0, #1
 00001F40   LDR            R2, [PC, #116]                ; [0x00001FB8]
 00001F42   STR            R0, [R2]
 00001F44   CMP            R1, #15
 00001F46   BGE            0x00001F52
 00001F48   AND            R0, R9, #255
 00001F4C   BL             0x0000060A
 00001F50   B              0x00001F5C
 00001F52   MOV.W          R9, #0
 00001F56   MOVS           R0, #0
 00001F58   LDR            R1, [PC, #92]                 ; [0x00001FB8]
 00001F5A   STR            R0, [R1]
 00001F5C   LDR            R0, [SP, #72]
 00001F5E   ADDS           R0, R0, #1
 00001F60   STR            R0, [SP, #72]
 00001F62   B              0x00001B36

It seems to run in this range, while putting the CCCCC.. sequence out (and waiting for the escapes or above-mentioned crc-protected commands).

I will try out different addresses in this range to jump at.

[rkompass]

Perhaps obvious and trivial (sorry in this case): If branching into an address, that is hopefully the bootloader: Do we have to exchange the instruction set? The MP framework and interpreter, do they use ARM or Thumb instructions? If I write a @micropython.asm_thumb function in MP to try out different addresses to jump in. I now think I have to use bx address and have the lowest bit of the address to be kept at 0, to ensure it switches to ARM mode (it should be set to 1 to ensure it switches to thumb mode - this was the issue that prevented executing native/viper/asm_thumb in this port previously.

[robert-hh]

Have to try. The reset vector has the lowest bis set to 1.

[robert-hh]

Then in ozone did a Reset and Halt. Now I'm here:

Interestingly you are at 584. With my board here, Ozone stops at 51e, which matches the address at 0x04. What do you see at this address?

[rkompass]

At 0x04 I have 0x051f <-- o.k. I see there the lowest bit is added, so this might be a branch with switch to thumb-2 mode to 0x051e. At 0x051e there is:

0000051E   LDR            R0, [PC, #56]                 ; [0x00000558]
 00000520   BX             R0
 00000522   B              0x00000522
 00000524   B              0x00000524
 00000526   B              0x00000526
 00000528   B              0x00000528
 0000052A   B              0x0000052A
 0000052C   B              0x0000052C
 0000052E   B              0x0000052E
 00000530   B              0x00000530
 00000532   B              0x00000532
 00000534   B              0x00000534
 00000536   NOP

Looks like an area of addresses you may jump at, like allowing the lowest bits of the address to be not fixed.

[robert-hh]

Another thought: It might not be possible to jump directly into any ROM function (like Reset) out of the MPY context. I tried that with the reset vector, it technically works and the device executes some ROM, but no reset is performed. Besides things that would have to be disabled like interrupts and setting the vector table address, a hardware reset also clears & resets the hardware registers to a defined state, which will not happen if you call it by software. For that purpose the machine.reset() method uses a Watchdog to force hard reset

[rkompass]

You seem to be right.

Perhaps not (second attempt):

I managed to jump into the generation of the CCCCC.. with:

# ----  we try to branch to an address specified
#              as argument to jump_to() ----------

# puts the address into register r0,
# sets the lowest bit of the address
# and then branches there
#
@micropython.asm_thumb
def jump_to(r0):
    mov(r1, 1)
    orr(r0, r1)
    bx(r0)

# jump_to(0x580)    # <--- wrong address, causes hard fault.
jump_to(0x1b40)     # seems to work, press a key 

after running that and then pressing a key or CTRL-C, first a few strange symbols are generated, then the CCCCs:

���������������CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
CCCCCCCCCC

Did not try to flash in that state Tried it now and it does not seem to work.

[robert-hh]

Looks promising. I'll try later.

[rkompass]

Going back to the old ideas:

The flash erase method is not bad, if we safeguard it. How about a warning message in machine.bootloader() telling that after confirmation with (Y)/n the machine will always boot into the bootloader after reset, so the old MP will be gone?

After reset when "our" functions start (main.c for example) we could check whether the reset has a pattern (two resets then a short break and another reset and delete the 0x08002000 area then. The user would then have to press reset in a rhythm like .. - .. to enforce the flashing.

[robert-hh]

I came to the same result. I get a serial sync sucess. but then nothing more. The double reset option is what some other boards do to start a (non-destructive) bootloader. The problem here could be to have a time memory which is not lost after reset. Edit: The time.ticks_xx() counters are set to 0 on hardware reset.

[robert-hh]

That should not be a problem at all. Both way require a running MPY task. Assuming we have a machine.bootloader() which asks for confirmation, and that function is also called by the ESC check & response, we can simply check for a few ESC characters in a fast sequence and then call machine.bootloader() internally. That could even be done in a Python script.

[robert-hh]

The branch w60x_r is updated with the following changes for enabling the bootloader:

1. Calling machine.bootloader() The user is asked for confirmation. If the answer is 'Y' the device always boots into the bootloader, until a new firmware image is loaded.
2. Check for a burst of 27 ESC characters on boot in a row with not more that 20ms time between them. The flash updater sends ESC every 10 ms. If such a series is seen, the bootloader is started.

[rkompass]

Next success. Nice.

I now think that some safeguards are not bad. Like keeping the .fls file format for flashing the chip. This means keeping the update methods in wm_tool and w600tool.py.

I'm not in a hurry to finish wm_tool_r.c , need some more time.. . We won't need to change anything in w600tool.py.

[robert-hh]

I asked WinnerMicro for a suggestion for a bootloader mechanism, and here it is:

https://github.com/WinnerMicro/micropython/issues/31

I would prefer to use a magic number in RAM, as this will most likely disappear on a power cycle.

[robert-hh]

It works. I added another commit to w60x_r with the changes.

[rkompass]

Huge congrats. .... :-)

I's also very nice that wdyichen is responsive. Perhaps we should invite him to join here...

I understood that the bootloader is at address 0x1B17. Just a little bit above the range I suspected above.. And to enter it the function has to be declared as being of a certain type: void bootloader(int, int), to be called with arguments (0,0). Is that correct?

In main.c, after checking the magic mark, did you still check for the series of ESC's? After checking the magic mark, will it be deleted? Otoh the bootloader should now check the magic in the .fls header and refuse to erase the old MP, if it's no valid. (To be checked by us).

I have the part of wm_tool_r.c that generates the .fls file finished. There are many more options and procedures in this program however (still over 1500 LOCs), that I'm struggling through currently. Can we drop many of them? Just define a working serial speed, for example and one option: to upload. Calling it like wmtool2 wm_w600.bin for generation of wm_w600.fls or wmtool2 wm_w600.bin -u for generation of wm_w600.fls and immediate upload. or wmtool2 -u wm_w600.fls for just uploading the wm_w600.fls.

This is all we need for the make process, I think. As for changing parameters the other 2 programs are fine.

So the setting of the MAC address and of Wifi parameters I would like to drop also, also because we should make that available as a setting from within MP anyway.

My next suggestion if you like to continue.. :-)

• Perhaps implement machine.freq() to set an optional clock of 40 MHz. This would be only worth it, if the current consumption could be reduced significantly. There are void tls_sys_clk_set(u32 clk) and void tls_sys_clk_get(tls_sys_clk *sysclk)
• Use tls_param_get() and tls_param_set() as a way to set/change/inquire about parameters from within MP. Ideally as part of the w600 module. The country code setting, of course, would belong to the network module. I'm not sure: Should the MAC address setting perhaps belong to the network module? In the network module the WLAN should certainly be required to be inactive for changes of settings.

[robert-hh]

In main.c, after checking the magic mark, did you still check for the series of ESC's?

Yes. To start the bootloading if the bootloader is already running at reset time.

After checking the magic mark, will it be deleted?

Yes. To avoid double action.

Otoh the bootloader should now check the magic in the .fls header and refuse to erase the old MP, if it's no valid.

That's independent from this change.#

machine.freq() to set an optional clock of 40 MHz.

That's already implemented

[robert-hh]

I have the part of wm_tool_r.c that generates the .fls file finished.

I want to add secboot presence as a build option to the code. Should be easy. For MP the whole difference is the start address of MP.

[rkompass]

This we already have with the current wm_tool_r. As known: Dropping the -b secboot.img option gives the new .fls version, and including it the old .fls version with secondary boot loader. All the other gzip images are also generated but that doesn't hurt.

Perhaps we can have then a Makefile option for including the secondary bootloader, that would imply setting the binary location to 0x08010000. Or for excluding it, in which case ideally all the images wouldn't be generated, just the .fls. I would then just look at the options and see whether I may exclude the generation of additional images for this case and let everything else as it is. (Much easier than my current source rework - just adding many if then elses).

[robert-hh]

I have updated the branch w60x_r with an option to select SECBOOT=0/1 when running make. There is as we a file Makefile_build_options.txt showing the reasonable make build options.

[rkompass]

Nice. Will soon be finished with wm_tool.

[robert-hh]

The last commit drops the difference between the 1M and 2M boards.

[rkompass]

With wm_tool there is the option

 -it image_type       , firmware image layout type, default is 1M layout
                                   <0 | 3> or <1M | 2M>

It will put a 0 for 1M or a 3 for 2M into the image type word in the image header but this, as I understand it, is only for the secondary bootloader. In the .fls file there is also the secboot image and this has the image type 2. Therefore we chose to set in the new .fls file the image type to 2, as the run image is now exactly at the same place like the secboot loader previously, so the ROM bootloader doesn't notice and just writes all the stuff (after checking and then removing the first header) to flash.

So we need not use this option anymore, if we generate for the new .fls format.

---

So I changed the working wm_tool_r only minimally, so that it does not generate zipped versions of the image file. An .img file will be generated together with the .fls file. Although it's not needed by the user, it is used to generate the .fls.

In a further change I made wm_tool_r2. This now does not need to generate the .img file to produce the .fls. If the secboot image option is specified or the debug option is put on the .img will be generated though. The generated .fls is identical to the .fls generated by the versions before. Internally the new function wm_tool_pack_fls_aut() is used (_aut for autonomous).

Usage: wm_tool_r2 -b wm_w600.bin -it 1M -ra 2100 -o wm_w600

I tested also the -dl option for download, but it didn't work.

connecting serial...
can not open serial

It didn't work before, I suppose, because I didn't change the code for it. Just tested this with the old wm_tool and it gave the same error message. So, I suppose, you also flashed with the python w600tool.py so far?

With wm_tool_pack_fls_aut() it would be easy to make a simplified version of wm_tool, as suggested before. But, as you say: "Never change a running system" (too much). In our case I would say this was not a running system, however.

wm_tool_r.zip wm_tool_r2.zip

[robert-hh]

Yes. That option is still there, but not effective, since the memory layout is the same for both sizes.

[rkompass]

machine.freq() to set an optional clock of 40 MHz.

That's already implemented

Measured: The current consumption goes down to 5/6 by this (without network active). From ~29 mA to ~24 mA.

[rkompass]

The last commit drops the difference between the 1M and 2M boards.

Can we set the position of the LFS in the flash from within MP?

[robert-hh]

No. Even if the internal start address could be changed, it has to be known at boot time, when the FS is mounted. You can change it at compile time with the CODESIZE=0x..... option.

[rkompass]

The last commit drops the difference between the 1M and 2M boards.

So we don't have a umount and mount again, like in unix.

My idea with the 3x4K parameter area to be put after the w600 parameters (which probably wouldn't have worked, because the 0x08002100 address is somehow fixed) originally was that we don't have to have different compiles for 1M or 2M boards at all. But this is apparently already the case. So these 12K parameters are established at the end of the 1M or 2M flash when _boot detects that there is not file system? Or how are they placed automatically?

[robert-hh]

Or how are they placed automatically?

The addresses are set in wm_fwup.c. The name is historic and somehow confusing. During the boot process the firmware detects the flash size, and in wm_fwup.c the addresses are calculated.

[robert-hh]

So we don't have a umount and mount again, like in unix.

Technically you could do that. But _boot.py already tries to mount the file system, and if that fails, it creates a new one, which causes old content to be lost.

[rkompass]

Sorry, of course you told that already. How is your feeling wrt wm_tool ? Try to remedy the serial connection/download or let it as it is for a while? And start to go through the tests, especially Wlan + Asyncio tests?

[rkompass]

Your Makefile build options are nice. I will edit the README.md tonight. When we want to help others to compile - there is the issue that the recent gcc-arm-none-... does not work. I overwrote my gcc-arm with older files from a zip and was not sure first everything would work.

What does the V=s in make do? Can we remove it (make it automatically be present)? Could we include the make of the submodules (check for mbedtls) automatic and perhaps even the make in mpy-cross (i.e. checking if its already done?) ?

The nice thing now: With all your work I have almost nothing to do/edit to get a new updated binary. Thank you.

I suggest we add the w600tool.py to the w60x directory and a "make upload" option based on it. I use: python w600tool.py -b 115200 -e for erase and python w600tool.py --upload-baud 115200 --upload build-GENERIC/wm_w600.fls for upload. Erase is perhaps not needed at all?

[robert-hh]

About V=s:

It related to SDK.../Tools/toolchain.def. With V=s the arm toolchain is selected. So by modifying that file we can get rid of V=s. But that file is quite a mess.

[robert-hh]

Attached is a modified butnot cleaned version of toolchain.def, which does not need V=s.

#---------------------------------------------------------------------------
# Description: tool_chain_def
# 
# Copyright (c) 2014 Winner Microelectronics Co., Ltd. 
# All rights reserved. 
# 
# Author : kevin 
# 
# Date : 2014-6-12 
#---------------------------------------------------------------------------
TOOL_GNU = 1
WPS_FEATURE = 1

#---------------------------------------------------------------------------
# Define Path
#---------------------------------------------------------------------------
ifeq ($(TOOL_GNU),1)
#TOOL_PATH   = C:/Program\ Files\ \(x86\)/GNU\ Tools\ ARM\ Embedded/4.9\ 2014q4
TOOL_PATH   = 
#CROSS       = $(TOOL_PATH)/bin/arm-none-eabi-
CROSS       = arm-none-eabi-
LIBDIR = GNU
else
TOOL_PATH   = "C:/Keil/ARM/ARMCC/BIN"
CROSS       = $(TOOL_PATH)/
TOOL_INCLUD_PATH = "C:/Keil/ARM"
TOOL_INCLUD = $(TOOL_INCLUD_PATH)
LIBDIR = 
endif

#---------------------------------------------------------------------------
# Define Toolchains
#---------------------------------------------------------------------------
ifeq ($(TOOL_GNU),1)
    CC          = $(CROSS)gcc
    ASM         = $(CROSS)gcc
    AR          = $(CROSS)ar
    LD          = $(CROSS)ld
    ELF         = $(CROSS)objcopy
    STRIP       = $(CROSS)strip
    ECHO        = echo
    RM          = rm
    CP          = cp
    MAKE        = make
else
  CC          = $(CROSS)armcc
  ASM         = $(CROSS)armasm
  AR          = $(CROSS)armar
  LD            = $(CROSS)armlink
  ELF           = $(CROSS)fromelf
  ECHO        = echo
endif

#---------------------------------------------------------------------------
# Complier options
#---------------------------------------------------------------------------

ifeq ($(DEBUG), 1)
CXX_optimization = -Og -g
else
CXX_optimization = -Os
endif

ifeq ($(TOOL_GNU),1)
  CFLAGS := -Wall \
      -DGCC_COMPILE=1 \
      -mthumb \
      $(CXX_optimization) \
      --function-sections \
      --data-sections \
      -mcpu=cortex-m3 \
      -std=gnu99 \
      -mabi=aapcs \
      -march=armv7-m \
      -fno-builtin
  ARMCFLAGS := -Wall \
         -DGCC_COMPILE=1 -DWM_W600=1 \
         -mthumb \
         $(CXX_optimization) \
         --function-sections \
         --data-sections \
         -mcpu=cortex-m3 \
         -std=gnu99 \
         -march=armv7-m \
         -mabi=aapcs \
         -fno-builtin
  ASMFLAGS := -Wall \
        -mthumb-interwork \
        -mthumb \
        -std=gnu99 \
        -mcpu=cortex-m3 \
        -march=armv7-m \
        -mabi=aapcs \
        -fno-builtin \
        $(CXX_optimization)
  ARFLAGS := ru
  ARMASMFLAGS := -Wall
  #ARMASMFLAGS += -mthumb-interwork
  ARMARFLAGS := ru \
          -mcpu=cortex-m3 \
          -std=gnu99
else
  CFLAGS := --cpu Cortex-M3 --li -g \
        -DGCC_COMPILE=0 -DWM_W600=1 \
        --thumb \
        -c \
        -O2 \
        --apcs=interwork \
        --c99 \
        --gnu
  ASMFLAGS := --cpu Cortex-M3 \
          --li -g \
              --apcs=interwork
  ARFLAGS := --create
endif

ifeq ($(COST_DOWN), 1)
  CFLAGS += -DTLS_COST_DOWN=1
else
  CFLAGS += -DTLS_COST_DOWN=0
endif

_SVN_MAIN_DIR_=${TOPDIR}/.svn 
ifeq ($(_SVN_MAIN_DIR_), $(wildcard $(_SVN_MAIN_DIR_)))
then
  $(shell svn info $(TOPDIR)>$(TOPDIR)/revision )
  _SVN_Revision := $(shell grep "Revision" $(TOPDIR)/revision)
  ifeq ($(_SVN_Revision),)
    $(shell sed -i '/¡Á?o¨®/d' $(TOPDIR)/revision)
    _SVN_Revision := $(shell grep "¡ã?¡À?:" $(TOPDIR)/revision)
  endif
  _SVN_Revision := $(wordlist 2,2, $(_SVN_Revision))
  CFLAGS += -D_WM_REVISION_=$(_SVN_Revision)
  $(shell rm -rf $(TOPDIR)/revision)
endif

#---------------------------------------------------------------------------
# Define Lib Type
#---------------------------------------------------------------------------
ifeq ($(TOOL_GNU),1)
  LIBTYPE := a
else
  LIBTYPE := lib
endif

#---------------------------------------------------------------------------
# include 
#---------------------------------------------------------------------------
INCLUDES := -I$(TOPDIR)/Include/ \
        -I$(TOPDIR)/Src/OS/uCOS-II/ports/ \
        -I$(TOPDIR)/Src/OS/uCOS-II/source/ \
        -I$(TOPDIR)/Src/OS/uCOS-II/ \
        -I$(TOPDIR)/Src/OS/RTOS/include/ \
        -I$(TOPDIR)/Src/Wlan/Driver/ \
        -I$(TOPDIR)/Src/Wlan/Supplicant/ \
        -I$(TOPDIR)/Platform/Common/Params/ \
        -I$(TOPDIR)/Platform/Common/task/ \
        -I$(TOPDIR)/Platform/Common/mem/ \
        -I$(TOPDIR)/Platform/Common/fwup/ \
        -I$(TOPDIR)/Platform/Common/utils/ \
        -I$(TOPDIR)/Platform/Common/crypto/ \
        -I$(TOPDIR)/Platform/Common/crypto/symmetric/ \
        -I$(TOPDIR)/Platform/Common/crypto/digest/ \
        -I$(TOPDIR)/Platform/Common/crypto/math/ \
        -I$(TOPDIR)/Platform/Inc/ \
        -I$(TOPDIR)/Platform/Sys/ \
        -I$(TOPDIR)/Src/App/wm_atcmd/ \
        -I$(TOPDIR)/Src/App/matrixssl/ \
        -I$(TOPDIR)/Src/App/matrixssl/core/ \
        -I$(TOPDIR)/Src/App/libupnp-1.6.19/ixml/inc/ \
        -I$(TOPDIR)/Src/App/libupnp-1.6.19/upnp/inc/ \
        -I$(TOPDIR)/Src/App/libupnp-1.6.19/ixml/include/ \
        -I$(TOPDIR)/Src/App/libupnp-1.6.19/threadutil/include/ \
        -I$(TOPDIR)/Src/App/libupnp-1.6.19/upnp/include/ \
        -I$(TOPDIR)/Src/App/gmediarender-0.0.6/ \
        -I$(TOPDIR)/Src/App/web/ \
        -I$(TOPDIR)/Src/App/OTA/ \
        -I$(TOPDIR)/Src/App/cloud/ \
        -I$(TOPDIR)/Src/App/cJSON/ \
        -I$(TOPDIR)/Src/App/ajtcl-15.04.00a/inc/ \
        -I$(TOPDIR)/Src/App/ajtcl-15.04.00a/target/winnermicro/ \
        -I$(TOPDIR)/Src/App/ajtcl-15.04.00a/external/sha2/ \
        -I$(TOPDIR)/Src/App/cJSON/ \
        -I$(TOPDIR)/Src/App/cloud/ \
        -I$(TOPDIR)/Src/App/oneshotconfig/ \
        -I$(TOPDIR)/Src/App/dhcpserver/ \
        -I$(TOPDIR)/Src/App/dnsserver/ \
        -I$(TOPDIR)/Src/App/ping/ \
        -I$(TOPDIR)/Src/App/iperf/ \
        -I$(TOPDIR)/Src/App/libcoap/include/ \
        -I$(TOPDIR)/Src/App/polarssl/include/ \
        -I$(TOPDIR)/Src/App/mDNS/mDNSCore/ \
        -I$(TOPDIR)/Src/App/mDNS/mDNSPosix/ \
        -I$(TOPDIR)/Src/App/mqtt/ \
        -I$(TOPDIR)/Src/App/easylogger/inc/ \
        -I$(TOPDIR)/Demo/ \
        -I$(TOPDIR)/Include/App/ \
        -I$(TOPDIR)/Include/Net/ \
        -I$(TOPDIR)/Include/WiFi/ \
        -I$(TOPDIR)/Include/OS/ \
        -I$(TOPDIR)/Include/Driver/ \
        -I$(TOPDIR)/Include/Platform/
#       -I$(TOPDIR)/Platform/Drivers/litepoint/
#       -I$(TOPDIR)/Src/App/demo/

ifeq ($(TOOL_GNU),1)
  INCLUDES += -I$(TOPDIR)/Platform/Boot/gcc/
else
  INCLUDES += -I$(TOOL_INCLUD_PATH)/RV31/INC/ \
          -I$(TOOL_INCLUD)/CMSIS/Include/
endif

  INCLUDES += -I$(TOPDIR)/Src/Network/api2.0.3/ \
          -I$(TOPDIR)/Src/Network/lwip2.0.3/ \
          -I$(TOPDIR)/Src/Network/lwip2.0.3/include/ \
          -I$(TOPDIR)/Src/Network/lwip2.0.3/include/arch/ \
          -I$(TOPDIR)/Src/Network/lwip2.0.3/include/lwip/ \
          -I$(TOPDIR)/Src/Network/lwip2.0.3/include/netif/ 

INCLUDES += -I$(TOPDIR)/Src/App/libwebsockets-2.1-stable/ \
        -I$(TOPDIR)/Src/App/httpclient/

INCLUDES += -I$(TOPDIR)/Src/App/lwm2m-wakaama/core/\
            -I$(TOPDIR)/Src/App/lwm2m-wakaama/core/er-coap-13/\
            -I$(TOPDIR)/Src/App/lwm2m-wakaama/examples/shared/\
            -I$(TOPDIR)/Src/App/lwm2m-wakaama/examples/

LIB_DIR := ${TOPDIR}/Lib/${LIBDIR}

_IGNORE_GCC_INC_DEF=-D_IN_ADDR_T_DECLARED -D__MACHINE_ENDIAN_H__ -D_TIMEVAL_DEFINED -D__INSIDE_CYGWIN_NET__

CFLAGS += $(_IGNORE_GCC_INC_DEF)
ARMCFLAGS += $(_IGNORE_GCC_INC_DEF)
ASMFLAGS += $(_IGNORE_GCC_INC_DEF)
#ARFLAGS += $(_IGNORE_GCC_INC_DEF)
ARMASMFLAGS += $(_IGNORE_GCC_INC_DEF)
ARMARFLAGS += $(_IGNORE_GCC_INC_DEF)

[rkompass]

Again: Nice and thanks. Should I find the directories that are no longer there? A question I had: Is the -Os compiler option present somewhere? has now it's answer.

Is the -D_xxx.. the unsetting of a compiler flag xxx?

[robert-hh]

-D sets a compiler flag just as #define does, -U clears it just like #undef.