search

robert-hh / micropython

MicroPython - a lean and efficient Python implementation for microcontrollers and constrained systems
https://micropython.org
Other
15 stars 4 forks source link

Compilation of w600 port III #20

Closed rkompass closed 7 months ago

rkompass commented 1 year ago

We continue here the discussions Compilation of w600 port and Compilation of w600 port II (closed now) of development/debugging the w60x port in MP branch w60x.

robert-hh commented 1 year ago

Do you know if a Micropython port of the w60x family is planned?

That port exists and was the starting point for my work. See https://github.com/WinnerMicro/micropython

rkompass commented 1 year ago

Oops, changed it to Do you know if a Micropython port of the w80x family is planned? That was what I wanted to ask. Hope wdyichen will answer the more detailed memory questions too.

Is it possible with the Segger debugging probe and the right configuration to check which code/process accesses the relevant areas of the 128K DRAM, esp. the first 64K of it?

Tbh I hope we can have 180 KB heap in the end. I read that MP has the functionality to add a second RAM area to the heap. This is what would be needed, once there is clarity that this area is unused.

I tried all the async web functionality of microdot yesterday and it is working now (wasn't before the socket update). So microdot is a nice option for a web server framework again now. Seems like this problem also was releated i.e. all select/poll functionality was not there. I wanted to ask you for your opinion on this issue first, then decided to check myself, which requires to learn more about select and poll. I will have to set up a minimal testing example for this. If you like and are faster then please go ahead.-), we could give a response to this issue now. I guess that also umqtt wasn't working before and will work now. Needs to be tested too.

Yesterday I got stuck with testing microdot SSL functionality. Will continue on that and the previous topic in perhaps 10 days (holiday).

rkompass commented 1 year ago

Hello Robert,

am I right in the assumption that we have now the updated GC? But still no SSL Context, as that is not commited to MP yet?

robert-hh commented 1 year ago

Yes, you're right. I also re-stated the Pin.PA_xx and Pin.PB_xx constants. At least one person used it. Can be dropped in a possible version 2.x, which will have breaking changes in MP as well.

rkompass commented 10 months ago

Hello Robert,

thank you for keeping this port up to date. I was quite distracted recently. Do you plan to incorporate the recent changes for 1.22-23, perhaps even tickless-ness?

I have a modification of the spi code almost ready which would save some RAM. Nice greetings, Raul

robert-hh commented 10 months ago

Hello Raul, whenever there are changes to the master branch I update the W60x branch and follow structural changes as far as reasonable. The branch in the github repository is updated now. What exactly do you mean with tickless-ness?

rkompass commented 10 months ago

You have it incorporated. I will try it out.

rkompass commented 7 months ago

In improving/combining/generalizing my test scripts I found another little flaw of the Timer:

The periodic software timer times fall (precisely) 1 ms short. The periodic hardware timer times are accurate.

The script:

from machine import Pin, Timer
from time import ticks_us, ticks_diff, sleep_ms
from sys import platform
import network

if platform in ['rp2', 'w600', 'mimxrt']:
    led = Pin("LED", Pin.OUT, value=1)
elif platform in ['esp8266', 'esp32']:
    led = Pin(2, Pin.OUT, value=1)

N_Runs = const(400)         # 10 seconds total
Period_ms = 25              # flicker of 20 Hz (recognizable)

sta_if = network.WLAN(network.STA_IF)
sta_if.active(False)

def tim_measure(t):
    global t_now, i_now
    t_now = ticks_us()
    i_now += 1
    led(not led())              # esp8266, esp32 don't have led.toggle()

t_then, t_now = ticks_us(), 0
i_then, i_now = 0, 0
Period_us = 1000 * Period_ms
tdif = []

tim = Timer(1)                 # -1..software timer, 1..hardware timer
tim.init(mode=Timer.PERIODIC, period = Period_ms, callback=tim_measure)

try:
    while i_then < N_Runs+3:
        if i_now > i_then:
            if i_now % 10 == 9:  # prevent longer USB inactivity (problematic on mimxrt)
                print('.', end='')
            if i_now > 3:
                tdif.append(ticks_diff(t_now, t_then)-Period_us)
            if (i_now == 3):
                t_total = t_now
            elif (i_now == N_Runs+3):
                t_total = ticks_diff(t_now, t_total)
            t_then = t_now
            i_then += 1
finally:
    tim.deinit()
    led(1)
    print('\nRuns:', i_now-3, '(of', i_now, ')')
    tdif.sort()
    t_total /= 1000.0
    print('Total period (ms):', t_total, 'deviation:', abs(t_total-N_Runs*Period_ms))
    print('Deviations (us) lowest:', tdif[0:10])
    print('Deviations (us) highest:', tdif[-11:-1])

Result with w600, software timer:

Runs: 400 (of 403 )
Total period (ms): 9600.002 deviation: 399.998
Deviations (us) lowest: [-1227, -1218, -1217, -1107, -1106, -1106, -1106, -1106, -1105, -1105]
Deviations (us) highest: [-898, -898, -898, -898, -897, -897, -896, -896, -782, -782]

with hardware timer (no Wifi):

Runs: 400 (of 403 )
Total period (ms): 10000.0 deviation: 0.001953125
Deviations (us) lowest: [-14, -12, -12, -12, -12, -10, -10, -10, -10, -10]
Deviations (us) highest: [11, 11, 11, 11, 11, 11, 11, 11, 11, 12]

Another minor issue to mention are the string representations of the timers:

After tim = Timer(-1) print(tim) gives Timer(20017d80; timerid=2000c2a8, period=25, auto_reload=1) on w600 Timer(3ffef4e4) on esp8266 Timer(mode=PERIODIC, period=25) on mimxrt.

The w600 is not bad in this comparison. But shouldn't the fact whether it is software or hardware timer be in the representation? Then it somehow should reflect the arguments of ìnit() shouldn't it? So mode=PERIODIC would be preferrable to auto_reload=1. I don't know about the two identities reported. What do you think?

robert-hh commented 7 months ago

I cannot confirm your observation for the soft timer. I get;

>>> import timertest
iDeviations (us) lowest: [-12, -11, -10, -6, -6, -6, -6, -6, -6, -5]
Deviations (us) highest: [7, 7, 7, 7, 7, 7, 7, 8, 8, 9]

And if I pulse a Pin in the callback, I see a spike every 10 ms at the logic analyzer. I had seen this event in the past for a short while, but that is gone. Tested with a Wemos W600-Pico.

About the print: The common rule is, that the printout shall match the constructor. So mode=PERIODIC is the right way. So anything else should be dropped. But that is in general still a long way to go for all ports and all devices.

robert-hh commented 7 months ago

When using the oscilloscope, which has a nice statistics feature, I get the following distribution for soft timer. Not that on some runs of timertest.py the values are in the range -10 top +10, on other runs they are in the range -200 to +100.

softtimer_10ms

The figure using hard timer is:

hardtimer_10ms

rkompass commented 7 months ago

Checked: I get it only with a certain period of 25 ms, the below script uses two periods: All_Periods_ms = [25, 10] and obviously only with 25ms the deviation is there. It seems you used 10ms period.?.?

I just now started to include different periods, sorry. I thought this deviation would be universal. Could you test with this script?

test_timer2.zip

Here are my results. In the script are results for other platforms. The RP2 has an issue there too.

........................................
Platform: w600, Wifi: active, Timer: soft, Period: 25 ms, 400 (of 403) runs. 
Total time (ms) Set value: 10000.000 Actual value: 9600.005 Deviation: -399.995
Single deviations (us) lowest: [-3594, -3587, -3584, -3582, -1731]
Single deviations (us) highest: [-400, -261, 1596, 1596, 1599]

........................................
Platform: w600, Wifi: active, Timer: hard, Period: 25 ms, 400 (of 403) runs. 
Total time (ms) Set value: 10000.000 Actual value: 10000.009 Deviation: 0.009
Single deviations (us) lowest: [-2602, -2597, -2577, -1607, -1603]
Single deviations (us) highest: [1619, 1623, 1630, 2625, 2625]

....................................................................................................
Platform: w600, Wifi: active, Timer: soft, Period: 10 ms, 1000 (of 1003) runs. 
Total time (ms) Set value: 10000.000 Actual value: 9999.999 Deviation: -0.001
Single deviations (us) lowest: [-2602, -2597, -2597, -2594, -2593]
Single deviations (us) highest: [2600, 2600, 2600, 2601, 2601]

....................................................................................................
Platform: w600, Wifi: active, Timer: hard, Period: 10 ms, 1000 (of 1003) runs. 
Total time (ms) Set value: 10000.000 Actual value: 9999.991 Deviation: -0.009
Single deviations (us) lowest: [-2320, -2312, -2308, -2308, -2305]
Single deviations (us) highest: [2329, 2329, 2330, 2330, 2330]

........................................
Platform: w600, Wifi: inactive, Timer: soft, Period: 25 ms, 400 (of 403) runs. 
Total time (ms) Set value: 10000.000 Actual value: 9600.000 Deviation: -400.000
Single deviations (us) lowest: [-1232, -1220, -1218, -1213, -1105]
Single deviations (us) highest: [-901, -898, -787, -785, -784]

........................................
Platform: w600, Wifi: inactive, Timer: hard, Period: 25 ms, 400 (of 403) runs. 
Total time (ms) Set value: 10000.000 Actual value: 10000.000 Deviation: 0.000
Single deviations (us) lowest: [-11, -10, -8, -8, -8]
Single deviations (us) highest: [7, 7, 7, 7, 10]

....................................................................................................
Platform: w600, Wifi: inactive, Timer: soft, Period: 10 ms, 1000 (of 1003) runs. 
Total time (ms) Set value: 10000.000 Actual value: 9999.996 Deviation: -0.004
Single deviations (us) lowest: [-16, -15, -13, -10, -9]
Single deviations (us) highest: [10, 11, 13, 13, 16]

....................................................................................................
Platform: w600, Wifi: inactive, Timer: hard, Period: 10 ms, 1000 (of 1003) runs. 
Total time (ms) Set value: 10000.000 Actual value: 9999.997 Deviation: -0.003
Single deviations (us) lowest: [-12, -12, -12, -12, -12]
Single deviations (us) highest: [11, 11, 12, 12, 13]
rkompass commented 7 months ago

Will have to try out the statistics feature !!!

rkompass commented 7 months ago

From the selection All_Periods_ms = [50, 33, 25, 20, 16, 10, 8] the systematic deviation (that the timing falls 1ms short) is there at 33 and 25 ms.

robert-hh commented 7 months ago

I just tried 25ms and I see it here with the oscilloscope as well. Interesting .... Some bug in the softtimer implementation of tls_os_timer.

robert-hh commented 7 months ago

b.t.w. I made a few other changes;

Edit: uart.irq() not not yet in the mainline of MicroPython. Even if Damien favored to have that feature, it may take a while for the PR to get processed. When looking at the W600 UART design, I noticed that it is pretty similar to the RP2 UART. But it has a register which I miss at the RP2. This register tells the number of items in the UART FIFO.

rkompass commented 7 months ago

b.t.w. I made a few other changes;

this is nice news.

I searched a bit for the tls_os_timer

found in in wm_osal_rtos.c:

void tls_os_timer_init(void)
{
    tls_sys_clk sysclk;

    tls_sys_clk_get(&sysclk);
    SysTick_Config(sysclk.cpuclk*UNIT_MHZ/HZ);
    NVIC_PriorityGroupConfig(NVIC_PriorityGroup_4);
}

and

 tls_os_status_t tls_os_timer_create(tls_os_timer_t **timer,
        TLS_OS_TIMER_CALLBACK callback,
        void *callback_arg,
        u32 period,
        bool repeat,
        u8 *name)
{
    tls_os_status_t os_status;

    if(0 == period)
        period = 1;
#if configUSE_TIMERS
    *timer = (xTIMER *)xTimerCreateExt( (signed char *)name, period, repeat, NULL, callback, callback_arg );
#endif
    if (*timer  != NULL)
        os_status = TLS_OS_SUCCESS;
    else
        os_status = TLS_OS_ERROR;

    return os_status;
}

and xTimerCreateExt in rtostimers.c:

xTimerHandle xTimerCreateExt( const signed char *pcTimerName, portTickType xTimerPeriodInTicks, unsigned portBASE_TYPE uxAutoReload, void *pvTimerID, tmrTIMER_CALLBACK pxCallbackFunction, void *callback_arg )
{
xTIMER *pxNewTimer;

    /* Allocate the timer structure. */
    if( xTimerPeriodInTicks == ( portTickType ) 0U )
    {
        pxNewTimer = NULL;
        configASSERT( ( xTimerPeriodInTicks > 0 ) );
    }
    else
    {
        pxNewTimer = ( xTIMER * ) pvPortMalloc( sizeof( xTIMER ) );
        if( pxNewTimer != NULL )
        {
            /* Ensure the infrastructure used by the timer service task has been
            created/initialised. */
            prvCheckForValidListAndQueue();

            /* Initialise the timer structure members using the function parameters. */
            pxNewTimer->pcTimerName = pcTimerName;
            pxNewTimer->xTimerPeriodInTicks = xTimerPeriodInTicks;
            pxNewTimer->uxAutoReload = uxAutoReload;
            pxNewTimer->pvTimerID = pvTimerID;
            pxNewTimer->pxCallbackFunction = pxCallbackFunction;
            pxNewTimer->callback_arg = callback_arg;    //add by dave
            vListInitialiseItem( &( pxNewTimer->xTimerListItem ) );

            traceTIMER_CREATE( pxNewTimer );
        }
        else
        {
            traceTIMER_CREATE_FAILED();
        }
    }

    return ( xTimerHandle ) pxNewTimer;
}

I don't see anything dubious. Could the error be in the MP part?

I should concentrate on the testing part: Will try to determine finer resolution of period range leading to the ms shortage.

Have you had a look at the testing results for the other ports? esp8266 and esp32 are quite precise, even in wifi mode. But I don't think the large timer ISR jitter with the w600 is a serious problem. But how about setting the timer priority higher than that of the WLan?

robert-hh commented 7 months ago

From the selection All_Periods_ms = [50, 33, 25, 20, 16, 10, 8] the systematic deviation (that the timing falls 1ms short) is there at 33 and 25 ms.

The answer is probably very simple. The W600 internal timer tick runs at 500Hz. So the resolution if the software timer can only be 2 ms, and the problem should appear at every odd period value.

Edit: And no, I do not care about the jitter.

rkompass commented 7 months ago

The answer is probably very simple. The W600 internal timer tick runs at 500Hz. So the resolution if the software timer can only be 2 ms, and the problem should appear at every odd period value.

Confirmed:

....................
Platform: w600, Wifi: inactive, Timer: soft, Period: 50 ms, 200 (of 203) runs. 
Total time (ms) Set value: 10000.000 Actual value: 9999.998 Deviation: -0.002
Single deviations (us) lowest: [-111, -110, -103, -100, -100]
Single deviations (us) highest: [100, 100, 101, 103, 108]

.................................
Platform: w600, Wifi: inactive, Timer: soft, Period: 33 ms, 333 (of 336) runs. 
Total time (ms) Set value: 10989.000 Actual value: 10656.003 Deviation: -332.997
Single deviations (us) lowest: [-1206, -1193, -1185, -1010, -1010]
Single deviations (us) highest: [-990, -988, -988, -814, -802]

........................................
Platform: w600, Wifi: inactive, Timer: soft, Period: 25 ms, 400 (of 403) runs. 
Total time (ms) Set value: 10000.000 Actual value: 9600.006 Deviation: -399.994
Single deviations (us) lowest: [-1115, -1106, -1106, -1106, -1106]
Single deviations (us) highest: [-894, -894, -894, -894, -893]

..........................................
Platform: w600, Wifi: inactive, Timer: soft, Period: 24 ms, 416 (of 419) runs. 
Total time (ms) Set value: 9984.000 Actual value: 9984.005 Deviation: 0.005
Single deviations (us) lowest: [-106, -99, -92, -91, -8]
Single deviations (us) highest: [9, 10, 91, 96, 102]

...........................................
Platform: w600, Wifi: inactive, Timer: soft, Period: 23 ms, 435 (of 438) runs. 
Total time (ms) Set value: 10005.000 Actual value: 9569.996 Deviation: -435.004
Single deviations (us) lowest: [-1181, -1099, -1099, -1099, -1099]
Single deviations (us) highest: [-902, -901, -901, -901, -900]

..................................................
Platform: w600, Wifi: inactive, Timer: soft, Period: 20 ms, 500 (of 503) runs. 
Total time (ms) Set value: 10000.000 Actual value: 9999.999 Deviation: -0.001
Single deviations (us) lowest: [-9, -9, -9, -9, -8]
Single deviations (us) highest: [8, 8, 8, 8, 8]

..............................................................
Platform: w600, Wifi: inactive, Timer: soft, Period: 16 ms, 625 (of 628) runs. 
Total time (ms) Set value: 10000.000 Actual value: 9999.997 Deviation: -0.003
Single deviations (us) lowest: [-108, -106, -106, -106, -106]
Single deviations (us) highest: [104, 104, 105, 105, 105]

....................................................................................................
Platform: w600, Wifi: inactive, Timer: soft, Period: 10 ms, 1000 (of 1003) runs. 
Total time (ms) Set value: 10000.000 Actual value: 9999.994 Deviation: -0.006
Single deviations (us) lowest: [-120, -118, -112, -112, -112]
Single deviations (us) highest: [108, 108, 112, 112, 121]

.............................................................................................................................
Platform: w600, Wifi: inactive, Timer: soft, Period: 8 ms, 1250 (of 1253) runs. 
Total time (ms) Set value: 10000.000 Actual value: 9999.995 Deviation: -0.005
Single deviations (us) lowest: [-111, -108, -108, -108, -108]
Single deviations (us) highest: [108, 109, 109, 110, 110]

Shouldn't the timer tick run at 1 kHz? How long is the timer tick processing? We have > 20_000 operations per ms.

robert-hh commented 7 months ago

You can try. It's set at line 73 in WM_SDK_W60X_G3.04.00/Src/OS/RTOS/source/tasks.c.

rkompass commented 7 months ago

Will do that soon.

The board manages to go through 13 single tests then it hangs. Repeatedly. Inspection of memory yields a memory loss, even with gc.collect():

....................
gc.mem_free(): 97248
Platform: w600, Wifi: active, Timer: soft, Period: 50 ms, 200 (of 203) runs. 
Total time (ms) Set value: 10000.000 Actual value: 10000.011 Deviation: 0.011
Single deviations (us) lowest: [-2587, -2585, -2585, -2585, -2582]
Single deviations (us) highest: [587, 2580, 2581, 2582, 2585]

.................................
gc.mem_free(): 94688
Platform: w600, Wifi: active, Timer: soft, Period: 33 ms, 333 (of 336) runs. 
Total time (ms) Set value: 10989.000 Actual value: 10655.996 Deviation: -333.004
Single deviations (us) lowest: [-1208, -1194, -1193, -1027, -1026]
Single deviations (us) highest: [-975, -975, -954, -819, -817]

........................................
gc.mem_free(): 92112
Platform: w600, Wifi: active, Timer: soft, Period: 25 ms, 400 (of 403) runs. 
Total time (ms) Set value: 10000.000 Actual value: 9600.009 Deviation: -399.991
Single deviations (us) lowest: [-3699, -3594, -3588, -3587, -3585]
Single deviations (us) highest: [1595, 1595, 1596, 1598, 1610]

..........................................
gc.mem_free(): 89552
Platform: w600, Wifi: active, Timer: soft, Period: 24 ms, 416 (of 419) runs. 
Total time (ms) Set value: 9984.000 Actual value: 9983.999 Deviation: -0.001
Single deviations (us) lowest: [-2597, -2591, -2590, -2590, -2588]
Single deviations (us) highest: [2599, 2599, 2600, 2600, 2600]

...........................................
gc.mem_free(): 86992
Platform: w600, Wifi: active, Timer: soft, Period: 23 ms, 435 (of 438) runs. 
Total time (ms) Set value: 10005.000 Actual value: 9570.001 Deviation: -434.999
Single deviations (us) lowest: [-3699, -3591, -3586, -3584, -3583]
Single deviations (us) highest: [1596, 1596, 1596, 1598, 1599]

..................................................
gc.mem_free(): 84432
Platform: w600, Wifi: active, Timer: soft, Period: 20 ms, 500 (of 503) runs. 
Total time (ms) Set value: 10000.000 Actual value: 9999.999 Deviation: -0.001
Single deviations (us) lowest: [-2699, -2691, -2585, -2585, -2584]
Single deviations (us) highest: [2594, 2594, 2599, 2600, 2699]

..............................................................
gc.mem_free(): 79824
Platform: w600, Wifi: active, Timer: soft, Period: 16 ms, 625 (of 628) runs. 
Total time (ms) Set value: 10000.000 Actual value: 10000.002 Deviation: 0.002
Single deviations (us) lowest: [-2704, -2589, -2586, -2584, -2584]
Single deviations (us) highest: [2596, 2596, 2597, 2599, 2601]

....................................................................................................
gc.mem_free(): 75216
Platform: w600, Wifi: active, Timer: soft, Period: 10 ms, 1000 (of 1003) runs. 
Total time (ms) Set value: 10000.000 Actual value: 9999.996 Deviation: -0.004
Single deviations (us) lowest: [-2597, -2591, -2588, -2585, -2584]
Single deviations (us) highest: [2601, 2601, 2602, 2603, 2606]

.............................................................................................................................
gc.mem_free(): 66512
Platform: w600, Wifi: active, Timer: soft, Period: 8 ms, 1250 (of 1253) runs. 
Total time (ms) Set value: 10000.000 Actual value: 10000.012 Deviation: 0.012
Single deviations (us) lowest: [-2713, -2606, -2602, -2602, -2602]
Single deviations (us) highest: [2601, 2601, 2601, 2602, 2607]

....................
gc.mem_free(): 64944
Platform: w600, Wifi: active, Timer: hard, Period: 50 ms, 200 (of 203) runs. 
Total time (ms) Set value: 10000.000 Actual value: 9999.988 Deviation: -0.012
Single deviations (us) lowest: [-1031, -1028, -307, -40, -39]
Single deviations (us) highest: [39, 39, 40, 308, 1032]

.................................
gc.mem_free(): 62384
Platform: w600, Wifi: active, Timer: hard, Period: 33 ms, 333 (of 336) runs. 
Total time (ms) Set value: 10989.000 Actual value: 10987.149 Deviation: -1.851
Single deviations (us) lowest: [-1838, -1734, -1728, -1725, -724]
Single deviations (us) highest: [746, 748, 749, 1742, 1751]

........................................
gc.mem_free(): 59824
Platform: w600, Wifi: active, Timer: hard, Period: 25 ms, 400 (of 403) runs. 
Total time (ms) Set value: 10000.000 Actual value: 10000.007 Deviation: 0.007
Single deviations (us) lowest: [-2338, -2332, -2332, -1527, -1454]
Single deviations (us) highest: [1359, 1483, 1547, 2356, 2356]

..........................................
gc.mem_free(): 57264
Platform: w600, Wifi: active, Timer: hard, Period: 24 ms, 416 (of 419) runs. 
Total time (ms) Set value: 9984.000 Actual value: 9983.960 Deviation: -0.040
Single deviations (us) lowest: [-1303, -1199, -1197, -1196, -1196]
Single deviations (us) highest: [1214, 1214, 1214, 1215, 1216]

Again 13 runs --- or --- memory loss down to ~ 50k then stall.

robert-hh commented 7 months ago

So the free memory decreases by constantly per run. The most plausible candidate to cause this is the tdif list, which is dynamically growing. You could try to replace it by some pre-allocated list or array. Still, if the device runs out of memory, it should not just hang, unless that happens in the callback.

robert-hh commented 7 months ago

Another thought: You call del(tim), and there is a method del() to remove the timer, but it may not be called. AFAIK, calling a destructor requires the object to be created with a finalizer. So creating the timer object in function machine_timer_make_new() of machine_timer.c (line 143?) requires to use m_new_obj_with_finaliser() instead of m_new_obj(). Otherwise more and more timer tasks are created, until eventually the stack overflows.

rkompass commented 7 months ago

Yes, agree: I guess tls_os_timer_delete() is not executed in all situations necessary.

I reduced the script to a minimal version: WLan is not active. Apart from losing memory in the running process here the stall of the script is demonstrated, which depends on the timers. From about 8/9 th run the timing is wrong. But the script passes.

In the version with tim object created anew each run after 8 or 13 runs the script stalls.

# We have a memory loss and the script runs only 8 or 13 single loops. Why?

from machine import Pin, Timer
import gc

led = Pin("LED", Pin.OUT, value=1)

def tim_measure(t):
    global i_now
    i_now += 1
    led(not led())

All_Runs =       [20, 33, 40, 41, 43, 50, 62, 100, 125]   # 10 seconds total
All_Periods_ms = [50, 33, 25, 24, 23, 20, 16,  10,   8]   # flicker of 20, 30, 40 Hz, .. >= 50 Hz (unrecognizable)

tim2 = [Timer(-1), Timer(1)]                    # !!!! alternative that seems to pass !!!!

for hard in (0, 1, 0, 1, 0, 1, 0, 1):
    for k, n_runs in enumerate(All_Runs):  
        i_now = 0
        tim = tim2[hard]                        # !!!! alternative that seems to pass !!!!
        # tim = Timer(1 if hard else -1)    # -1..software timer, 1..hardware timer
        tim.init(mode=Timer.PERIODIC, period=All_Periods_ms[k], callback=tim_measure)

        while i_now <= n_runs+3:
            pass
#        tim.deinit()             # <---  !!!! this has to be commented out in alternative that seems to pass !!!!
#         del(tim)   # with or without that: only 8 runs
        led(1)
        gc.collect; print('gc.mem_free():', gc.mem_free())

Seems creating a lot of different but thorough test scripts also is a noble task :-)

robert-hh commented 7 months ago

Looking into the code: when timer.init() is called on a timer that already exists, this timer is destroyed before a new timer is created. That way using timer.init() in the loop works.

rkompass commented 7 months ago

Still after the 9th run the overall time is wrong. Suddenly it's much faster.

robert-hh commented 7 months ago

Using mp_obj_malloc_with_finaliser() instead of mp__new_obj() did not cause the _dell__ method to be called. But I added the call to machine_timer_del(self_in); at the end of the deinit() method, and then your test continues to run, with a slow decrease in memory.

>>> import timertest2
gc.mem_free(): 99968
gc.mem_free(): 99920
gc.mem_free(): 99872
gc.mem_free(): 99824
gc.mem_free(): 99776
gc.mem_free(): 99728
gc.mem_free(): 99680
gc.mem_free(): 99632
gc.mem_free(): 99584
gc.mem_free(): 99504
gc.mem_free(): 99456
gc.mem_free(): 99408
gc.mem_free(): 99360
gc.mem_free(): 99312
gc.mem_free(): 99264
gc.mem_free(): 99216
gc.mem_free(): 99168
gc.mem_free(): 99120
gc.mem_free(): 99040
gc.mem_free(): 98992
gc.mem_free(): 98944
gc.mem_free(): 98896
gc.mem_free(): 98848
gc.mem_free(): 98800
gc.mem_free(): 98752
gc.mem_free(): 98704
gc.mem_free(): 98656
gc.mem_free(): 98576
gc.mem_free(): 98528
gc.mem_free(): 98480
gc.mem_free(): 98432
gc.mem_free(): 98384
gc.mem_free(): 98336
gc.mem_free(): 98288
gc.mem_free(): 98240
gc.mem_free(): 98192
gc.mem_free(): 98112
gc.mem_free(): 98064
gc.mem_free(): 98016
gc.mem_free(): 97968
gc.mem_free(): 97920
gc.mem_free(): 97872
gc.mem_free(): 97824
gc.mem_free(): 97776
gc.mem_free(): 97728
gc.mem_free(): 97648
gc.mem_free(): 97600
gc.mem_free(): 97552
gc.mem_free(): 97504
gc.mem_free(): 97456
gc.mem_free(): 97408
gc.mem_free(): 97360
gc.mem_free(): 97312
gc.mem_free(): 97264
gc.mem_free(): 97184
gc.mem_free(): 97136
gc.mem_free(): 97088
gc.mem_free(): 97040
gc.mem_free(): 96992
gc.mem_free(): 96944
gc.mem_free(): 96896
gc.mem_free(): 96848
gc.mem_free(): 96800
gc.mem_free(): 96720
gc.mem_free(): 96672
gc.mem_free(): 96624
gc.mem_free(): 96576
gc.mem_free(): 96528
gc.mem_free(): 96480
gc.mem_free(): 96432
gc.mem_free(): 96384
gc.mem_free(): 96336
>>>
rkompass commented 7 months ago

I'm ready to clone...

robert-hh commented 7 months ago

The branch is updated. For now with another commit. I'll squash that later with the previous timer fix commit. I did not check what caused the test to speed up when the timer was re-used.

rkompass commented 7 months ago

I assume there is a free for the new machine_timer_obj_t *self = mp_obj_malloc(machine_timer_obj_t, &machine_timer_type);.

Will test it now.

rkompass commented 7 months ago

I set const unsigned int HZ = 1000; in the SDK, as you recommended.

And found that I had forgotten to add the () to gc.collect(); Arrgh--

Everything is nice now. The script runs through all 28 single tests/timer creations. Biggest total deviation is 0.034 aka 34uS. Biggest single deviations are ~2.7ms (the 1 ms better, of course, than before). The free memory oscillates between 94 and 101k.

I think we could try (if there is a straightforward way) to set the hard timer to have a higher priority than Wifi, because: What else is it for, if not for acting like an ordinary hard interrupt? At the moment it behaves quite similar to the soft timer. With Wifi the jitter is only slightly reduced if the timer is hard. And if the programmer does not want to interfere with Wlan at all, then of course he uses soft timers.

But again: Everything is nice now. If you are not interested I'm fine too. The w600 is just a little bit worse than the esp8266 and esp32 wrt timers. Which it doesn't have to.?.?..

robert-hh commented 7 months ago

mp_obj_malloc() and mp_obj_malloc_with_finaliser() replaced m_new_obj() + self->base.type = &<object>_type; a while ago. gc_collect() and finally gc_sweep_all() during soft_reset() will take care of freeing the object. I could make a general replace in the w600 port. But it is just a style change.

robert-hh commented 7 months ago

I think we could try (if there is a straightforward way) to set the hard timer to have a higher priority than Wifi.

I cannot recall that the hard timer runs in a task. Looking into the code it looks like the hard timer create just sets up a hardware and the callback is called by a hard ISR handler. See WM_SDK_W60X_G3.04.00/Platform/Drivers/timer/wm_timer.c.

rkompass commented 7 months ago

Ok. nice to have a look at: Timer 0 is used for delays. 1..5 are available, it seems. Should be kept for the docs perhaps. And yes, it writes hardware registers. I did not see a IRQ priority.

If there were an IRQ priority at all, then in the ARM interrupt registers.

The NVIC supports software-assigned priority levels. You can assign a priority level from 0 to 255 to an interrupt by writing to the eight-bit PRI_N field in an Interrupt Priority Register,

Where are those written? The hardware timers 1..5 should have different interrupt priorities.

robert-hh commented 7 months ago

Where are those written? The hardware timers 1..5 should have different interrupt priorities.

I do not know. There's a lot of configuration in WM_SDK_W60X_G3.04.00/Platform/Inc/misc.h and WM_SDK_W60X_G3.04.00/Platform/Boot/gcc/misc.c.

rkompass commented 7 months ago

I didn't find anything, after thoroughly searching. I'm glad you found the soft timer priority. Fine. Done.

robert-hh commented 7 months ago

b.t.w.: With gc.collect() properly called and the timer object being created with finaliser, the del(tim) works as it should.

rkompass commented 7 months ago

del(tim) and tim.deinit() both should work without a collection, though. Which they do now, i suppose. One problem before (in the reduced test script: timer running too fast) was obviously that the hard timer was not setup correctly, and this makes sense now: as it was the same timer again and again because of the one hardware slot and this was not properly torn down. It was o.k after your changes.

robert-hh commented 7 months ago

del(tim) does not cause the destructor del() to be called immediately. That will happen with the next garbage collection. But in calling tim.init() the previous timer instance at that object were cleared,. I think the speedup effect when omitting tim.deinit() is caused by the fact, that the repective other timer still kept running, causing the callback to be called.

I changed the code of init() to call del() instead of doubling the same code. But that is just a style change.

rkompass commented 7 months ago

Thank you for the feedback.

I improved my testing program a bit more and observed some differences between platforms that might be ameliorated without much effort:

  1. A pin name 'LED' should be introduced in esp8266, esp32, nrf52(currently only 'LED1') and pyboard. This should be independent of the LED class of pyboard. Currently I have:
dled = {'pyboard':'B4','rp2':'LED','w600':'LED','mimxrt':'LED','rp2':'LED','esp8266':2,'esp32':2,'nrf52':'LED1'}
led = Pin(dled[platform], Pin.OUT, value=1)
  1. Pin.toggle() is missing in pyboard, esp8266 and esp32.

A bit more demanding would be:

  1. The rp2 currently has no hard timer. But the PWM hardware has all prerequisites for a hard timer: From the datasheet:

Each PWM slice is equipped with the following: • 16-bit counter • 8.4 fractional clock divider • Two independent output channels, duty cycle from 0% to 100% inclusive • Dual slope and trailing edge modulation • Edge-sensitive input mode for frequency measurement • Level-sensitive input mode for duty cycle measurement • Configurable counter wrap value ◦ Wrap and level registers are double buffered and can be changed race-free while PWM is running • Interrupt request and DMA request on counter wrap

This could be implemented by much the same as the code for pwm, only the pins should not be set to PWM alternative function mode.

  1. The pyboards timer is called much different. There should be a compatible Timer class in machine.
  2. The pyboard Timer has no ONESHOT mode although the hardware is able to do that, iirc.
  3. The pyboard has no soft timer.
  4. I observed that on the pyboard there the pyb.Timer (which is hardware) has a .counter() method. I suppose that the other hard timers also could use such a .counter() method.

Shall I raise an issue for that or would you just like to go starting to work on these with me entertaining you with feedback and tests? What do you think?

Motivation: A. Much more testing/benchmarking is needed, and to do that with moderate effort, given all the different platforms, we need acceptable uniform interfaces to the MP capabilities. B. This will not interfere with the plans to get a MP 3.0 but rather help and make developments easier. C. The benchmarking will be a motivation for finding good implementations.

robert-hh commented 7 months ago

Quite a few comments. About LED: Pin names like LED are defined at board level. Can be done, but it is quite some work. And if, it should match an existing LED of the board.

About pyboard: The pyboard hardware layer is known to be incompatible. AFAIK it is planned for a major rehaul with v2.x. So I do not expect a change there.

About hard vs. soft timer: I had the impression that Damien prefers the soft timer approach, which will be consistent across the ports. I had hard timers at the mimxrt port initially, and Damien was happy when I changed that to soft timer, keeping the hard timers for tasks with more strict timing demands.

robert-hh commented 7 months ago

About counter: There are long outstanding PRs for counter and quadrature decoders for ESP32 and MIMXRT. Jimmo told that he is working on a common approach but no output since months. Once that is done, I can add that to other ports like SAMD .....

robert-hh commented 7 months ago

Raising issues about these topics os not required. It's known. A useful support would be a machine.timer class for the stm32 port using soft timers. That should be easy and work on all boards. Maybe it exists.

robert-hh commented 7 months ago

MicroPython is really missing some kind of product management. Damien seems mostly interested in the MP core and Pyboard as reference platform. And since MP as company unlike Adafruit or Arduino is not really a board seller, he is less interested in having many ports. That is only work and makes trouble. Jimmo and Angus also do not put effort in a consistent product development, or they are not allowed to.

robert-hh commented 7 months ago

Back to the w600 port. I rearranged the commit a little bit, combining commits with a similar topic. Just pulling form that branch should may cause trouble, but I'm sure you can handle it.

rkompass commented 7 months ago

product management.

I really have the same impression, although from somewhat more far away.

About hard vs. soft timer: I had the impression that Damien prefers the soft timer approach, which will be consistent across the ports. I had hard timers at the mimxrt port initially, and Damien was happy when I changed that to soft timer, keeping the hard timers for tasks with more strict timing demands.

Now with pyboard being Georges most original product: That has only hard timers. Hard and soft should complement each other.

keeping the hard timers for tasks with more strict timing demands.

Should be also allowed for the competent user.

The pyboard exhibits these results:

gc.mem_free(): 94240
Platform: pyboard (168.0 MHz), Wifi: not present, Timer: hard, Period: 16 ms, 625 (of 628) runs. 
Total time (ms) Set value: 10000.000 Actual value: 10000.000 Deviation: 0.000
Single deviations (us) lowest: [0, 0, 0, 0, 0]
Single deviations (us) highest: [0, 0, 0, 0, 0]

same for other time intervals. We see the perfect alignment of the callbacks. That may be needed, e.g. in sampling.

The rp2 should allow for the same precision, as it has no underlying FreeRTOS.

What about Jimmo? I miss his contributions. With the Bluetooth discussions recently he seemed to be absent, given the aioble is his product.

robert-hh commented 7 months ago

Pyboard resp STM32 was Damien's first port. Over time, it grew wild and is hardly manageable any more. Too many MCU families. The next port by Damien is RP2 which shows the improvements and lessons learned in the time between pyboard and RP2. For the ports I extended (MIMXRT and SAMD and W600) I took RP2 as the template.

Some more history: The MIMXRT port was started by Philipp Ebensberger, and we continued it together for a while. Renesas used the STM32 branch, which I consider as a bad decision. They should have used the RP2 port as reference. but maybe it did not exist when they started. The W600 port seems to follow more the CC3200 port, which was made by Daniel Campora, Daniel then founded Pycom, which is closed now. The ESP8266 port was made by Paul Sokolovsky, who later left the team. I forgot who started the ESP32 port initially, but it had many contributors. The NRF port is as well quite old, almost as old as MicroPython itself. And it is quite different and requires a major rework.

rkompass commented 7 months ago

I think we should have:

Signal should be an option to the Pin class. How to integrate the DMA is a question.

robert-hh commented 7 months ago

Pin with access to different flank info, if available in hardware (those bits that elicit interrupts are usually somewhere, even if interrupts are not allowed). The pins, of course should allow for interrupts, hard and soft. If interrupt modes are active, then the ISR should be able to get a time-stamp of the eliciting impulse (done by MP within a few nanoseconds), perhaps as an option. This is like a rudimentary registration of events."

The event that triggered a PIN irq is already available to the IRQ handler, trough .irq().flags(). A timestamp would be easy to implement the same way, but not all applications need it and it would still cost time.

Timer, with hard and soft modes, if hardware/RTOS allow for it, ONESHOT and PERIODIC. Possibly in addition to the callback a trigger option to start other Timer/TimePulse/Counter/Pin-change/ADC/DAC conversions.

Not all ports & boards can support hard timers. And overloading it is not helpful. The various activities you mention are supposed to be done by the callback. Timed ADC/DAC should be a feature of the respective ADC/DAC class.

TimePulse, blocking and non-blocking for timing pulses from level duration or different flanks/pins. Hard and soft again, if possible.

Time/pulse can be left to ports which can support that, like RP2. Cannot be a general feature.

Counter, also with hard and soft mode, if hardware/RTOS allow for it. Different modes of counting, including an QEncoder (quadrature encoding) mode.

Additional Counters and QEncoder are only reasonable as hard devices. Otherwise it can implemented with Pin irq.

PWM, with some options for stepper motors, or an extra Stepper class

Some port have already some support for stepper motors. Other ports simply cannot support it, missing features like invert or dead time insertion. What I'm missing in PWM is to output a fixed number of cycle and then stop/callback.

UART, SPI, I2C, I2S, OneWire, of course, which seem quite nice now, but also could do with a nonblocking mode, possibly in connection with DMA.

There is work ongoing to support it. The problem e.g. with sending UART, SPI, I2C non-blocking is, that it either requires buffering the data in the driver or a signalling mechanism (e.g. callback) that the transfer has finished, avoiding the the data buffer is cleared by python before the transfer has finished. Otherwise there will be nasty error situations. Whether it's DMA or Interrupt does not matter for sending, except for I2S, where it's already DMA. Some UART implementations are already non-blocking, if the data size is smaller than txbuf.

ADC / DAC

Not all ports can support both. The MIMXRT chips for instance do not have DAC hardware. The W600 has neither ADC or DAC.

PID, for control, with interfaces to counter/encoder and PWM/Stepper ADC/DAC

Make the basic mechanisms fucntional and leave the rest to Pyhton code.

WDT

WDT Is implemented, where the hardware supports it

Signal should be an option to the Pin class.

It exists and is just a compile option in mpconfigport.h.

rkompass commented 7 months ago

Thanks for the feedback. That was just an idea of an ideal world. Of course, many of such features could not be implementend on all ports. OTOH, for example quadrature encoding is implemented in circuitpython on the software level, in C. which is fine, as it is much faster than an implementation in MP. The user can activate it and saves a lot of brain effort, to make his MP solution nearly as effective as a solution in C, that he would do e.e. in Arduino. An alternative would be to make viper or native mode independent of the MP setup, which costs a lot of reaction time in the ISR, as you know.

Defining all these features in detail not automatically should imply a duty to implement them, but could be a point that realistic implementations would move towards. This way the different implementations (of which there are many now and it will become much more) would automatically converge, not diverge. Saving development time in future attempts to unify the features.

robert-hh commented 7 months ago

I expect that once there is an "official" API for QEncoder, it can be implemented as a soft module for MCU without hardware support.