search

mocleiri / tensorflow-micropython-examples

A custom micropython firmware integrating tensorflow lite for microcontrollers and ulab to implement the tensorflow micro examples.
MIT License
170 stars 79 forks source link

Guru Meditation Error: Core 1 panic'ed (InstrFetchProhibited). Exception was unhandled. #84

Closed MATTYGILO closed 2 years ago

MATTYGILO commented 2 years ago

I'm running on the optimised neural network firmware

However after doing 8 executions on my 9th this occurs, I'm uncertain of what the error is.

Guru Meditation Error: Core  1 panic'ed (InstrFetchProhibited). Exception was unhandled.

Core  1 register dump:
PC      : 0x00000000  PS      : 0x00060c30  A0      : 0x8200cbf2  A1      : 0x3fce4500  
A2      : 0x3fcf47ec  A3      : 0x3fce3c44  A4      : 0x0000000b  A5      : 0x00000000  
A6      : 0x00000000  A7      : 0x3db9a7f0  A8      : 0x8200c119  A9      : 0x3fce44e0  
A10     : 0x3fcf478c  A11     : 0x3db9a7f0  A12     : 0x00000000  A13     : 0x42303448  
A14     : 0xffffff80  A15     : 0x3db3f428  SAR     : 0x0000001d  EXCCAUSE: 0x00000014  
EXCVADDR: 0x00000000  LBEG    : 0x40056f5c  LEND    : 0x40056f72  LCOUNT  : 0xffffffff  

Backtrace:0xfffffffd:0x3fce45000x4200cbef:0x3fce4530 0x42001b2e:0x3fce4550 0x4200195a:0x3fce4570 0x420655c9:0x3fce4590 0x4206d87d:0x3fce45b0 0x4037886d:0x3fce45d0 0x420656e0:0x3fce4670 0x4206d87d:0x3fce46e0 0x4037886d:0x3fce4700 0x420656e0:0x3fce47a0 0x4206d7bd:0x3fce47d0 0x403787e1:0x3fce47f0 0x420656e0:0x3fce4890 0x4206d7bd:0x3fce4900 0x403787e1:0x3fce4920 0x420656e0:0x3fce49c0 0x4206d7bd:0x3fce4a30 0x403787e1:0x3fce4a50 0x420656e0:0x3fce4af0 0x4206d7bd:0x3fce4b40 0x403787e1:0x3fce4b60 0x420656e0:0x3fce4c00 0x4206d7e5:0x3fce4c50 0x420ae45c:0x3fce4c70 0x420ae6ac:0x3fce4d00 0x42089564:0x3fce4d40 

ELF file SHA256: 74adcc789bc7d279

All help would be much appreciated.

Thanks

MATTYGILO commented 2 years ago

From the docs https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/fatal-errors.html

InstrFetchProhibited

This CPU exception indicates that the CPU could not read an instruction because the address of the instruction does not belong to a valid region in instruction RAM or ROM.

Usually, this means an attempt to call a function pointer, which does not point to valid code. PC (Program Counter) register can be used as an indicator: it will be zero or will contain a garbage value (not 0x4xxxxxxx).

MATTYGILO commented 2 years ago

Decoded the backtrace

0x00000bac: ?? ??:0
0x42001b2e: libtf_interpreter_invoke at ??:?
0x4200195a: interpreter_invoke at tensorflow-microlite.c:?
0x420655c9: fun_builtin_1_call at /home/runner/work/tensorflow-micropython-examples/tensorflow-micropython-examples/micropython/py/objfun.c:71
0x4206d87d: mp_call_function_n_kw at /home/runner/work/tensorflow-micropython-examples/tensorflow-micropython-examples/micropython/py/runtime.c:664
 (inlined by) mp_call_method_n_kw at /home/runner/work/tensorflow-micropython-examples/tensorflow-micropython-examples/micropython/py/runtime.c:680
0x4037886d: mp_execute_bytecode at /home/runner/work/tensorflow-micropython-examples/tensorflow-micropython-examples/micropython/py/vm.c:1008
0x420656e0: fun_bc_call at /home/runner/work/tensorflow-micropython-examples/tensorflow-micropython-examples/micropython/py/objfun.c:289
0x4206d87d: mp_call_function_n_kw at /home/runner/work/tensorflow-micropython-examples/tensorflow-micropython-examples/micropython/py/runtime.c:664
 (inlined by) mp_call_method_n_kw at /home/runner/work/tensorflow-micropython-examples/tensorflow-micropython-examples/micropython/py/runtime.c:680
0x4037886d: mp_execute_bytecode at /home/runner/work/tensorflow-micropython-examples/tensorflow-micropython-examples/micropython/py/vm.c:1008
0x420656e0: fun_bc_call at /home/runner/work/tensorflow-micropython-examples/tensorflow-micropython-examples/micropython/py/objfun.c:289
0x4206d7bd: mp_call_function_n_kw at /home/runner/work/tensorflow-micropython-examples/tensorflow-micropython-examples/micropython/py/runtime.c:664
0x403787e1: mp_execute_bytecode at /home/runner/work/tensorflow-micropython-examples/tensorflow-micropython-examples/micropython/py/vm.c:923
0x420656e0: fun_bc_call at /home/runner/work/tensorflow-micropython-examples/tensorflow-micropython-examples/micropython/py/objfun.c:289
0x4206d7bd: mp_call_function_n_kw at /home/runner/work/tensorflow-micropython-examples/tensorflow-micropython-examples/micropython/py/runtime.c:664
0x403787e1: mp_execute_bytecode at /home/runner/work/tensorflow-micropython-examples/tensorflow-micropython-examples/micropython/py/vm.c:923
0x420656e0: fun_bc_call at /home/runner/work/tensorflow-micropython-examples/tensorflow-micropython-examples/micropython/py/objfun.c:289
0x4206d7bd: mp_call_function_n_kw at /home/runner/work/tensorflow-micropython-examples/tensorflow-micropython-examples/micropython/py/runtime.c:664
0x403787e1: mp_execute_bytecode at /home/runner/work/tensorflow-micropython-examples/tensorflow-micropython-examples/micropython/py/vm.c:923
0x420656e0: fun_bc_call at /home/runner/work/tensorflow-micropython-examples/tensorflow-micropython-examples/micropython/py/objfun.c:289
0x4206d7bd: mp_call_function_n_kw at /home/runner/work/tensorflow-micropython-examples/tensorflow-micropython-examples/micropython/py/runtime.c:664
0x403787e1: mp_execute_bytecode at /home/runner/work/tensorflow-micropython-examples/tensorflow-micropython-examples/micropython/py/vm.c:923
0x420656e0: fun_bc_call at /home/runner/work/tensorflow-micropython-examples/tensorflow-micropython-examples/micropython/py/objfun.c:289
0x4206d7e5: mp_call_function_n_kw at /home/runner/work/tensorflow-micropython-examples/tensorflow-micropython-examples/micropython/py/runtime.c:664
 (inlined by) mp_call_function_0 at /home/runner/work/tensorflow-micropython-examples/tensorflow-micropython-examples/micropython/py/runtime.c:638
0x420ae45c: parse_compile_execute at /home/runner/work/tensorflow-micropython-examples/tensorflow-micropython-examples/micropython/shared/runtime/pyexec.c:116
0x420ae6ac: pyexec_friendly_repl at /home/runner/work/tensorflow-micropython-examples/tensorflow-micropython-examples/micropython/shared/runtime/pyexec.c:662
0x42089564: mp_task at /home/runner/work/tensorflow-micropython-examples/tensorflow-micropython-examples/micropython/ports/esp32/main.c:175
MATTYGILO commented 2 years ago

This is a memory error, what I'd suggest is cutting your code to the bare basics.

As bare as you can go and debug as much as possible, use mem_info to find block sizes etc

MATTYGILO commented 2 years ago

So after a tonne of debugging, I have worked out that you should reload the whole interpreter before doing a large batch of detections. Take this example.

Load interpreter (Use a global memoryview)
while True:
    Record
    ReLoad interpreter (Use the global memoryview)
    Identify
    Tons of random other functions
MATTYGILO commented 2 years ago

It would be great if you can assign a memory view for the arena instead of the passing in a arena size