RobertK66 / obc_1769_core

Implements hardware abstractions and Layer2(3) modules for usage of the OBC hardware in cubesat projects
GNU General Public License v3.0
1 stars 1 forks source link

Main Loop Monitoring - Debug help #17

Open RobertK66 opened 2 years ago

RobertK66 commented 2 years ago

Meassure runtime per module. include 'current running' module to monitor structure make and persist events absout treshholds violations

make 'module enabled/disabled' possible. Switch on/off (with 'reInit' or without !? switch off with DeInit !?)

kodsurf commented 1 year ago

I added header file that stores global variable

LAST_STARTED_MODULE

Included that header to all files.

For every function writen I MANUALLY set up unique index corresponding to this function.

image

When any function withing the module starts - it sets LAST_STARTED_MODULE to corresponding index

example : index 11 means that it is l4_thruster_main() 1111 -is void thr_execute_sequence(int procedure_id) function within l4_thruster.c

This way when HARD FAULT OCCURES global variable with index of last started function will be saved and visible for debug


EXAMPLE :

The goal of implementing this watchdog is to investigate what causes failure when executing sequences in feature/thruster-fire branch (ISSUE-25) https://github.com/RobertK66/obc_1769_core/issues/25

Current issue : When command "8 0" is sent. THR_HARDCODED_SEQUENCE[0] is executed (test sequence of function pointers )

After "stage 6" SOMETIMES failure occures.

image image image

https://github.com/RobertK66/obc_1769_core/commit/5a38f4553de015b407adf8d3ed08f7f554ad0c69

kodsurf commented 1 year ago

I discovered that failure is caused by thr_execute_sequence() function within l4_thruster.c

Thus I added additional "breakpoints" to find out where exactly error is occuring

image

I launched sequence with "8 0" command again and waited for failure.

First attempt resulted in no error occuring (sometimes work, sometimes not) image

After second launch of sequence

image

With an error after image

thus

image

Error occured by executing function at specific pointer (probably wrong pointer ?)

@RobertK66 Please look through this procces of investigation

kodsurf commented 1 year ago

I were able to SUCCESFULLY execute "8 0" ANY NUMBER OF TIMES without an error.

Error occures ONLY AFTER another sequence was executed

for example "8 1" - after that "8 0" will trigger an error at "stage 6"

image image

So it failed when trying to execute a function of THR_HARDCODED_SEQUENCE[proccedure_id].sequence[8].function

In ORIGINAL sequence this is thr_wait() function. And its defination and argv are absolutely correct image

BUT !!!!!!!!!!!!!!!!!!!!!!!!

I noticed that when I programmed sequences in l4_init() !!!!!!!!!!!!!!!!

I used temporal variable image

NOW IMPORTANT BUG !!! - Length of original "0" th sequence is 12

length of sequence "0" is 7.

Execution of "0"th sequence after execution of "2" sequence causes failure due to wrong function pointer !!!!

Why is that ?!?!?!

kodsurf commented 1 year ago

image

I think problem is in this defination ...

idea behind thr_hardcoded_sequences_t THR_HARDCODED_SEQUENCES[5] is to store ARRAY of a type thr_sequences_t

image

is meant to be AN ARRAY of function pointers and arguments

and thr_sequence_t is a typedef of single element in this array

image probably here is an error in defination ????


I want to save array of structures as an element of another structure ! How can I do it ? image

RobertK66 commented 1 year ago

Ok, lets keep this as a 'feature issue' to implement the 'current running module for everybody in a basic mainloop implementation. I also plan to include this and the generic string / raw event somewhere in the (ado?) module base with this.....

RobertK66 commented 9 months ago

I think I had one instance of 'Hardfault' triggered by the current calculate Runtime method in the mainloop -> this is desasterous and I really think now to remove this 'complicated code' from mainloop .... if kept, how to test for all possible timing and math problems here ..... ???