Update momo shell with better help features and ability to add packages

[timburke]

This PR adds a new syslog implementation that does not require sprintf'ing things, allows the logging of typed data and in the future can facilitate hardware testing by allowing the logging of typed data into a special 'test' log that can be inspected to see if hardware tests were successful for or not. The implementation of syslog and usage is detailed here.

The reason why I wanted to change it was so that we can get better data on running momo systems including the actual RAM variables for debugging and development purposes. This required some fairly extensive additional work to automatically match python types with firmware types automatically and is used in syslog as well as read_ram to read controller ram variables and in the future can be used for symbolic debugging in the simulators as well.

The main disadvantage of the new systemlog vs the old one is that you now need a log definition file in order to translate the log messages into english text and match types with the logged parameters. If you don't have a log definition handy for the module, the syslog will do the best it can and always show you as much as it can with the information that it has. There is enough metadata stored with each logged parameter to display it as either an integer, ascii string or binary blob without a log definition but with a log definition file you can give each parameter a name and associate rich types with each binary blob that can have complicated processing and display routines.

Changes from Old System Log

• any amount of data can be logged with a message
• the message string is not embedded in the log entry but instead a unique 32-bit identifier is stored that is used to look up the message and learn how to parse and display the typed data (as detailed in 92-SystemLog.md linked above)
• modules that want to use the syslog need to define the format of the log entries that they use in a log_definition.ldf file according to the above linked documentation. This has been done for mainboard and shared/pic24 code.
• the systemlog can be inspected and displayed using the momo tool

  Changes to the MoMo Tool

• Documentation for the MoMo tool is now here
• typedargs has been updated to allow for dynamically adding types from external packages. This is needed so that firmware modules can define python types for the variables that they wish to log or otherwise expose to the user.
• MoMo has an optional config file that allows the running of initialization commands for certain contexts to repetitively inject types and log definitions for commonly used modules. This is described in the above documentation for the momo tool.

  Additional Changes

• Scons now tracks pic24 include statements to add dependencies on header files so that modifying a header triggers a rebuild.
• MoMo tool help system is fixed (it could throw exceptions sometimes before)
• MoMo tool compatibility with windows is fixed to have working tab completion
• Unit tests are now run for pymomo using the nose testing framework (provision.sh and build_all.sh have been updated to run these tests and import the required packages)

Additional work that can be done on these changes (for a future PR) is detailed in Issue #152.

[timburke]

Hold off on this one for a sec. I need to resolve a build dependency issue.

[timburke]

Alright this should be good to go now. I had to update the log definition file compiler to pull in any special python types that the log might reference and added an option to run momo without using its config file so that builds can be repeatable without any dependence on the machine's environment (i.e. the contents of the momo_config.txt file).

[amcgee]

I have a few thoughts about how we could make this even more awesome, but it's pretty awesome as-is!

Merging, will open an issue for future enhancements.

[timburke]

Looking forward to it. I love the awesomeness.

On 1/7/15 3:58 PM, Austin McGee wrote:

I have a few thoughts about how we could make this even more awesome, but it's pretty awesome as-is!

Merging, will open an issue for future enhancements.

— Reply to this email directly or view it on GitHub https://github.com/WellDone/MoMo-Firmware/pull/153#issuecomment-69114807.