Adds support for Monitor a Port to observe if it is blocked

[joyeshmishra]

Issue Number:

Objective of pull request:

Summary of Changes:

+        sender.run(condition=RunSteps(2),
+                   run_cfg=Loihi1SimCfg(),
+                   compile_config={"long_event_timeout": 2.0,
+                                   "short_event_timeout": 1.0})

1. Configs can be sent via compile_configs. Defaults are 600.0s and 60.0s.

2. Compiler now adds a WatchdogManagerBuilder to executable. This will be built by the runtime.

3. Runtime invokes WatchdogManagerBuilder.build(). Depending on the log level, we either get a WatchdogManager or a NoOPWatchdogManager

4. When runtime initializes, we start the WatchdogManager first before any other initialization

4. Upon start of WatchdogManager, it first allocates the Multiprocessing Manager which will create all proxy objects and take care of Serde/Pickling as well as Synchronization. 2 queues for long event monitoring and short event monitoring are created respectively. 2 OS Processes namely the EventMonitors are started each monitoring one of the queues.

5. Event Monitor essentially monitors the queue and spins up a thread whenever it receives an Event to monitor. The Event comes as (Event, ChannelName+PortName, Function). For example (event, LIF.a_in, "send"). The thread will wait for event to be set with a timeout. If the timeout occurs, it will print msg. Once the event is set, the thread will finish monitoring this event and join back.

6. PyPyChannel will essentially get 4 watchdogs. One for src_send, src_join, dst_recv and dst_join. The send/recv/join methods essentially will enter the watchdogs (implemented as context managers).

7. Default behaviour of NoOPWatchdogManager is to use a NoOPWatchdog which has empty __enter__ and __exit__ methods. In Python, it is nearly impossible to not execute without breaking interface semantics. So this is close to what NoOP would be as per me. (I wanted to call this Lamedog, but that sounded silly :))

8. No changes to lava-loihi is expected (on PyC or CPy channels via channel broker) as they use the NoOPWatchdog.

Profiling will happen next. Most of the code is in watchdog.py. Rest are mostly plumbing in and management of data structures.

Pull request checklist

Your PR fulfills the following requirements:

• [ ] Issue created that explains the change and why it's needed
• [ ] Tests are part of the PR (for bug fixes / features)
• [ ] Docs reviewed and added / updated if needed (for bug fixes / features)
• [ ] PR conforms to Coding Conventions
• [ ] PR applys BSD 3-clause or LGPL2.1+ Licenses to all code files
• [ ] Lint (flakeheaven lint src/lava tests/) and (bandit -r src/lava/.) pass locally
• [ ] Build tests (pytest) passes locally

Pull request type

Please check your PR type:

• [ ] Bugfix
• [ ] Feature
• [ ] Code style update (formatting, renaming)
• [ ] Refactoring (no functional changes, no api changes)
• [ ] Build related changes
• [ ] Documentation changes
• [ ] Other (please describe):

What is the current behavior?

-

What is the new behavior?

-

Does this introduce a breaking change?

• [ ] Yes
• [ ] No

Supplemental information