Have Boron wake up OpenMV and send data

[BentleySettin]

Possible steps to accomplish Boron recieving data from OpenMV.

• [ ] Boron is asleep and OpenMV asleep
• [ ] Boron wakes up every X minutes
• [ ] Boron tells OpenMV to wake up, send "Flood" or "No Flood".
• [ ] Boron uploads this to the cloud
• [ ] Open MV and Boron go to sleep again.

[ebgoldstein]

https://forums.openmv.io/t/sleep-mode-wakeup-with-external-interrupt/767/2

[BentleySettin]

https://docs.openmv.io/library/pyb.RTC.html

https://forums.openmv.io/t/sleep-mode/1083/2

[SUPScientist]

OpenMV sleep is necessary; OpenMV RTC is not. Using Boron's cloud-based knowledge of real time to control timing.

[BentleySettin]

1. (strongly recommended) Have Boron acknowledge receipt of "flood"/"no flood" communication by sending back an "ACK" message to OpenMV
2. Put both controllers to sleep
3. OpenMV goes to sleep after receiving "ACK"
4. Boron goes to sleep after sending "ACK"
5. Use Boron's timer to wake Boron
6. Have Boron wake OpenMV and tell it to take photo/decide/send "flood"/"no flood" response (I don't know how to implement this yet but I see you posted on a GH issue some OpenMV sleep documentation).
7. Repeat.

[BentleySettin]

@SUPScientist @ebgoldstein

I added https://github.com/TinyCamML/Boron/blob/c7e7e1a098de0205ec5586d115e11ee586ce266a/Boron%20--%3E%20OpenMV#L28 to Boron code.

I added https://github.com/TinyCamML/OMVmodel/blob/980fde8f0597616af7be8aead3dba843cb5bf8c7/updatedOpenMVCode.py#L73 to the OpenMV code.

The OpenMV code is successfully recieving the character and undergoing the 58 second sleep. However I am not recieving any "Flood" on the Boron's serial monitor.

[BentleySettin]

^^ Looks like it is all with exact timing of when both are awake. I changed the times and I got one successful run but then it does not continue to print Flood on VS serial monitor after the allotted time.

[BentleySettin]

Serial connection closed. Attempting to reconnect... Serial monitor opened successfully: FloodFlood

Serial connection closed. Attempting to reconnect... Serial monitor opened successfully: FloodFlood

Serial connection closed. Attempting to reconnect... Serial monitor opened successfully: FloodFlood

Serial connection closed. Attempting to reconnect... Serial monitor opened successfully: FloodFlood

Serial connection closed. Attempting to reconnect...

This is what I am consistently recieving now on VS Serial Monitor. I have been using my phone timer to see if the times I put in for intervalled samples line up with my phone. Current code I am using has OPENMV sleeping for 60 seconds, on my phone this is actually 62 seconds, I am assuming added time is for set up. On the Boron I have it set to sample every 60 seconds (with an added 1 second delay in total), so overall 61 seconds. My phone time records it as 61 seconds but when it reads Flood it takes another 5 seconds to continue into next cycle.

[SUPScientist]

I recommend investigating options to put OpenMV to sleep only to be woken up by a communication from the Boron (this would be called an "external interrupt"). Getting both devices to lock into the same timing will be too difficult. In other words, do away with a sleep timer on the OpenMV altogether, assuming there's an option to send the OpenMV some wakeup command, or change the state of another pin (e.g., some unused GPIO line on the Boron tied to an unused GPIO on the OpenMV changes state from low to high which wakes up the OpenMV).

Learning what options exist for sleep and wakeup will take some reading. For instance, https://forums.openmv.io/t/wakeup-by-external-interrupt-from-deepsleep/1229.

Passing this back to you, @BentleySettin, to read up on OpenMV sleep and wakeup options. Another starting point: https://docs.openmv.io/library/machine.html#machine.sleep.

[BentleySettin]

Okay, I will look into this tomorrow morning! I tried using an if statement to run the while(True) loop if it recieved the ACK but that did not work so well. I used a few of those options but will continue to find the one that works best, thank you!

[anardek]

Just commenting here that I think Phil's approach (that he emailed this morning) is the right one. Don't worry about the RTC component now. Just figure out how to get information passing regularly with a timer. Looks like you guys are already on it!

[BentleySettin]

Got it to work for about 5 minutes straight at 30 seconds intervals! OpenMV is sending information upon request of Boron and then both enter sleep.

I pumped it up to 60 seconds and the openMV is showing a MemManage Err.

[BentleySettin]

I got rid of the MemManage Err. It is reporting "Flood" correctly but will inconsistently not report during, what seems to be, random cycles.

Update: I changed delays and for 10 minutes it was reporting Flood fine but has not begun reporting "FloodFloodFlood" for the past 5 minutes.

[SUPScientist]

Sounds like good progress. Memory management is always an important (and, to the ignorant programmers like me, mysterious) element of embedded design.

What did you do to get rid of the MemManage Err?

If you haven't already, please commit/push changes to the repo, then post a link to the relevant commit(s) here (note that this is typically the desired workflow: work on/test changes, commit/push to GH, comment in issue regarding progress/completion of relevant issue; @ebgoldstein, please feel free to suggest refinements to that proposed workflow).

[BentleySettin]

To get rid of the MemManage Err I just restarted the camera.

I only made a slight change to Boron's code https://github.com/TinyCamML/Boron/blob/56f0d5ef6c6e3706647e866d3b8f72be76b9fd9d/updatedBoronCode.ino#L21

OpenMV code changes: https://github.com/TinyCamML/OMVmodel/blob/aab409b108450b18e7d7667cc9d99481ab0e8eee/updatedOpenMVCode.py#L10

https://github.com/TinyCamML/OMVmodel/blob/aab409b108450b18e7d7667cc9d99481ab0e8eee/updatedOpenMVCode.py#L40

lines 40 - 48

Can you let me know if there is a better way to reference updated lines of code and blocks of code. Overall these were the changes.

[SUPScientist]

Got it—thanks. That works. As long as you reference something specific (a file, line of code, commit, etc.) in your comments here, we can cross-reference your description of the status/ongoing issue with the relevant code.

[SUPScientist]

More specifically, I'm still concerned about https://github.com/TinyCamML/OMVmodel/blob/aab409b108450b18e7d7667cc9d99481ab0e8eee/updatedOpenMVCode.py#L82 Prescribing a timed sleep for the OpenMV seems risky. Over a few minutes to hours of lab testing, the OpenMV and Boron clocks may stay synced well enough for this to work. But over longer deployments, differential clock drift will likely cause the timers to be too far apart and either (1) the Boron will end up polling a still sleeping OpenMV or (2) the OpenMV will wake too early and waste power, both forms of failure and/or inefficiency.

I still recommend an external interrupt from GPIO from the Boron to the OpenMV to wake the OpenMV using ONLY the Boron's clock.

[ebgoldstein]

(agree w/ @SUPScientist - i think GPIO wakeup is the way to go, then we rely on one clock)

[BentleySettin]

@ebgoldstein @SUPScientist

Apologies, that was commented out in my original script so that should not be there. I will edit that out. I have been running the code as is without those last two lines and it has been smooth.

[BentleySettin]

I edited it, the code below is the exact code I have used today which is working well. It runs strictly based on an external interrupt and not on timing.

https://github.com/TinyCamML/OMVmodel/commit/aeeb494a62d77ec100895ad152d8dadbaa6e63a1

[SUPScientist]

Thanks, Bentley. Now it seems like the OpenMV isn't going to sleep in between samples at all? Once it enters the while(True) loop, it will be stuck there, and without a sleep command/sequence built into your while() loop, it won't sleep.

[BentleySettin]

I believe it does go to sleep. It has been successfully entering machine.sleep() until it recieves a callback from the Boron via pin P0. Atleast this is how I am interpreting the code and OpenMV behavior while it is running this code.

When I run this code and open up the openMV serial monitor when it enters the machine.sleep() you can see it "disconnect" from the openMV IDE as if there is no camera which tells me it is entering some sort of sleep.

[SUPScientist]

Hmm, OK, we'll probably have to take a look together in person. I'm not following how the code could be allowing that as you don't have a sleep function built into your loop. As soon as the code enters the loop, I don't see how it could get out of the loop nor go to sleep.

Note that your callback https://github.com/TinyCamML/OMVmodel/blob/main/updatedOpenMVCode.py#L40 doesn't appear to do anything, unless I'm missing something. I'm no micropython whiz, but this looks like a prototype and not a real function to me. But that's not the core issue.

The core issue in my interpretation of the code is that you call sleep before the loop so as soon as it gets into the loop—even if it wakes from GPIO interrupt after that first sleep—it will stay awake indefinitely.

[BentleySettin]

I am taking a look into this now in the lab! I attached the code I am working with from the OpenMV github.

https://github.com/openmv/openmv/blob/master/scripts/examples/50-OpenMV-Boards/61-Low-Power/extint_wakeup.py

[SUPScientist]

Looks like a good reference. Have you played with that example to understand its behavior? For instance, if you run that code on the OpenMV with no external connections, what does it do? Then if you add a Boron with a GPIO connected to the OpenMV to wake it up (i.e., your current Boron<—>OpenMV wiring), what does it do?

Now that your OpenMV code does not have its own sleep timing hard-coded into it, I think you are headed in the right direction!

[BentleySettin]

Okay, trying to figure our the behavior now. Seems like it runs through the code once (obviously because there is no loop) and then either continues on with the while(true) loop, which is just flashing blue, or the machine sleeps. So I am going to try to put this into a loop so that it will continuously be waiting for a message from Boron. I am struggling with the fact that once the machine sleeps it stops looking for a message from the Boron so maybe I should try another sleep function.

[SUPScientist]

Noting that ran a successful test of the sleep/wake/LED blink/sleep example by making some changes.

• Added the sleep() call to the while(True) loop
• Woke up the OpenMV using external interrupt from Boron:
  • Boron sleeps for a (currently hard-coded) duration
  • wakes using its own RTC
  • sends a pulse over a GPIO to OpenMV

[BentleySettin]

https://forums.openmv.io/t/sleep-mode-wakeup-with-external-interrupt/767/2

Link to an OpenMV forum where they successfully had the OpenMV wake up from extint and then seems to cycle through this.

[BentleySettin]

Update: With the additional wire from Boron A0 to OpenMV P2 there is successfull cycling between waking up and going to sleep from both devices. However OpenMV is not writing across the uart anymore.

An issue I am seeing is that the openMV is only running when I have it plugged into my computer. It does not work when I have it powered by battery and voltage in from Boron. Possibly a power supply issue.

Within the loop I added blue led on and off after it decides and prints flood or no flood. I am continuously getting the blue led on and off which tells me it is running through the whole loop and should be printing across UART. I have seen some openmv forums where it will not print when connected to USB, however, I still think the main issue is power because all code is the same from when it was successfully printing.

[BentleySettin]

https://forums.openmv.io/t/not-receiving-uart-data-when-disconnected-from-ide/2051/2

May be relevant to previous comment^^^

[BentleySettin]

I have begin recieving a new response on VS serial monitor. The serial monitor prints "R" as if it is coming from the camera. "R" is the character I am sending to openMV as an extint.

Thinking out loud: Is UART struggling to recieve and transmit from both devices? This has worked before so maybe this is still connected to a power issue?

[SUPScientist]

As in our other issue thread, hard to say when I don't know what firmware you're running nor how things are wired. However, this sounds like something we discussed in person last week. The last test we ran together last week stopped using the Boron's Serial1 and the OpenMV's UART 1 for wakeup. It still uses those for true UART communications (e.g., sending/receiving messages). But the wakeup we moved to a different GPIO on both boards.

The rationale was that you have to configure the wakeup pin on the OpenMV in a certain way that is likely not compatible with UART comms, so you have to use a different pin, at least on the OpenMV. I suppose the wakeup command from the Boron could still come from the Boron's Serial1 TX, but we began testing using a different GPIO.

[BentleySettin]

Gotcha! I took out sending anything from boron to openMV.

When I run Boron code without any sleep I recieved Flood but not consistently. I run the current firmware I linked to the other issue (it has sleep) and now I see the Boron wake up, openMV wake up and blink blue LED but there is no Flood print.

This is all with Boron plugged in to USB and battery on openMV which is good news.

[BentleySettin]

Good news!

updated Boron code: https://github.com/TinyCamML/Boron/commit/a71b074a1892f0a9a5f01f4fb962ee019b8a3ef1

I added in the void loop the volatage to repeat going low to high. I noticed that for some reason the openMV registers the low to high once by indicating a blue flash but no "Flood" report so I ran it through voltage change one more time.

Here is what I am seeing on VS monitor:

Serial connection closed. Attempting to reconnect... Serial monitor opened successfully: No Flood

Serial connection closed. Attempting to reconnect... Serial monitor opened successfully: Flood

Serial connection closed. Attempting to reconnect... Serial monitor opened successfully: No Flood

Serial connection closed. Attempting to reconnect... Serial monitor opened successfully: Flood

Possible Issue: I notieced after a few minutes is that the openMV led starts flashing white which apparently means "the firmware on the board panicking because there's a hardware failure." This happened twice when I removed the battery and then added it again. However when I unplugged boron and then replugged it into usb the issue has not happened since and it has been about 10 minutes.

[SUPScientist]

This sounds promising. So can you confirm that:

1. both the Boron and OpenMV go to sleep for a prescribed amount of time,
2. that the Boron's clock determines when to wake the Boron,
3. that the Boron wakes the OpenMV,
4. that the Boron asks the OpenMV for its flood/no flood detection,
5. that, having received the response from OpenMV, the Boron and OpenMV go back to step 1?

If so, I suggest we close this issue as complete!

It does appear that we need to open two more issues in its wake, however: one related to the white LED flashing due to "the firmware on the board panicking because there's a hardware failure" and one related to timing. Your wakeup pin low/high/low/high timing is fairly slow (multiple delay(1000) calls which keep things awake longer than should be necessary; this is a comparatively small issue, but should be called out and optimized eventually).

[BentleySettin]

Yes all of this is complete! I will close this issue and move into the power supply issue in regard to that last part. In regard to the delays I only used those to help me count when things should be occurring in my own head so I can remove those!