pimoroni / enviro

MIT License
101 stars 79 forks source link

RTC stopped? #79

Open MrDrem opened 1 year ago

MrDrem commented 1 year ago

This is a bit of an odd one. It looks to me like the RTC has stopped, and thus the wake command wasn't issued.

The 1:15 UTC reading was taken fine, but it didn't wake up, and so no more readings were sent. I woke and io.adafruit.com was showing no readings for ~6 hours, so I went outside and poked things (at what the log shows as the first 2022-09-08 01:17:08 entry, but was in fact closer to 07:56 GMT).

In fact looking at the log, you can see that the RTC is reporting that the initial time for each of the last 3 readings is 01:17:08.

The board is reporting the battery remaining as 3.534, and I can see from prior readings that it's been logged lower than that and fine.

I'm going to wipe the log so it grabs time again, and see what happens

2022-09-08 01:15:02 [debug    / 114kB] > performing startup
2022-09-08 01:15:02 [info     / 123kB]   - wake reason: unknown
2022-09-08 01:15:02 [debug    / 121kB]   - turn on activity led
2022-09-08 01:15:02 [info     / 120kB]   - config loaded
2022-09-08 01:15:02 [info     / 118kB]   - clock already set
2022-09-08 01:15:02 [info     / 115kB]   - disk space OK
2022-09-08 01:15:02 [info     / 112kB]   - wind speed started
2022-09-08 01:15:02 [info     / 110kB]   - wind speed state recorded
2022-09-08 01:15:03 [info     / 106kB]   - wind speed sensor tick
2022-09-08 01:15:03 [info     / 103kB]   - Not enough wind ticks captured
2022-09-08 01:15:04 [info     /  95kB]   - wind direction captured
2022-09-08 01:15:04 [info     /  93kB]   - sensor readings OK
2022-09-08 01:15:04 [info     / 123kB]   - cached OK
2022-09-08 01:15:04 [info     / 121kB] > 1 cache files need uploading
2022-09-08 01:15:04 [info     / 118kB] > connecting to wifi network 'Celestia'
2022-09-08 01:15:06 [debug    / 112kB]   - connecting
2022-09-08 01:15:09 [debug    /  98kB]   - connecting
2022-09-08 01:15:12 [info     /  88kB]   - ip address:  192.168.1.208
2022-09-08 01:15:13 [info     / 110kB]   - uploaded 2022-09-08T01:15:04Z.json to adafruit_io
2022-09-08 01:15:13 [info     / 107kB] > going to sleep
2022-09-08 01:15:13 [debug    / 105kB]   - clearing and disabling timer and alarm
2022-09-08 01:15:13 [info     / 103kB]   - setting alarm to wake at 01:20am
2022-09-08 01:15:13 [info     / 101kB]   - shutting down
2022-09-08 01:17:08 [debug    / 114kB] > performing startup
2022-09-08 01:17:08 [info     / 123kB]   - wake reason: unknown
2022-09-08 01:17:08 [debug    / 121kB]   - turn on activity led
2022-09-08 01:17:08 [info     / 120kB]   - config loaded
2022-09-08 01:17:08 [info     / 118kB]   - clock already set
2022-09-08 01:17:08 [info     / 115kB]   - disk space OK
2022-09-08 01:17:08 [info     / 112kB]   - wind speed started
2022-09-08 01:17:08 [info     / 110kB]   - wind speed state recorded
2022-09-08 01:17:09 [info     / 105kB]   - Not enough wind ticks captured
2022-09-08 01:17:10 [info     /  96kB]   - wind direction captured
2022-09-08 01:17:10 [info     /  94kB]   - sensor readings OK
2022-09-08 01:17:10 [info     /  91kB]   - cached OK
2022-09-08 01:17:10 [info     / 108kB] > 1 cache files need uploading
2022-09-08 01:17:10 [info     / 105kB] > connecting to wifi network 'Celestia'
2022-09-08 01:17:12 [debug    /  99kB]   - connecting
2022-09-08 01:17:15 [debug    /  86kB]   - connecting
2022-09-08 01:17:17 [info     / 122kB]   - ip address:  192.168.1.208
2022-09-08 01:17:18 [info     / 104kB]   - uploaded 2022-09-08T01:17:10Z.json to adafruit_io
2022-09-08 01:17:18 [info     / 102kB] > going to sleep
2022-09-08 01:17:18 [debug    / 100kB]   - clearing and disabling timer and alarm
2022-09-08 01:17:18 [info     /  97kB]   - setting alarm to wake at 01:20am
2022-09-08 01:17:19 [info     /  95kB]   - shutting down
2022-09-08 01:17:08 [debug    / 114kB] > performing startup
2022-09-08 01:17:08 [info     / 123kB]   - wake reason: unknown
2022-09-08 01:17:08 [debug    / 121kB]   - turn on activity led
2022-09-08 01:17:08 [info     / 120kB]   - config loaded
2022-09-08 01:17:08 [info     / 117kB]   - clock already set
2022-09-08 01:17:08 [info     / 115kB]   - disk space OK
2022-09-08 01:17:08 [info     / 112kB]   - wind speed started
2022-09-08 01:17:09 [info     / 110kB]   - wind speed state recorded
2022-09-08 01:17:10 [info     / 104kB]   - Not enough wind ticks captured
2022-09-08 01:17:10 [info     /  96kB]   - wind direction captured
2022-09-08 01:17:10 [info     /  94kB]   - sensor readings OK
2022-09-08 01:17:10 [info     /  91kB]   - cached OK
2022-09-08 01:17:10 [info     / 122kB] > 1 cache files need uploading
2022-09-08 01:17:10 [info     / 120kB] > connecting to wifi network 'Celestia'
2022-09-08 01:17:12 [debug    / 114kB]   - connecting
2022-09-08 01:17:15 [debug    / 100kB]   - connecting
2022-09-08 01:17:21 [info     /  78kB]   - ip address:  192.168.1.208
2022-09-08 01:17:22 [info     / 107kB]   - uploaded 2022-09-08T01:17:10Z.json to adafruit_io
2022-09-08 01:17:22 [info     / 104kB] > going to sleep
2022-09-08 01:17:22 [debug    / 102kB]   - clearing and disabling timer and alarm
2022-09-08 01:17:22 [info     / 100kB]   - setting alarm to wake at 01:20am
2022-09-08 01:17:22 [info     /  98kB]   - shutting down
2022-09-08 01:17:08 [debug    / 114kB] > performing startup
2022-09-08 01:17:08 [info     / 123kB]   - wake reason: unknown
2022-09-08 01:17:08 [debug    / 121kB]   - turn on activity led
2022-09-08 01:17:08 [info     / 120kB]   - config loaded
2022-09-08 01:17:08 [info     / 118kB]   - clock already set
2022-09-08 01:17:08 [info     / 115kB]   - disk space OK
2022-09-08 01:17:08 [info     / 112kB]   - wind speed started
2022-09-08 01:17:08 [info     / 110kB]   - wind speed state recorded
2022-09-08 01:17:09 [info     / 105kB]   - Not enough wind ticks captured
2022-09-08 01:17:10 [info     /  97kB]   - wind direction captured
2022-09-08 01:17:10 [info     /  95kB]   - sensor readings OK
2022-09-08 01:17:10 [info     /  91kB]   - cached OK
2022-09-08 01:17:10 [info     / 122kB] > 1 cache files need uploading
2022-09-08 01:17:10 [info     / 120kB] > connecting to wifi network 'Celestia'
2022-09-08 01:17:12 [debug    / 114kB]   - connecting
2022-09-08 01:17:15 [debug    / 100kB]   - connecting
2022-09-08 01:17:17 [info     /  91kB]   - ip address:  192.168.1.208
2022-09-08 01:17:19 [info     / 109kB]   - uploaded 2022-09-08T01:17:10Z.json to adafruit_io
2022-09-08 01:17:19 [info     / 107kB] > going to sleep
2022-09-08 01:17:19 [debug    / 105kB]   - clearing and disabling timer and alarm
2022-09-08 01:17:19 [info     / 102kB]   - setting alarm to wake at 01:20am
2022-09-08 01:17:19 [info     / 100kB]   - shutting down
2022-09-08 01:17:19 [debug    /  98kB]   - on usb power (so can't shutdown) halt and reset instead
MrDrem commented 1 year ago

It may be that this is a battery issue, I'm recharging the battery, and we'll see how it goes.

MrDrem commented 1 year ago

So, I'm not sure what upset things, but the clock didn't want to reset or connect to the time server to get a new time. Reflashing with the 0.0.8 image again seems to have resolved the issue for the moment.

MrDrem commented 1 year ago

So there is certainly something odd going on with the Real Time Clock on my enviro weather.

I've had it sat indoors charging via the Amigo Pro today, and it has been reporting every 5 minutes perfectly. I took it off of power, and it continued to report fine. Then I took it outside, and plugged in just the wind sensors (as the rainfall one will cause it to hang as per issue #70).

The RTC stopped updating at this point.

The below are the readings from io.adafruit.com.

3:00:03PM to 3:50:03PM - Charging via Amigo Pro 3:55:03PM to 5:45:03PM - Running on the charged High Capacity Lithium Ion Battery Pack 5:50:03PM - 5:56:25PM (x3) - 50 & 55 are normal reading with the wind plugged in, the 3 5:56:25PM reading were with it plugged in and the poke button pressed to force a reading 6:00:03PM - unplugged from the wind sensors, and once again on the working RTC, even if it is now running around 30 minutes behind the correct time (it's not updating the time as it thinks it's correct!

There appear to be 2 issues going on here, and I don't understand either of them!

  1. Why does the RTC stop when poked with the wind sensor in? is something shorting or something? It does seem to have actually stopped, as otherwise the time would be correct once the readings come back.
  2. This is new behaviour, why has it suddenly started, as it was going back to working correctly when poked (as documented under issue #70 with the rainfall, and it coming back with a poke from those fails).

I'm really confused by this one!

Voltage Values to show charge times
2022/09/09 6:00:03PM | 4.183
2022/09/09 5:56:25PM | 4.183
2022/09/09 5:56:25PM | 4.185
2022/09/09 5:56:25PM | 4.179
2022/09/09 5:55:03PM | 4.184
2022/09/09 5:50:03PM | 4.182
2022/09/09 5:45:03PM | 4.182
2022/09/09 5:40:03PM | 4.185
2022/09/09 5:35:03PM | 4.185
2022/09/09 5:30:03PM | 4.185
2022/09/09 5:25:03PM | 4.185
2022/09/09 5:20:03PM | 4.187
2022/09/09 5:15:03PM | 4.185
2022/09/09 5:10:03PM | 4.186
2022/09/09 5:05:03PM | 4.182
2022/09/09 5:00:03PM | 4.187
2022/09/09 4:55:03PM | 4.188
2022/09/09 4:50:03PM | 4.183
2022/09/09 4:45:03PM | 4.189
2022/09/09 4:40:03PM | 4.184
2022/09/09 4:35:03PM | 4.188
2022/09/09 4:30:03PM | 4.189
2022/09/09 4:25:03PM | 4.192
2022/09/09 4:20:03PM | 4.19
2022/09/09 4:15:03PM | 4.194
2022/09/09 4:10:03PM | 4.195
2022/09/09 4:05:03PM | 4.196
2022/09/09 4:00:03PM | 4.195
2022/09/09 3:55:03PM | 4.198
2022/09/09 3:50:03PM | 3.547
2022/09/09 3:45:03PM | 3.551
2022/09/09 3:40:03PM | 3.547
2022/09/09 3:35:03PM | 3.548
2022/09/09 3:30:03PM | 3.54
2022/09/09 3:25:03PM | 3.539
2022/09/09 3:20:03PM | 3.533
2022/09/09 3:15:03PM | 3.53
2022/09/09 3:10:03PM | 3.529
2022/09/09 3:05:03PM | 3.522
2022/09/09 3:00:03PM | 3.519
Wind direction values (90 is also the standard reading if not connected)

2022/09/09 6:00:03PM    90      
2022/09/09 5:56:25PM    315     
2022/09/09 5:56:25PM    90      
2022/09/09 5:56:25PM    315     
2022/09/09 5:55:03PM    225     
2022/09/09 5:50:03PM    135     
2022/09/09 5:45:03PM    90
MrDrem commented 1 year ago

OK, so here's another, and this is getting really really odd, sorry!

Log file below.

20:55:02 - wake, connected to wind sensors, all looks good, wake set to 21:00 21:00:01 - wake happens, all looks good, @21:00:11 wake is set to 21:05 ~ 21:17 - There have been no wakes, I go to poke it 21:00:01 - log restarts following poke, and readings all look good, except for the time. ~ 21:18 I turn it off and bring it inside to look at the logs ~21:19 plugged in and connected to my laptop, time has been set to 2000-01-01 00:00:01, connects to ntc, corrects & time 21:19:39 - restart happens all looks good, @21:19:41 wake set to 21:20 21:20:02 - start up starts and looks good, but there is time oddness before hand

I don't understand why the RTC suddenly seems to have and issue and why it would only appear when the wind sensors are plugged in, which they weren't before.

2022-09-09 20:55:02 [debug    / 114kB] > performing startup
2022-09-09 20:55:02 [info     / 123kB]   - wake reason: unknown
2022-09-09 20:55:02 [debug    / 121kB]   - turn on activity led
2022-09-09 20:55:02 [info     / 120kB]   - config loaded
2022-09-09 20:55:02 [info     / 118kB]   - clock already set
2022-09-09 20:55:02 [info     / 115kB]   - disk space OK
2022-09-09 20:55:02 [info     / 112kB]   - wind speed started
2022-09-09 20:55:02 [info     / 111kB]   - wind speed state recorded
2022-09-09 20:55:02 [info     / 108kB]   - wind speed sensor tick
2022-09-09 20:55:03 [info     / 103kB]   - Not enough wind ticks captured
2022-09-09 20:55:03 [info     / 100kB]   - wind direction captured
2022-09-09 20:55:03 [info     /  98kB]   - sensor readings OK
2022-09-09 20:55:04 [info     /  95kB]   - cached OK
2022-09-09 20:55:04 [info     /  92kB] > 1 cache files need uploading
2022-09-09 20:55:04 [info     / 124kB] > connecting to wifi network 'Celestia'
2022-09-09 20:55:06 [debug    / 118kB]   - connecting
2022-09-09 20:55:09 [debug    / 104kB]   - connecting
2022-09-09 20:55:15 [info     /  81kB]   - ip address:  192.168.1.208
2022-09-09 20:55:16 [info     / 108kB]   - uploaded 2022-09-09T20:55:04Z.json to adafruit_io
2022-09-09 20:55:16 [info     / 106kB] > going to sleep
2022-09-09 20:55:16 [debug    / 104kB]   - clearing and disabling timer and alarm
2022-09-09 20:55:16 [info     / 101kB]   - setting alarm to wake at 21:00pm
2022-09-09 20:55:16 [info     /  99kB]   - shutting down
2022-09-09 21:00:01 [debug    / 114kB] > performing startup
2022-09-09 21:00:01 [info     / 123kB]   - wake reason: unknown
2022-09-09 21:00:01 [debug    / 121kB]   - turn on activity led
2022-09-09 21:00:01 [info     / 120kB]   - config loaded
2022-09-09 21:00:01 [info     / 118kB]   - clock already set
2022-09-09 21:00:01 [info     / 115kB]   - disk space OK
2022-09-09 21:00:01 [info     / 112kB]   - wind speed started
2022-09-09 21:00:01 [info     / 110kB]   - wind speed state recorded
2022-09-09 21:00:03 [info     / 105kB]   - Not enough wind ticks captured
2022-09-09 21:00:03 [info     / 102kB]   - wind direction captured
2022-09-09 21:00:03 [info     / 100kB]   - sensor readings OK
2022-09-09 21:00:03 [info     /  96kB]   - cached OK
2022-09-09 21:00:03 [info     /  94kB] > 1 cache files need uploading
2022-09-09 21:00:03 [info     /  91kB] > connecting to wifi network 'Celestia'
2022-09-09 21:00:05 [debug    / 119kB]   - connecting
2022-09-09 21:00:08 [debug    / 105kB]   - connecting
2022-09-09 21:00:10 [info     /  96kB]   - ip address:  192.168.1.208
2022-09-09 21:00:11 [info     / 115kB]   - uploaded 2022-09-09T21:00:03Z.json to adafruit_io
2022-09-09 21:00:11 [info     /  98kB] > going to sleep
2022-09-09 21:00:11 [debug    /  96kB]   - clearing and disabling timer and alarm
2022-09-09 21:00:11 [info     /  94kB]   - setting alarm to wake at 21:05pm
2022-09-09 21:00:11 [info     /  92kB]   - shutting down
2022-09-09 21:00:01 [debug    / 114kB] > performing startup
2022-09-09 21:00:01 [info     / 123kB]   - wake reason: unknown
2022-09-09 21:00:01 [debug    / 121kB]   - turn on activity led
2022-09-09 21:00:01 [info     / 120kB]   - config loaded
2022-09-09 21:00:01 [info     / 118kB]   - clock already set
2022-09-09 21:00:01 [info     / 115kB]   - disk space OK
2022-09-09 21:00:01 [info     / 112kB]   - wind speed started
2022-09-09 21:00:01 [info     / 110kB]   - wind speed state recorded
2022-09-09 21:00:02 [info     / 105kB]   - wind speed sensor tick
2022-09-09 21:00:03 [info     / 103kB]   - Not enough wind ticks captured
2022-09-09 21:00:03 [info     / 100kB]   - wind direction captured
2022-09-09 21:00:03 [info     /  98kB]   - sensor readings OK
2022-09-09 21:00:03 [info     /  94kB]   - cached OK
2022-09-09 21:00:03 [info     /  91kB] > 1 cache files need uploading
2022-09-09 21:00:03 [info     / 123kB] > connecting to wifi network 'Celestia'
2022-09-09 21:00:05 [debug    / 117kB]   - connecting
2022-09-09 21:00:08 [debug    / 103kB]   - connecting
2022-09-09 21:00:10 [info     /  94kB]   - ip address:  192.168.1.208
2022-09-09 21:00:11 [info     / 114kB]   - uploaded 2022-09-09T21:00:03Z.json to adafruit_io
2022-09-09 21:00:11 [info     / 112kB] > going to sleep
2022-09-09 21:00:11 [debug    / 110kB]   - clearing and disabling timer and alarm
2022-09-09 21:00:11 [info     / 108kB]   - setting alarm to wake at 21:05pm
2022-09-09 21:00:11 [info     / 106kB]   - shutting down
2000-01-01 00:00:01 [debug    / 114kB] > performing startup
2000-01-01 00:00:01 [info     / 123kB]   - wake reason: unknown
2000-01-01 00:00:01 [debug    / 122kB]   - turn on activity led
2000-01-01 00:00:01 [info     / 120kB]   - config loaded
2000-01-01 00:00:01 [info     / 118kB] > clock not set, synchronise from ntp server
2000-01-01 00:00:01 [info     / 116kB] > connecting to wifi network 'Celestia'
2000-01-01 00:00:03 [debug    / 110kB]   - connecting
2000-01-01 00:00:06 [debug    /  96kB]   - connecting
2000-01-01 00:00:08 [info     /  86kB]   - ip address:  192.168.1.208
2022-09-09 21:19:39 [info     / 122kB]   - rtc synched
2022-09-09 21:19:39 [info     / 120kB]   - disk space OK
2022-09-09 21:19:39 [info     / 117kB]   - wind speed started
2022-09-09 21:19:39 [info     / 115kB]   - wind speed state recorded
2022-09-09 21:19:40 [info     / 110kB]   - Not enough wind ticks captured
2022-09-09 21:19:40 [info     / 107kB]   - wind direction captured
2022-09-09 21:19:40 [info     / 105kB]   - sensor readings OK
2022-09-09 21:19:40 [info     / 101kB]   - cached OK
2022-09-09 21:19:40 [info     /  99kB] > 1 cache files need uploading
2022-09-09 21:19:40 [info     /  96kB] > already connected to wifi
2022-09-09 21:19:41 [info     / 112kB]   - uploaded 2022-09-09T21:19:40Z.json to adafruit_io
2022-09-09 21:19:41 [info     / 110kB] > going to sleep
2022-09-09 21:19:41 [debug    / 107kB]   - clearing and disabling timer and alarm
2022-09-09 21:19:41 [info     / 105kB]   - setting alarm to wake at 21:20pm
2022-09-09 21:19:41 [info     / 103kB]   - shutting down
2022-09-09 21:19:41 [debug    / 101kB]   - on usb power (so can't shutdown) halt and reset instead
2022-09-09 21:21:01 [debug    /  99kB]   - reset
2022-09-09 21:20:02 [debug    / 114kB] > performing startup
2022-09-09 21:20:02 [info     / 123kB]   - wake reason: unknown
2022-09-09 21:20:02 [debug    / 121kB]   - turn on activity led
2022-09-09 21:20:02 [info     / 120kB]   - config loaded
2022-09-09 21:20:02 [info     / 118kB]   - clock already set
2022-09-09 21:20:02 [info     / 115kB]   - disk space OK
2022-09-09 21:20:02 [info     / 112kB]   - wind speed started
2022-09-09 21:20:02 [info     / 110kB]   - wind speed state recorded
2022-09-09 21:20:04 [info     /  90kB]   - Not enough wind ticks captured
2022-09-09 21:20:04 [info     / 122kB]   - wind direction captured
2022-09-09 21:20:04 [info     / 120kB]   - sensor readings OK
2022-09-09 21:20:04 [info     / 116kB]   - cached OK
2022-09-09 21:20:04 [info     / 113kB] > 1 cache files need uploading
2022-09-09 21:20:04 [info     / 111kB] > connecting to wifi network 'Celestia'
2022-09-09 21:20:06 [debug    / 105kB]   - connecting
2022-09-09 21:20:09 [debug    /  91kB]   - connecting
2022-09-09 21:20:11 [info     /  81kB]   - ip address:  192.168.1.208
2022-09-09 21:20:12 [info     / 108kB]   - uploaded 2022-09-09T21:20:04Z.json to adafruit_io
2022-09-09 21:20:12 [info     / 106kB] > going to sleep
2022-09-09 21:20:12 [debug    / 104kB]   - clearing and disabling timer and alarm
2022-09-09 21:20:12 [info     / 102kB]   - setting alarm to wake at 21:25pm
2022-09-09 21:20:13 [info     / 100kB]   - shutting down
2022-09-09 21:20:13 [debug    /  97kB]   - on usb power (so can't shutdown) halt and reset instead
MrDrem commented 1 year ago

OK, so its not the wind sensors, as I've just put it outside without them connected either, and it's still falling over after a couple of readings in exactly the same way. I'm now really struggling to understand this.

I have realised that I may be mistaken here, and that what I was seeing was readings at the wrong times, caused by the RTC reporting a time earlier than it actually was. I've updated the RTC time, and put it outside again to see what happens.

Just being outside and within the Stevenson Screen is fine. I'm going to try plugging in the wind sensors again shortly.

MrDrem commented 1 year ago

I guess shortly is relative, but it's now reconnected, and seems to be working, although I think that I am updating the RTC every reading.

I still don't understand the issue I was seeing though.

MrDrem commented 1 year ago

Spoke too soon, it stopped again, log files to follow later. :(

I'm going to break the below into a set of records so that the errors can be seen more clearly.

You can see that each time it wakes, the time reported is a couple of seconds behind where the previous log ended, and is then corrected when the RTC sync is forced.

~22:26

2022-09-10 22:24:44 [info     /  99kB]   - shutting down
2022-09-10 22:24:41 [debug    / 115kB] > performing startup
2022-09-10 22:24:41 [info     / 122kB]   - wake reason: unknown
2022-09-10 22:24:41 [debug    / 120kB]   - turn on activity led
2022-09-10 22:24:41 [info     / 118kB]   - config loaded
2022-09-10 22:24:41 [info     / 116kB] > connecting to wifi network 'Celestia'
2022-09-10 22:24:43 [debug    / 110kB]   - connecting
2022-09-10 22:24:46 [debug    /  96kB]   - connecting
2022-09-10 22:24:47 [info     /  93kB]   - ip address:  192.168.1.208
2022-09-10 22:26:47 [info     /  90kB]   - rtc synched
2022-09-10 22:26:47 [info     /  88kB]   - clock already set
2022-09-10 22:26:47 [info     / 123kB]   - disk space OK
2022-09-10 22:26:47 [info     / 120kB]   - wind speed started
2022-09-10 22:26:47 [info     / 118kB]   - wind speed state recorded
2022-09-10 22:26:48 [info     / 112kB]   - Not enough wind ticks captured
2022-09-10 22:26:48 [info     / 109kB]   - wind direction captured
2022-09-10 22:26:48 [info     / 107kB]   - sensor readings OK
2022-09-10 22:26:48 [info     / 104kB]   - cached OK
2022-09-10 22:26:48 [info     / 101kB] > 1 cache files need uploading
2022-09-10 22:26:48 [info     /  99kB] > already connected to wifi
2022-09-10 22:26:49 [info     / 114kB]   - uploaded 2022-09-10T22:26:48Z.json to adafruit_io
2022-09-10 22:26:49 [info     / 112kB] > going to sleep
2022-09-10 22:26:49 [debug    / 110kB]   - clearing and disabling timer and alarm
2022-09-10 22:26:49 [info     / 108kB]   - setting alarm to wake at 22:30pm
2022-09-10 22:26:49 [info     / 106kB]   - shutting down

~00:40

2022-09-10 22:26:47 [debug    / 115kB] > performing startup
2022-09-10 22:26:47 [info     / 122kB]   - wake reason: unknown
2022-09-10 22:26:47 [debug    / 120kB]   - turn on activity led
2022-09-10 22:26:47 [info     / 118kB]   - config loaded
2022-09-10 22:26:47 [info     / 116kB] > connecting to wifi network 'Celestia'
2022-09-10 22:26:49 [debug    / 110kB]   - connecting
2022-09-10 22:26:52 [debug    /  97kB]   - connecting
2022-09-10 22:26:53 [info     /  93kB]   - ip address:  192.168.1.208
2022-09-11 00:40:08 [info     /  90kB]   - rtc synched
2022-09-11 00:40:08 [info     /  88kB]   - clock already set
2022-09-11 00:40:08 [info     / 123kB]   - disk space OK
2022-09-11 00:40:08 [info     / 119kB]   - wind speed started
2022-09-11 00:40:08 [info     / 117kB]   - wind speed state recorded
2022-09-11 00:40:09 [info     / 112kB]   - Not enough wind ticks captured
2022-09-11 00:40:09 [info     / 109kB]   - wind direction captured
2022-09-11 00:40:09 [info     / 107kB]   - sensor readings OK
2022-09-11 00:40:09 [info     / 103kB]   - cached OK
2022-09-11 00:40:09 [info     / 101kB] > 1 cache files need uploading
2022-09-11 00:40:09 [info     /  98kB] > already connected to wifi
2022-09-11 00:40:10 [info     / 114kB]   - uploaded 2022-09-11T00:40:09Z.json to adafruit_io
2022-09-11 00:40:10 [info     / 112kB] > going to sleep
2022-09-11 00:40:10 [debug    / 110kB]   - clearing and disabling timer and alarm
2022-09-11 00:40:10 [info     / 108kB]   - setting alarm to wake at 00:45am
2022-09-11 00:40:11 [info     / 106kB]   - shutting down

~00:40 again

2022-09-11 00:40:08 [debug    / 115kB] > performing startup
2022-09-11 00:40:08 [info     / 122kB]   - wake reason: unknown
2022-09-11 00:40:08 [debug    / 120kB]   - turn on activity led
2022-09-11 00:40:08 [info     / 118kB]   - config loaded
2022-09-11 00:40:08 [info     / 116kB] > connecting to wifi network 'Celestia'
2022-09-11 00:40:10 [debug    / 110kB]   - connecting
2022-09-11 00:40:13 [debug    /  97kB]   - connecting
2022-09-11 00:40:15 [info     /  87kB]   - ip address:  192.168.1.208
2022-09-11 00:40:22 [info     / 122kB]   - rtc synched
2022-09-11 00:40:22 [info     / 120kB]   - clock already set
2022-09-11 00:40:22 [info     / 118kB]   - disk space OK
2022-09-11 00:40:22 [info     / 115kB]   - wind speed started
2022-09-11 00:40:22 [info     / 113kB]   - wind speed state recorded
2022-09-11 00:40:23 [info     / 107kB]   - Not enough wind ticks captured
2022-09-11 00:40:23 [info     / 104kB]   - wind direction captured
2022-09-11 00:40:23 [info     / 102kB]   - sensor readings OK
2022-09-11 00:40:23 [info     /  98kB]   - cached OK
2022-09-11 00:40:23 [info     /  96kB] > 1 cache files need uploading
2022-09-11 00:40:23 [info     /  93kB] > already connected to wifi
2022-09-11 00:40:24 [info     / 111kB]   - uploaded 2022-09-11T00:40:23Z.json to adafruit_io
2022-09-11 00:40:24 [info     / 108kB] > going to sleep
2022-09-11 00:40:24 [debug    / 106kB]   - clearing and disabling timer and alarm
2022-09-11 00:40:25 [info     / 104kB]   - setting alarm to wake at 00:45am
2022-09-11 00:40:25 [info     / 102kB]   - shutting down

~07:15

2022-09-11 00:40:22 [debug    / 115kB] > performing startup
2022-09-11 00:40:22 [info     / 122kB]   - wake reason: unknown
2022-09-11 00:40:22 [debug    / 120kB]   - turn on activity led
2022-09-11 00:40:22 [info     / 118kB]   - config loaded
2022-09-11 00:40:22 [info     / 116kB] > connecting to wifi network 'Celestia'
2022-09-11 00:40:24 [debug    / 110kB]   - connecting
2022-09-11 00:40:27 [debug    /  97kB]   - connecting
2022-09-11 00:40:29 [info     /  87kB]   - ip address:  192.168.1.208
2022-09-11 07:15:43 [info     / 122kB]   - rtc synched
2022-09-11 07:15:43 [info     / 120kB]   - clock already set
2022-09-11 07:15:43 [info     / 118kB]   - disk space OK
2022-09-11 07:15:43 [info     / 115kB]   - wind speed started
2022-09-11 07:15:43 [info     / 113kB]   - wind speed state recorded
2022-09-11 07:15:44 [info     / 107kB]   - Not enough wind ticks captured
2022-09-11 07:15:44 [info     / 104kB]   - wind direction captured
2022-09-11 07:15:44 [info     / 102kB]   - sensor readings OK
2022-09-11 07:15:44 [info     /  99kB]   - cached OK
2022-09-11 07:15:44 [info     /  96kB] > 1 cache files need uploading
2022-09-11 07:15:44 [info     /  94kB] > already connected to wifi
2022-09-11 07:15:45 [info     / 111kB]   - uploaded 2022-09-11T07:15:44Z.json to adafruit_io
2022-09-11 07:15:45 [info     / 109kB] > going to sleep
2022-09-11 07:15:45 [debug    / 107kB]   - clearing and disabling timer and alarm
2022-09-11 07:15:45 [info     / 104kB]   - setting alarm to wake at 07:20am
2022-09-11 07:15:45 [info     / 102kB]   - shutting down

~07:16

2022-09-11 07:15:43 [debug    / 115kB] > performing startup
2022-09-11 07:15:43 [info     / 122kB]   - wake reason: unknown
2022-09-11 07:15:43 [debug    / 120kB]   - turn on activity led
2022-09-11 07:15:43 [info     / 118kB]   - config loaded
2022-09-11 07:15:43 [info     / 116kB] > connecting to wifi network 'Celestia'
2022-09-11 07:15:45 [debug    / 110kB]   - connecting
2022-09-11 07:15:48 [debug    /  97kB]   - connecting
2022-09-11 07:15:54 [info     / 113kB]   - ip address:  192.168.1.208
2022-09-11 07:16:02 [info     / 110kB]   - rtc synched
2022-09-11 07:16:02 [info     / 108kB]   - clock already set
2022-09-11 07:16:02 [info     / 106kB]   - disk space OK
2022-09-11 07:16:02 [info     / 103kB]   - wind speed started
2022-09-11 07:16:02 [info     / 101kB]   - wind speed state recorded
2022-09-11 07:16:03 [info     /  95kB]   - Not enough wind ticks captured
2022-09-11 07:16:03 [info     /  92kB]   - wind direction captured
2022-09-11 07:16:03 [info     / 116kB]   - sensor readings OK
2022-09-11 07:16:03 [info     / 112kB]   - cached OK
2022-09-11 07:16:04 [info     / 110kB] > 1 cache files need uploading
2022-09-11 07:16:04 [info     / 107kB] > already connected to wifi
2022-09-11 07:16:05 [info     /  88kB]   - uploaded 2022-09-11T07:16:03Z.json to adafruit_io
2022-09-11 07:16:05 [info     /  86kB] > going to sleep
2022-09-11 07:16:05 [debug    /  84kB]   - clearing and disabling timer and alarm
2022-09-11 07:16:05 [info     / 123kB]   - setting alarm to wake at 07:20am
2022-09-11 07:16:05 [info     / 121kB]   - shutting down

~13:49

2000-01-01 00:00:02 [debug    / 115kB] > performing startup
2000-01-01 00:00:02 [info     / 122kB]   - wake reason: unknown
2000-01-01 00:00:02 [debug    / 120kB]   - turn on activity led
2000-01-01 00:00:02 [info     / 118kB]   - config loaded
2000-01-01 00:00:02 [info     / 116kB] > connecting to wifi network 'Celestia'
2000-01-01 00:00:04 [debug    / 110kB]   - connecting
2000-01-01 00:00:07 [debug    /  97kB]   - connecting
2000-01-01 00:00:10 [info     /  87kB]   - ip address:  192.168.1.208
2022-09-11 13:49:45 [info     / 122kB]   - rtc synched
2022-09-11 13:49:45 [info     / 120kB]   - clock already set
2022-09-11 13:49:45 [info     / 118kB]   - disk space OK
2022-09-11 13:49:45 [info     / 115kB]   - wind speed started
2022-09-11 13:49:45 [info     / 113kB]   - wind speed state recorded
2022-09-11 13:49:46 [info     / 108kB]   - Not enough wind ticks captured
2022-09-11 13:49:46 [info     / 105kB]   - wind direction captured
2022-09-11 13:49:46 [info     / 103kB]   - sensor readings OK
2022-09-11 13:49:46 [info     /  99kB]   - cached OK
2022-09-11 13:49:46 [info     /  97kB] > 1 cache files need uploading
2022-09-11 13:49:46 [info     /  94kB] > already connected to wifi
2022-09-11 13:49:47 [info     / 111kB]   - uploaded 2022-09-11T13:49:46Z.json to adafruit_io
2022-09-11 13:49:47 [info     / 109kB] > going to sleep
2022-09-11 13:49:47 [debug    / 107kB]   - clearing and disabling timer and alarm
2022-09-11 13:49:47 [info     / 104kB]   - setting alarm to wake at 13:50pm
2022-09-11 13:49:47 [info     / 102kB]   - shutting down
2022-09-11 13:49:47 [debug    / 100kB]   - on usb power (so can't shutdown) halt and reset instead
MrDrem commented 1 year ago

I've gone back to the 0.0.7 firmware by flashing with the flash_nuke.uf2 file, then the pimoroni-picow_enviro-1.19.7-micropython-enviro-v0.0.7-test.uf2 file and finally the enviro-v0.0.7-filesystem-only.uf2 file, and although it got stuck, I'm not sure that the RTC was the issue, as it's now come back, and I was seeing this behaviour previously with 0.0.7 I think.

Back to the logs soon!

MrDrem commented 1 year ago

Dagnabbit.

18:55:02 - expected

2022-09-11 18:55:02 [debug    / 115kB] > performing startup
2022-09-11 18:55:02 [info     / 113kB]   - wake reason: unknown
2022-09-11 18:55:02 [debug    / 111kB]   - turn on activity led
2022-09-11 18:55:03 [info     / 102kB] > 1 cache files need uploading
2022-09-11 18:55:03 [info     / 100kB] > connecting to wifi network 'Celestia'
2022-09-11 18:55:05 [debug    / 124kB]   - connecting
2022-09-11 18:55:08 [debug    / 110kB]   - connecting
2022-09-11 18:55:10 [info     / 101kB]   - ip address:  192.168.1.208
2022-09-11 18:55:11 [info     / 121kB]   - uploaded 2022-09-11T18:55:03Z.json to adafruit_io
2022-09-11 18:55:11 [info     / 119kB] > going to sleep
2022-09-11 18:55:11 [debug    / 117kB]   - clearing and disabling timer and alarm
2022-09-11 18:55:11 [info     / 115kB]   - setting alarm to wake at 19:00pm
2022-09-11 18:55:11 [info     / 113kB]   - shutting down

18:58:37 - poked

2022-09-11 18:58:37 [debug    / 115kB] > performing startup
2022-09-11 18:58:37 [info     / 113kB]   - wake reason: unknown
2022-09-11 18:58:37 [debug    / 111kB]   - turn on activity led
2022-09-11 18:58:38 [info     / 102kB] > 1 cache files need uploading
2022-09-11 18:58:38 [info     / 100kB] > connecting to wifi network 'Celestia'
2022-09-11 18:58:40 [debug    / 123kB]   - connecting
2022-09-11 18:58:43 [debug    / 110kB]   - connecting
2022-09-11 18:58:45 [info     / 100kB]   - ip address:  192.168.1.208
2022-09-11 18:58:46 [info     / 119kB]   - uploaded 2022-09-11T18:58:38Z.json to adafruit_io
2022-09-11 18:58:46 [info     / 117kB] > going to sleep
2022-09-11 18:58:46 [debug    / 115kB]   - clearing and disabling timer and alarm
2022-09-11 18:58:46 [info     / 113kB]   - setting alarm to wake at 19:00pm
2022-09-11 18:58:47 [info     / 110kB]   - shutting down

18:58:37 - poked again

2022-09-11 18:58:37 [debug    / 115kB] > performing startup
2022-09-11 18:58:37 [info     / 113kB]   - wake reason: unknown
2022-09-11 18:58:37 [debug    / 111kB]   - turn on activity led
2022-09-11 18:58:38 [info     / 102kB] > 1 cache files need uploading
2022-09-11 18:58:38 [info     / 100kB] > connecting to wifi network 'Celestia'
2022-09-11 18:58:40 [debug    / 104kB]   - connecting
2022-09-11 18:58:43 [debug    /  91kB]   - connecting
2022-09-11 18:58:45 [info     /  82kB]   - ip address:  192.168.1.208
2022-09-11 18:58:46 [info     / 110kB]   - uploaded 2022-09-11T18:58:38Z.json to adafruit_io
2022-09-11 18:58:46 [info     / 108kB] > going to sleep
2022-09-11 18:58:46 [debug    / 105kB]   - clearing and disabling timer and alarm
2022-09-11 18:58:46 [info     / 103kB]   - setting alarm to wake at 19:00pm
2022-09-11 18:58:46 [info     / 101kB]   - shutting down
2022-09-11 18:58:37 [debug    / 115kB] > performing startup
2022-09-11 18:58:37 [info     / 113kB]   - wake reason: unknown
2022-09-11 18:58:37 [debug    / 111kB]   - turn on activity led
2022-09-11 18:58:38 [info     / 102kB] > 1 cache files need uploading
2022-09-11 18:58:38 [info     / 100kB] > connecting to wifi network 'Celestia'
2022-09-11 18:58:40 [debug    / 123kB]   - connecting

pokes where not at the time stated but the RTC, note they are doubled again. it seems to have failed between readings this time, which is interesting.

I have no idea what is going on here

MrDrem commented 1 year ago

Right, I'm back on 0.0.8

I've now had the Weather board running for the last 12 hours without the wind or rain sensors plugged in and it's once again been fine. I'll plug the wind sensors back in later to see what happens again.

MrDrem commented 1 year ago

So I plugged the wind speed sensor in @15:14, and everything stopped.

2022-09-12 15:10:02 [debug    / 115kB] > performing startup
2022-09-12 15:10:02 [info     / 123kB]   - wake reason: unknown
2022-09-12 15:10:02 [debug    / 121kB]   - turn on activity led
2022-09-12 15:10:03 [info     / 112kB] > 1 cache files need uploading
2022-09-12 15:10:03 [info     / 109kB] > connecting to wifi network 'Celestia'
2022-09-12 15:10:05 [debug    / 103kB]   - connecting
2022-09-12 15:10:08 [debug    /  90kB]   - connecting
2022-09-12 15:10:11 [info     / 118kB]   - ip address:  192.168.1.208
2022-09-12 15:10:11 [info     /  99kB]   - uploaded 2022-09-12T15:10:03Z.json to adafruit_io
2022-09-12 15:10:12 [info     /  97kB] > going to sleep
2022-09-12 15:10:12 [debug    /  95kB]   - clearing and disabling timer and alarm
2022-09-12 15:10:12 [info     /  93kB]   - setting alarm to wake at 15:15pm
2022-09-12 15:10:12 [info     /  91kB]   - shutting down
2000-01-01 00:00:02 [debug    / 115kB] > performing startup
2000-01-01 00:00:02 [info     / 123kB]   - wake reason: unknown
2000-01-01 00:00:02 [debug    / 121kB]   - turn on activity led
2000-01-01 00:00:02 [info     / 119kB] > clock not set, synchronise from ntp server
2000-01-01 00:00:02 [info     / 103kB] > connecting to wifi network 'Celestia'
2000-01-01 00:00:04 [debug    /  96kB]   - connecting
2000-01-01 00:00:07 [debug    / 120kB]   - connecting
2000-01-01 00:00:08 [info     / 117kB]   - ip address:  192.168.1.208
2020-05-11 01:02:52 [debug    / 115kB] > performing startup
2020-05-11 01:02:52 [info     / 123kB]   - wake reason: unknown
2020-05-11 01:02:52 [debug    / 121kB]   - turn on activity led
2020-05-11 01:02:52 [info     / 119kB] > clock not set, synchronise from ntp server
2020-05-11 01:02:52 [info     / 117kB] > connecting to wifi network 'Celestia'
2020-05-11 01:02:54 [debug    / 111kB]   - connecting
2020-05-11 01:02:57 [debug    /  97kB]   - connecting
2020-05-11 01:03:03 [info     / 113kB]   - ip address:  192.168.1.208
2022-09-12 15:22:14 [info     / 111kB]   - rtc synched
2022-09-12 15:22:15 [info     / 101kB] > 1 cache files need uploading
2022-09-12 15:22:15 [info     /  99kB] > already connected to wifi
2022-09-12 15:22:16 [info     / 115kB]   - uploaded 2022-09-12T15:22:15Z.json to adafruit_io
2022-09-12 15:22:16 [info     / 113kB] > going to sleep
2022-09-12 15:22:16 [debug    / 111kB]   - clearing and disabling timer and alarm
2022-09-12 15:22:16 [info     / 109kB]   - setting alarm to wake at 15:25pm
2022-09-12 15:22:16 [info     / 107kB]   - shutting down
2022-09-12 15:22:16 [debug    / 105kB]   - on usb power (so can't shutdown) halt and reset instead
MrDrem commented 1 year ago

Right, I've pulled out the wind speed sensor, and it's been working fine for ~5 hours.

I'm now wondering if it's my sensors getting upset, or something on the board or in the code, and how on earth I check this.

I don't at this point think it's the code, as it was working fine previously. I still don't get why a faulty sensor would crash the board though.

MrDrem commented 1 year ago

Nope, it dropped out last night with just the wind direction sensor going.

last reading from last night:

2022-09-13 00:05:02 [debug    / 115kB] > performing startup
2022-09-13 00:05:02 [info     / 123kB]   - wake reason: unknown
2022-09-13 00:05:02 [debug    / 121kB]   - turn on activity led
2022-09-13 00:05:03 [info     / 112kB] > 1 cache files need uploading
2022-09-13 00:05:03 [info     / 109kB] > connecting to wifi network 'Celestia'
2022-09-13 00:05:05 [debug    / 103kB]   - connecting
2022-09-13 00:05:08 [debug    /  90kB]   - connecting
2022-09-13 00:05:11 [info     / 118kB]   - ip address:  192.168.1.208
2022-09-13 00:05:11 [info     /  99kB]   - uploaded 2022-09-13T00:05:03Z.json to adafruit_io
2022-09-13 00:05:12 [info     /  97kB] > going to sleep
2022-09-13 00:05:12 [debug    /  95kB]   - clearing and disabling timer and alarm
2022-09-13 00:05:12 [info     /  93kB]   - setting alarm to wake at 00:10am
2022-09-13 00:05:12 [info     /  90kB]   - shutting down

reading taken at 09:13 this morning when I plugged it into the usb port to get at the logs:

2022-09-13 00:08:47 [debug    / 115kB] > performing startup
2022-09-13 00:08:47 [info     / 123kB]   - wake reason: unknown
2022-09-13 00:08:47 [debug    / 121kB]   - turn on activity led
2022-09-13 00:08:48 [info     / 112kB] > 1 cache files need uploading
2022-09-13 00:08:48 [info     / 110kB] > connecting to wifi network 'Celestia'
2022-09-13 00:08:50 [debug    / 103kB]   - connecting
2022-09-13 00:08:53 [debug    /  90kB]   - connecting
2022-09-13 00:08:54 [info     / 124kB]   - ip address:  192.168.1.208
2022-09-13 00:08:55 [info     / 106kB]   - uploaded 2022-09-13T00:08:48Z.json to adafruit_io
2022-09-13 00:08:55 [info     / 104kB] > going to sleep
2022-09-13 00:08:55 [debug    / 102kB]   - clearing and disabling timer and alarm
2022-09-13 00:08:55 [info     /  99kB]   - setting alarm to wake at 00:10am
2022-09-13 00:08:55 [info     /  97kB]   - shutting down
2022-09-13 00:08:55 [debug    / 114kB]   - on usb power (so can't shutdown) halt and reset instead

the RTC has clearly stopped again :(

MrDrem commented 1 year ago

I wonder if this is related to #87, although looking at their logs, the RTC is seeming to continue to run on, not stopping advancing like mine is, so maybe that's more akin to #70?

MrDrem commented 1 year ago

Without any sensors connected, it had kept going for over 24 hours. I connected the wind direction sensor first, and 20 minutes later added the wind speed sensor.

Looks like the board shut down correctly, but didn't wake, although I connected the wind speed sensor part way through a sleep cycle. The RTC seems to have kept going this time though.

I don't understand how it would upset the RTC

In the morning I'm going to reset and try running it without uploads, so that it's all local. I'll aim for an hour with no sensors, and hour with direction only and then an hour with speed and direction.

@helgibbons does this sound like a hardware failure? I don't know and don't understand it.

2022-09-15 17:56:02 [debug    / 114kB] > performing startup
2022-09-15 17:56:02 [info     / 121kB]   - wake reason: unknown
2022-09-15 17:56:02 [debug    / 119kB]   - turn on activity led
2022-09-15 17:56:02 [info     / 117kB]   - Config Good
2022-09-15 17:56:02 [info     / 115kB]   - Clock set
2022-09-15 17:56:02 [info     / 113kB]   - space good
2022-09-15 17:56:02 [info     / 110kB]   - Wind sensor inital state set
2022-09-15 17:56:03 [info     / 104kB]   - Wind direction read
2022-09-15 17:56:04 [info     / 102kB]   - sensor data collected
2022-09-15 17:56:04 [info     /  98kB]   - reading cached
2022-09-15 17:56:04 [info     /  96kB] > 1 cache files need uploading
2022-09-15 17:56:04 [info     /  93kB] > connecting to wifi network 'Celestia'
2022-09-15 17:56:06 [debug    / 123kB]   - connecting
2022-09-15 17:56:09 [debug    / 109kB]   - connecting
2022-09-15 17:56:11 [info     /  99kB]   - ip address:  192.168.1.208
2022-09-15 17:56:12 [info     / 116kB]   - uploaded 2022-09-15T17:56:04Z.json to adafruit_io
2022-09-15 17:56:12 [info     / 113kB]   - upload successful
2022-09-15 17:56:12 [info     / 111kB] > going to sleep
2022-09-15 17:56:12 [debug    / 109kB]   - clearing and disabling timer and alarm
2022-09-15 17:56:12 [info     / 107kB]   - setting alarm to wake at 18:00pm
2022-09-15 17:56:12 [info     /  90kB]   - shutting down
2022-09-15 18:00:02 [debug    / 114kB] > performing startup
2022-09-15 18:00:02 [info     / 121kB]   - wake reason: unknown
2022-09-15 18:00:02 [debug    / 119kB]   - turn on activity led
2022-09-15 18:00:02 [info     / 117kB]   - Config Good
2022-09-15 18:00:02 [info     / 115kB]   - Clock set
2022-09-15 18:00:02 [info     / 113kB]   - space good
2022-09-15 18:00:02 [info     / 110kB]   - Wind sensor inital state set
2022-09-15 18:00:03 [info     / 104kB]   - Wind direction read
2022-09-15 18:00:04 [info     / 102kB]   - sensor data collected
2022-09-15 18:00:04 [info     /  98kB]   - reading cached
2022-09-15 18:00:04 [info     /  95kB] > 1 cache files need uploading
2022-09-15 18:00:04 [info     /  93kB] > connecting to wifi network 'Celestia'
2022-09-15 18:00:06 [debug    / 122kB]   - connecting
2022-09-15 18:00:09 [debug    / 109kB]   - connecting
2022-09-15 18:00:09 [info     / 106kB]   - ip address:  192.168.1.208
2022-09-15 18:00:10 [info     /  87kB]   - uploaded 2022-09-15T18:00:04Z.json to adafruit_io
2022-09-15 18:00:10 [info     /  85kB]   - upload successful
2022-09-15 18:00:10 [info     /  83kB] > going to sleep
2022-09-15 18:00:10 [debug    / 120kB]   - clearing and disabling timer and alarm
2022-09-15 18:00:10 [info     / 118kB]   - setting alarm to wake at 18:04pm
2022-09-15 18:00:10 [info     / 116kB]   - shutting down
2022-09-15 18:20:48 [debug    / 114kB] > performing startup
2022-09-15 20:14:18 [debug    / 114kB] > performing startup
2022-09-15 20:14:18 [info     / 121kB]   - wake reason: unknown
2022-09-15 20:14:18 [debug    / 119kB]   - turn on activity led
2022-09-15 20:14:18 [info     / 117kB]   - Config Good
2022-09-15 20:14:18 [info     / 115kB]   - Clock set
2022-09-15 20:14:18 [info     / 113kB]   - space good
2022-09-15 20:14:18 [info     / 110kB]   - Wind sensor inital state set
2022-09-15 20:14:19 [info     / 104kB]   - Wind direction read
2022-09-15 20:14:19 [info     / 102kB]   - sensor data collected
2022-09-15 20:14:19 [info     /  98kB]   - reading cached
2022-09-15 20:14:20 [info     /  96kB] > 1 cache files need uploading
2022-09-15 20:14:20 [info     /  93kB] > connecting to wifi network 'Celestia'
2022-09-15 20:14:22 [debug    / 123kB]   - connecting
2022-09-15 20:14:25 [debug    / 109kB]   - connecting
2022-09-15 20:14:25 [info     / 106kB]   - ip address:  192.168.1.208
2022-09-15 20:14:26 [info     /  88kB]   - uploaded 2022-09-15T20:14:19Z.json to adafruit_io
2022-09-15 20:14:26 [info     /  86kB]   - upload successful
2022-09-15 20:14:26 [info     /  84kB] > going to sleep
2022-09-15 20:14:26 [debug    / 120kB]   - clearing and disabling timer and alarm
2022-09-15 20:14:26 [info     / 118kB]   - setting alarm to wake at 20:16pm
2022-09-15 20:14:26 [info     / 116kB]   - shutting down
2022-09-15 20:14:26 [debug    / 114kB]   - on usb power (so can't shutdown) halt and reset instead

Current versions of log.txt, config.py, main.py & weather.py attached here:

log.txt config.py.txt main.py.txt weather.py.txt

MrDrem commented 1 year ago

So I ran it newly wiped and flashed with 0.0.8 firmware, going through setup and just saving readings locally last night, cutting out all of the connection to network and it cut out after around 3.5 hours.

It seems to have been taking readings from all of the connected sensors for all of this time.

From the data files and the log files, it took all of the readings for the last data point and wrote them out, but then failed at some point before setting sleep alarms.

The RTC doesn't seem to have stopped, as it was still recording this morning at the correct (UTC) time when I brought it in to get the data off.

Data points

timestamp,temperature,humidity,pressure,light,wind_speed,rain,wind_direction,voltage
---
2022-09-16T01:30:03Z,12.88,66.05,1014.21,0.0,1.446175,0,180,4.118
2022-09-16T01:35:03Z,12.8,66.17,1014.23,0.0,0,0,315,4.118
2022-09-16T01:40:03Z,12.68,66.53,1014.19,0.0,0,0,315,4.119
2022-09-16T07:22:15Z,17.14,88.19,1015.39,2.55,0,0,90,5.008

Log file entries:

2022-09-16 01:40:02 [info     / 124kB]   - wake reason: unknown
2022-09-16 01:40:02 [debug    / 122kB]   - turn on activity led
2022-09-16 01:40:03 [info     / 113kB] > going to sleep
2022-09-16 01:40:03 [debug    / 111kB]   - clearing and disabling timer and alarm
2022-09-16 01:40:03 [info     / 109kB]   - setting alarm to wake at 01:45am
2022-09-16 01:40:03 [info     / 107kB]   - shutting down
2022-09-16 01:45:02 [debug    / 115kB] > performing startup
2022-09-16 01:45:02 [info     / 124kB]   - wake reason: unknown
2022-09-16 01:45:02 [debug    / 122kB]   - turn on activity led
2022-09-16 07:22:14 [debug    / 115kB] > performing startup
2022-09-16 07:22:14 [info     / 124kB]   - wake reason: unknown
2022-09-16 07:22:14 [debug    / 122kB]   - turn on activity led
2022-09-16 07:22:15 [info     / 113kB] > going to sleep
2022-09-16 07:22:15 [debug    / 111kB]   - clearing and disabling timer and alarm
2022-09-16 07:22:15 [info     / 108kB]   - setting alarm to wake at 07:25am
2022-09-16 07:22:15 [info     / 106kB]   - shutting down
2022-09-16 07:22:15 [debug    / 104kB]   - on usb power (so can't shutdown) halt and reset instead

files here:

2022-09-15.txt 2022-09-16.txt config.py.txt log.txt main.py.txt

helgibbons commented 1 year ago

I'm getting very similar failures to you with Weather and v0.0.8 (and I've tried a few different units) so I think it's more likely to be bugs in the alpha firmware than duff hardware - there's definitely an issue with a sleep trigger not been set after a rain event, but I'm seeing intermittent crashes like the one above too, where everything stalls after 'turn on activity LED'.

Our software team is planning on releasing a new Enviro build that should hopefully address some of these issues - summer holidays mean we're a bit of a skeleton crew at the moment though!

MrDrem commented 1 year ago

I suspect that you're right, especially as I'm running very frequent recording, which likely exacerbates things.

And given last night's "success" with writing to files rather then transmitting. I may go back and update the logging levels, and see what I get again.

helgibbons commented 1 year ago

Just checked in on our weather station (v0.0.8, mqtt endpoint) - looks like this time it had crashed whilst connecting to wifi (which meant it then emptied the battery rather quickly)

Going to try hooking it up to InfluxDB and see if it's any more stable!

2022-09-14 04:40:02 [debug    / 115kB] > performing startup
2022-09-14 04:40:02 [info     / 123kB]   - wake reason: unknown
2022-09-14 04:40:02 [debug    / 121kB]   - turn on activity led
2022-09-14 04:40:03 [info     / 123kB] > 1 cache files need uploading
2022-09-14 04:40:04 [info     / 107kB] > connecting to wifi network 'Patternweb'
2022-09-14 04:40:06 [debug    / 101kB]   - connecting
2022-09-14 04:40:09 [debug    /  86kB]   - connecting
helgibbons commented 1 year ago

Nope - ran for a couple of hours and then fell over at:

2022-09-16 12:00:02 [debug    / 115kB] > performing startup
2022-09-16 12:00:02 [info     / 123kB]   - wake reason: unknown
2022-09-16 12:00:02 [debug    / 121kB]   - turn on activity led
MrDrem commented 1 year ago

So, I ran my weather board recording every minute, but storing values locally rather than uploading them, and it has been running perfectly for over 12 hours (or over 720 records).

This makes me currently suspect that it's something in the upload code that is wonky.

MrDrem commented 1 year ago

I've now had the station running for a full 24 hours storing records locally, and it's not missed a beat.

Here's the data file - 2022-09-19.txt (there's more than an hour on the previous file, I keep forgetting it uses UTC for date/time)

MrDrem commented 1 year ago

I've switched to a local influxDB (1.8) instance, and my uploads are now working fine. Don't know how this helps, but it might point to something.

@helgibbons is your influxDB instance local or cloud? I'd guess local from your helpful guide that I followed.

helgibbons commented 1 year ago

I've got both on the go now :D

Still running into the occasional issue where an upload fails and leaves the board awake and the wifi 'stuck on', tanking the battery. My two boards going into Influx Cloud did this at around the same time last night (after running fine for weeks), so I'm wondering if the current code is getting confused somewhere if it encounters flaky wifi or has problems connecting to an endpoint.

MrDrem commented 1 year ago

I've now been running with a connection to a local InfluxDB rather than web services since the end of September, and this issue hasn't occurred since. I suspect that the fault that causes the issue is somewhere in the upload code. I've had one instance of the RTC slipping back in time and I've updated the code to check the time every 24 hours (or 48 uploads)

ZodiusInfuser commented 1 year ago

@MrDrem Thank you very much for the thorough investigation into this, especially through the period of our silence!!

Am I right in understanding then that you think the issue is no longer with the wind sensor itself, but rather the uploading of data to influxDB? I will admit I have only tested with local logging and adafruitio so far. May need @helgibbons's assistance to set up an InfluxDB instance for me to debug these issues.

MrDrem commented 1 year ago

@ZodiusInfuser I'm fairly sure that the issue isn't with the hardware or sensors, and am pretty sure that it's with the uploads.

I started out uploading to adafruit.io initially, and that's where I was seeing the dropouts, and the RTC stopping. I then switched to a local install of influxDB, and haven't had the same issue since (I had the odd issue of the RTC slipping, and can't work out when or why it happened as I didn't notice for a while).

I've seen others say they've had what look like similar issues with MQTT uploads, which I think that the Adafruit.io uploads are a form of too.

I found that increasing logging and uploads to once per minute drove it to occur faster, so you might want to consider that for testing with.

Maybe I've cursed it by saying that though, as it fell over at ~18:00 last night - I'll go investigate why shortly ;)