search

sparkfun / OpenLog_Artemis

The OpenLog Artemis is an open source datalogger the comes preprogrammed to automatically log IMU, GPS, serial data, and various pressure, humidity, and distance sensors. All without writing a single line of code!
https://www.sparkfun.com/products/15846
Other
86 stars 45 forks source link

(Re)Investigate initialization and data logging issues on SCD30 #67

Closed PaulZC closed 3 years ago

PaulZC commented 3 years ago

See these forum posts for details: https://forum.sparkfun.com/viewtopic.php?p=222428#p222428 https://forum.sparkfun.com/viewtopic.php?p=221750#p221750

Background: The power-on delay for the SCD30 was improved in this commit - the delay was extended to 5 seconds to allow the C)2 sensor to warm up and start correctly. But user jimbrookline reports that the SCD30 is failing to be detected?

jimbrookline commented 3 years ago

I ran the V1.8 OLA code with debug on. No additional error messages reported: (but debug on so got this additional line: detectQwiicDevices: something detected at address 0x61)

Artemis OpenLog v1.8 Logging to: dataLog00031.TXT Finding the next available log file. This could take a long time if the SD card contains many existing log files. Logging to: serialLog00000.TXT SD card online Data logging online Serial logging online IMU online detectQwiicDevices started Identifying Qwiic Muxes... Identifying Qwiic Devices... detectQwiicDevices: something detected at address 0x61 Known I2C address but device failed identification at address 0x61 Autodetect complete beginQwiicDevices: No devices detected printOnlineDevice: No devices detected rtcDate,rtcTime,aX,aY,aZ,gX,gY,gZ,mX,mY,mZ,imu_degC,output_Hz, 01/02/2000,17:58:02.52,-566.41,-129.88,-802.25,2.11,-1.29,3.19,-49.50,47.25,-30.60,28.17,333.33, 01/02/2000,17:58:02.62,-592.77,-130.86,-788.09,-0.92,-1.72,2.30,-48.00,46.50,-31.05,28.37,19.42, 01/02/2000,17:58:02.72,-587.89,-145.02,-803.22,-1.51,-1.18,-1.40,-49.35,46.95,-30.45,28.51,14.78,

PaulZC commented 3 years ago

Thanks Jim, I suspect Paul (vha) has hit the nail on the head with the .begin after power-up issue. I will implement a fix as soon as I can. All the best, Paul

PaulZC commented 3 years ago

More information here. There may be mux start-up issues too? https://forum.sparkfun.com/viewtopic.php?p=222486#p222486

PaulZC commented 3 years ago

Hi Jim (@jimbrookline ), Please see this update on the Forum: https://forum.sparkfun.com/viewtopic.php?p=222827#p222827 Can you please tell me if this has fixed your issue? Thanks! I'll leave this issue open until the full release of v1.9 goes live. All the best, Paul

PaulZC commented 3 years ago

Hi Jim (@jimbrookline ),

Thanks for the update on the forum.

I think I've managed to replicate your issue. For me, it is less than one time in ten (one SCD30 via a mux) but I have captured it:

08:18:12.907 -> Artemis OpenLog v1.9
08:18:13.384 -> Logging to: dataLog00083.TXT
08:18:14.233 -> Logging to: serialLog00006.TXT
08:18:14.658 -> SD card online
08:18:14.658 -> Data logging online
08:18:14.658 -> Serial logging online
08:18:14.658 -> IMU offline
08:18:14.658 -> detectQwiicDevices started
08:18:14.658 -> Identifying Qwiic Muxes...
08:18:14.658 -> detectQwiicDevices: something detected at address 0x72
08:18:14.658 -> detectQwiicDevices: multiplexer found at address 0x72
08:18:14.658 -> detectQwiicDevices: found 1 multiplexer
08:18:14.658 -> beginQwiicDevices: attempting to begin deviceType Multiplexer at address 0x72 using mux address 0x00 and port number 0
08:18:14.658 -> beginQwiicDevices: device is online
08:18:14.658 -> Identifying Qwiic Devices...
08:18:14.658 -> detectQwiicDevices: something detected at address 0x72
08:18:14.658 -> Multiplexers found. Scanning sub nets...
08:18:14.658 -> detectQwiicDevices: scanning the ports of multiplexer 0
08:18:14.705 -> detectQwiicDevices: scanning port number 0 on multiplexer 0
08:18:14.705 -> detectQwiicDevices: skipping device address 0x72 because we found one on the main branch
08:18:14.705 -> detectQwiicDevices: scanning port number 1 on multiplexer 0
08:18:14.705 -> detectQwiicDevices: skipping device address 0x72 because we found one on the main branch
08:18:14.705 -> detectQwiicDevices: scanning port number 2 on multiplexer 0
08:18:14.752 -> detectQwiicDevices: skipping device address 0x72 because we found one on the main branch
08:18:14.752 -> detectQwiicDevices: scanning port number 3 on multiplexer 0
08:18:14.752 -> detectQwiicDevices: skipping device address 0x72 because we found one on the main branch
08:18:14.752 -> detectQwiicDevices: scanning port number 4 on multiplexer 0
08:18:14.799 -> detectQwiicDevices: skipping device address 0x72 because we found one on the main branch
08:18:14.799 -> detectQwiicDevices: scanning port number 5 on multiplexer 0
08:18:15.506 -> Known I2C address but device failed identification at address 0x61
08:18:15.506 -> detectQwiicDevices: skipping device address 0x72 because we found one on the main branch
08:18:15.506 -> detectQwiicDevices: scanning port number 6 on multiplexer 0
08:18:15.506 -> detectQwiicDevices: skipping device address 0x72 because we found one on the main branch
08:18:15.506 -> detectQwiicDevices: scanning port number 7 on multiplexer 0
08:18:15.506 -> detectQwiicDevices: skipping device address 0x72 because we found one on the main branch
08:18:15.600 -> Autodetect complete
08:18:15.600 -> beginQwiicDevices: attempting to begin deviceType Multiplexer at address 0x72 using mux address 0x00 and port number 0
08:18:15.600 -> beginQwiicDevices: device is **NOT** online
08:18:15.874 -> Multiplexer failed to respond
08:18:15.874 -> Device count: 0
08:18:15.874 -> rtcDate,rtcTime,
08:18:15.968 -> 02/02/2021,08:18:19.33,

Now, here's the bad news... I captured the I2C data at the same time:

image

This is the OLA calling testDevice for address 0x61. It starts by trying to .begin an MCP9600 by reading its device ID. The OLA does a write to address 0x61 then writes 0x20 (the address of the MCP9600's device ID register). It then does a read from address 0x61 and the SCD30 responds with 0xDDDD. What should happen next is that the OLA should recognize that this isn't an MCP9600 and go on to trying to .begin a SCD30. The problem is that the I2C bus freezes at this point and no more traffic is seen. The 0xDDDD is the last activity on the bus...

Unfortunately I have seen this before. I see the same issue with the BNO080 IMU. It too somehow manages to crash the I2C bus. It's the reason why we haven't added support on the OLA for the BNO080. It crashes the Artemis hardware at a very deep level, causing all subsequent I2C reads and writes to timeout. I have dug right into the core and I can see where it happens, but the error takes place in hardware for reasons I just don't understand and for which I don't have a fix... This might have been fixed in v2.0 of the new Artemis Mbed core, but the core is not complete yet and migrating to v2.0 will not happen soon.

I'm going to dig into this more today... I've got some ideas about possible work-arounds (but not cures unfortunately)...

Cheers, Paul

PaulZC commented 3 years ago

Hi Jim (@jimbrookline ), The v1.9 BETA Binary now includes a work-around. If you open the hidden "d" debug menu from the main menu, you will now see an option to "Reset On Zero Device Count". If the I2C bus hangs, the OLA should think there are no qwiic devices connected - or none that have been detected and begun successfully. This new option makes the OLA perform a hardware reset if that happens. It clears the bug with my one SCD30 connected. We just need to check if it works for you and your two SCD30s. All the best, Paul

jimbrookline commented 3 years ago

Thanks, I will try your suggestion later today. I notice that you have to spend an inordinate amount of time babying the MCP9600. (at some point I might be able to delete those calls on my version, sounds like that could eliminate the issue).

Jim

jimbrookline commented 3 years ago

Hi Paul,

This seems to work! I watched it fail and then go through the HW reset and then it brought both devices on. Wasn’t able to see an event where it failed twice (and went into reset twice), so can’t say it will work in that case. Since this is an intermittent problem, as I was trying it without the reset it was failing only about 1 in 10 times, but I was able to show the new code worked for 2 fails.

Thanks for your continued assistance!

Jim

PaulZC commented 3 years ago

Thank you Jim, I suspect that's the best I can do for now - unless I'm ever able to resolve the low-level hardware/core issue behind this. I'm going to mark this as 'resolved' - even though you only really have a work-around, not a true solution. Thank you for working with us on this. Best wishes, Paul

PaulZC commented 3 years ago

'Resolved' in Version 1.9. Closing...

jimbrookline commented 3 years ago

Appreciate your help and tenacity!

From: Paul notifications@github.com Sent: Wednesday, February 3, 2021 4:48 AM To: sparkfun/OpenLog_Artemis OpenLog_Artemis@noreply.github.com Cc: jimbrookline jim.doscher@gmail.com; Mention mention@noreply.github.com Subject: Re: [sparkfun/OpenLog_Artemis] (Re)Investigate initialization and data logging issues on SCD30 (#67)

Thank you Jim, I suspect that's the best I can do for now - unless I'm ever able to resolve the low-level hardware/core issue behind this. I'm going to mark this as 'resolved' - even though you only really have a work-around, not a true solution. Thank you for working with us on this. Best wishes, Paul

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/sparkfun/OpenLog_Artemis/issues/67#issuecomment-772376741 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AR2SVXJV3BFMIPM4X4VGR6DS5ELTNANCNFSM4WG4NJLA . https://github.com/notifications/beacon/AR2SVXOFJBPLFQB2ODZ7KLLS5ELTNA5CNFSM4WG4NJLKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOFYEYRJI.gif