"RuntimeError: I2C slave address was NACK'd" with SCD30 on MCP2221A breakout

[rpavlik]

Might be related to adafruit/Adafruit_Blinka#262 but comment there said different boards should have different issues.

I'm using the sample from the SCD30 library docs:

import board
import busio
import adafruit_scd30

i2c = busio.I2C(board.SCL, board.SDA)
scd = adafruit_scd30.SCD30(i2c)

while True:
    # since the measurement interval is long (2+ seconds) we check for new data before reading
    # the values, to ensure current readings.
    if scd.data_available:
        print("Data Available!")
        print("eCO2:", scd.eCO2, "PPM")
        print("Temperature:", scd.temperature, "degrees C")
        print("Humidity:", scd.relative_humidity, "%%rH")
        print("")
        print("Waiting for new data...")
        print("")

    time.sleep(0.5)

Windows 10 2004, Adafruit MCP2221 breakout, Adafruit SCD30 breakout, Python 3.9.1 (installed with Scoop). The SCD30 works fine on the QT-Py I have here.

ryanp@RYZENSHINE-WIN   venv  E:\src-ssd\scd30                                                                                                                                                                                                   [09:57]
❯ $env:BLINKA_MCP2221=1
ryanp@RYZENSHINE-WIN   venv  E:\src-ssd\scd30                                                                                                                                                                                                   [09:57]
❯ & e:/src-ssd/scd30/venv/Scripts/python.exe e:/src-ssd/scd30/test.py
Traceback (most recent call last):
  File "e:\src-ssd\scd30\test.py", line 8, in <module>
    scd = adafruit_scd30.SCD30(i2c)
  File "E:\src-ssd\scd30\venv\lib\site-packages\adafruit_scd30.py", line 66, in __init__
    self.measurement_interval = 2
  File "E:\src-ssd\scd30\venv\lib\site-packages\adafruit_scd30.py", line 93, in measurement_interval  
    self._send_command(_CMD_SET_MEASUREMENT_INTERVAL, value)
  File "E:\src-ssd\scd30\venv\lib\site-packages\adafruit_scd30.py", line 220, in _send_command        
    i2c.write(self._buffer, end=end_byte)
  File "E:\src-ssd\scd30\venv\lib\site-packages\adafruit_bus_device\i2c_device.py", line 102, in write
    self.i2c.writeto(self.device_address, buf, start=start, end=end)
  File "E:\src-ssd\scd30\venv\lib\site-packages\busio.py", line 115, in writeto
    return self._i2c.writeto(address, memoryview(buffer)[start:end], stop=stop)
  File "E:\src-ssd\scd30\venv\lib\site-packages\adafruit_blinka\microcontroller\mcp2221\i2c.py", line 19, in writeto
    self._mcp2221.i2c_writeto(address, buffer, start=start, end=end)
  File "E:\src-ssd\scd30\venv\lib\site-packages\adafruit_blinka\microcontroller\mcp2221\mcp2221.py", line 315, in i2c_writeto
    self._i2c_write(0x90, address, buffer, start, end)
  File "E:\src-ssd\scd30\venv\lib\site-packages\adafruit_blinka\microcontroller\mcp2221\mcp2221.py", line 233, in _i2c_write
    raise RuntimeError("I2C slave address was NACK'd")
RuntimeError: I2C slave address was NACK'd

[ladyada]

i wonder if its because the computer is so much faster, there was an inadvertant delay between some commands. do you want to try editing adafruit_scd30.py to add some time.sleep(0.01)'s in the init code?

[rpavlik]

Hmm. even sleep(1) between self.reset() and self.measurement_interval = 2 doesn't help.

[ladyada]

what if you dont reset?

[rpavlik]

Then it works!

Data Available!
eCO2: 922.3878173828125 PPM
Temperature: 24.009307861328125 degrees C
Humidity: 27.9052734375 %%rH

Waiting for new data...

All I did was comment out the self.reset()

[ladyada]

hmm ok let me move this to the SCD30 library because it at least lives there @caternuson do you have an SCD30 to take a look at why resetting is so unhappy on cPy?

[rpavlik]

thanks - looks like it needs some of the sleeps in addition to no reset: needs 0.01 before self.ambient_pressure = ambient_pressure too.

Init looks like this for me now:

    def __init__(self, i2c_bus, ambient_pressure=0, address=SCD30_DEFAULT_ADDR):
        if ambient_pressure != 0:
            if ambient_pressure < 700 or ambient_pressure > 1200:
                raise AttributeError("`ambient_pressure` must be from 700-1200 mBar")

        self.i2c_device = i2c_device.I2CDevice(i2c_bus, address)
        self._buffer = bytearray(18)
        self._crc_buffer = bytearray(2)
        # self.reset()

        self.measurement_interval = 2
        self.self_calibration_enabled = True
        # sets ambient pressure and starts continuous measurements
        sleep(0.01)
        self.ambient_pressure = ambient_pressure

        # cached readings
        self._temperature = None
        self._relative_humidity = None
        self._e_co2 = None

[ladyada]

could you submit a PR for your changes? that will give us somewhere to start from

[rpavlik]

OK, filed as #3, split into two commits since the sleep seems innocuous but the removal of reset is probably not something to merge

[ladyada]

awesome, thank you - you can always just use your hacked version till we get to checking this out :)

[rpavlik]

yep, thanks for the super speedy response!

[caternuson]

@ladyada Sorry, don't have one yet. Will put in a NOTIFY ME to make sure and get one at some point though.

[rpavlik]

I thought I'd throw an analyzer on the bus. Can't find my fx2-based one, but found my Beagle. Looks like something squirrely is happening during the reset: the Beagle thinks there's a write to a device at address 23: (this trace shows a few attempts to fire it up with differing timing, etc. and thus has multiple reset calls in it)

When I omit the "reset" it looks perfectly fine, just communications to and from address 61.

Looks like there's still a glitch on the bus during reset from the qt-py (running circuit python 6.1.0 rc0), but none of those bogus address 0x23 writes.

Wonder if the SCD30 is doing something weird to the bus during reset that's confusing the mcp2221. That will probably require a DSO or at least a analog-capable Saleae, not just my digital debug tools. I saw the datasheet mentioned clock stretching in the SCD30, wonder if that's relevant here.

[caternuson]

I've repeated all of the same behavior you have documented above. Including the corrupt transaction(s) and subsequent attempt to write to 0x23.

Here's what that corrupt transaction looks like:

It happens even with the QT Py, so it's coming from the SCD30, not the MCP2221.

For the QT Py, it just keeps on trucking:

For the MCP2221, it tries to talk to 0x23 and gets a NAK:

GREEN = self.reset(), RED = "corrupt transaction", CYAN = self.measurement_interval = 2

I hacked in a reset of the MCP2221 for each call in the library's _send_command():

        with self.i2c_device as i2c:
            i2c.i2c._i2c._mcp2221._reset()
            i2c.write(self._buffer, end=end_byte)

and that got it working.

$ python3
Python 3.6.9 (default, Oct  8 2020, 12:12:24) 
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import board
>>> import adafruit_scd30
>>> scd = adafruit_scd30.SCD30(board.I2C())
>>> scd.CO2
995.8560180664062
>>> 

So it does seem like that odd I2C blurp is confusing the MCP2221 so that it's next real transaction is all screwed up.

The investigation continues. Just dumping info as I go.

[rpavlik]

Interesting... Very interesting. The SCD30 has some onboard micro with firmware, because there's some way to read the version out, iirc, but I don't know more about it than that. Bummer that such a fancy gadget has such a weird protocol quirk. Good luck with the debugging!

[caternuson]

Luckily the MCP2221 supports clock stretching:

Here's what the MCP2221 self.measurement_interval = 2 call looks like with the reset hack:

Basically, what one would expect.

[caternuson]

Just more info dumping without any resolution.

Commented out everything in SCD30.__init__() starting with call to reset() so I could at least create an instance. Then did things via REPL so state of MCP2221 could be check at various points.

$ python3
Python 3.8.5 (default, Jul 28 2020, 12:59:40) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import board
>>> i2c = board.I2C()
>>> mcp = i2c._i2c._mcp2221
>>> s1 = mcp._i2c_status()
>>> import adafruit_scd30
>>> scd = adafruit_scd30.SCD30(i2c)
>>> s2 = mcp._i2c_status()
>>> scd.reset()
>>> s3 = mcp._i2c_status()

Created a quick little report dumper helper:

>>> def i2c_status_report(resp):
...     print("State machine state value = ",resp[8])
...     print("Requested xfer bytes = ",resp[10], resp[9])
...     print("Number of bytes xfer'd = ", resp[12], resp[11])
...     print("Data buffer counter = ", resp[13])
...     print("Speed divider = ", resp[14])
...     print("Timeout = ", resp[15])
...     print("Address = ", resp[17], resp[16])
...     print("SCL = ", resp[22])
...     print("SDA = ", resp[23])
...     print("Edge = ", resp[24])
...     print("Read pending = ", resp[25])
... 
>>> 

and used that.

Before creating SCD30

>>> i2c_status_report(s1)
State machine state value =  0
Requested xfer bytes =  0 0
Number of bytes xfer'd =  0 0
Data buffer counter =  0
Speed divider =  117
Timeout =  0
Address =  0 0
SCL =  1
SDA =  1
Edge =  0
Read pending =  0

After creating SCD30

>>> i2c_status_report(s2)
State machine state value =  0
Requested xfer bytes =  0 0
Number of bytes xfer'd =  0 0
Data buffer counter =  0
Speed divider =  117
Timeout =  0
Address =  0 0
SCL =  1
SDA =  1
Edge =  0
Read pending =  0

After calling reset()

>>> i2c_status_report(s3)
State machine state value =  0
Requested xfer bytes =  0 2
Number of bytes xfer'd =  0 2
Data buffer counter =  2
Speed divider =  117
Timeout =  0
Address =  0 194
SCL =  1
SDA =  1
Edge =  0
Read pending =  0
>>> 

The internal state seems OK at each step. Not sure if Data buffer counter is expected to be 0 at the end? Also not sure about that residual value in Address. Here's the extent of the documentation on that:

[caternuson]

Ugh. There may be multiple things happening. There's the odd SDA/SCL twitch after an SCD30 reset that causes the MCP2221 to use an incorrect address. That can be fixed with an MCP2221 reset, which has been added in for current testing in SCD30.__init__() (hack):

        self.reset()
        sleep(0.1)
        i2c_bus._i2c._mcp2221._reset()

There also seems to be something weird with the call to set self calibration. Here's the trace: RED = self.measurement_interval = 2 GREEN = self.self_calibration_enabled = True CYAN = self.ambient_pressure = ambient_pressure (it fails there, and it does the same thing repeatably)

Setting self.self_calibration_enabled = False instead, we can get around it: (it fails first time changing from True to False, but works with all other subsequent runs)

[caternuson]

OK, at least have something that seems to work and can PR to open up for discussion. Checkout #9 in all its glory.

[caternuson]

Note to future self - Wish this could have been done more elegantly somehow. Like having the MCP2221 self check its state and resetting itself if needed. Investigated that but could not find any reliable way to do the actual detection.

[rpavlik]

Thanks for your hard work on this!