search

NVIDIA-AI-IOT / jetracer

An autonomous AI racecar using NVIDIA Jetson Nano
MIT License
1.06k stars 319 forks source link

tegra-i2c pio timed out #128

Open nighi opened 2 years ago

nighi commented 2 years ago

Hi, I have a waveshare JetRacer AI Kit 4GB, set up with the newest image from this repo and the jetracer repo from the waveshare fork.

I can control steering and throttle from the basic_motion notebook, but often a few seconds after activating the motors, the car dies. No hdmi output, black oLed and not reachable neither via jupyter nor via ssh, but the fan is still spinning. The behavior doesn't change with or without power supply, and I couldn't reproduce it in response to a special action. Sometimes it occurs right after setting throttle, sometimes the car drives for a couple seconds.

Something I noticed is that running the car constructor seems to last very long. 

car = NvidiaRacecar(steering_channel=0, throttle_channel=0)

And after looking at \var\log\syslog I get following lines:

jetson@nano-4gb-jp451:~$ tail -f /var/log/syslog
Apr  5 04:12:24 nano-4gb-jp451 sh[4210]: [I 04:12:24.101 LabApp] Kernel started: 4a01fbe1-f89c-4030-bfd1-4afd096486af, name: python3
Apr  5 04:12:37 nano-4gb-jp451 kernel: [  227.701164] tegra-i2c 7000c400.i2c: pio timed out addr: 0x40 tlen:0 rlen:0
Apr  5 04:12:37 nano-4gb-jp451 kernel: [  227.709166] tegra-i2c 7000c400.i2c: --- register dump for debugging ----
Apr  5 04:12:37 nano-4gb-jp451 kernel: [  227.716101] tegra-i2c 7000c400.i2c: I2C_CNFG - 0x22c00
Apr  5 04:12:37 nano-4gb-jp451 kernel: [  227.721353] tegra-i2c 7000c400.i2c: I2C_PACKET_TRANSFER_STATUS - 0x10001
Apr  5 04:12:37 nano-4gb-jp451 kernel: [  227.729186] tegra-i2c 7000c400.i2c: I2C_FIFO_CONTROL - 0xe0
Apr  5 04:12:37 nano-4gb-jp451 kernel: [  227.734824] tegra-i2c 7000c400.i2c: I2C_FIFO_STATUS - 0x800080
Apr  5 04:12:37 nano-4gb-jp451 kernel: [  227.740796] tegra-i2c 7000c400.i2c: I2C_INT_MASK - 0x7c
Apr  5 04:12:37 nano-4gb-jp451 kernel: [  227.746138] tegra-i2c 7000c400.i2c: I2C_INT_STATUS - 0x2
Apr  5 04:12:37 nano-4gb-jp451 kernel: [  227.751491] tegra-i2c 7000c400.i2c: i2c transfer timed out addr: 0x40
Apr  5 04:12:48 nano-4gb-jp451 kernel: [  237.940955] tegra-i2c 7000c400.i2c: pio timed out addr: 0x60 tlen:0 rlen:0
Apr  5 04:12:48 nano-4gb-jp451 kernel: [  237.948888] tegra-i2c 7000c400.i2c: --- register dump for debugging ----
Apr  5 04:12:48 nano-4gb-jp451 kernel: [  237.956871] tegra-i2c 7000c400.i2c: I2C_CNFG - 0x22c00
Apr  5 04:12:48 nano-4gb-jp451 kernel: [  237.962429] tegra-i2c 7000c400.i2c: I2C_PACKET_TRANSFER_STATUS - 0x10001
Apr  5 04:12:48 nano-4gb-jp451 kernel: [  237.970505] tegra-i2c 7000c400.i2c: I2C_FIFO_CONTROL - 0xe0
Apr  5 04:12:48 nano-4gb-jp451 kernel: [  237.976860] tegra-i2c 7000c400.i2c: I2C_FIFO_STATUS - 0x800080
Apr  5 04:12:48 nano-4gb-jp451 kernel: [  237.982922] tegra-i2c 7000c400.i2c: I2C_INT_MASK - 0x7c
Apr  5 04:12:48 nano-4gb-jp451 kernel: [  237.988274] tegra-i2c 7000c400.i2c: I2C_INT_STATUS - 0x2
Apr  5 04:12:48 nano-4gb-jp451 kernel: [  237.993738] tegra-i2c 7000c400.i2c: i2c transfer timed out addr: 0x60

This can be reproduced every time the NvidiaRacecar constructor is called, and the message is always the same (Address 0x40 and 0x60).

Running

sudo i2cdetect -y -r 1

returns 

     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:          -- -- -- -- -- -- -- -- -- -- -- -- -- 
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
30: -- -- -- -- -- -- -- -- -- -- -- -- 3c -- -- -- 
40: 40 41 -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
60: 60 -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
70: 70 -- -- -- -- -- -- --

which seems okay for my understanding.

Thank you for any help!

JihunDev commented 1 year ago

Hi @nighi Same issue occurred. If you solve it, please share.

nighi commented 1 year ago

Puh, unfortunately I haven't access to a jetracer anymore. If I remember it correctly I could fix this issue by choosing a different version of the image. But don't ask me which version has worked. Sorry

KSU-Chad commented 9 months ago

Similar issue, cannot find I2C device 0x60 (Using Pre-build image based on Jetpack4.5 from waveshare wiki)

from jetracer.nvidia_racecar import NvidiaRacecar

car = NvidiaRacecar()

results in Timeout Error

---------------------------------------------------------------------------
TimeoutError                              Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/adafruit_circuitpython_busdevice-5.0.6-py3.6.egg/adafruit_bus_device/i2c_device.py in __probe_for_device(self)
    153         try:
--> 154             self.i2c.writeto(self.device_address, b"")
    155         except OSError:

/usr/local/lib/python3.6/dist-packages/Adafruit_Blinka-6.10.0-py3.6.egg/busio.py in writeto(self, address, buffer, start, end, stop)
    157             return self._i2c.writeto(address, memoryview(buffer)[start:end], stop=stop)
--> 158         return self._i2c.writeto(address, buffer, stop=stop)
    159 

/usr/local/lib/python3.6/dist-packages/Adafruit_Blinka-6.10.0-py3.6.egg/adafruit_blinka/microcontroller/generic_linux/i2c.py in writeto(self, address, buffer, start, end, stop)
     48             end = len(buffer)
---> 49         self._i2c_bus.write_bytes(address, buffer[start:end])
     50 

/usr/local/lib/python3.6/dist-packages/Adafruit_PureIO-1.1.8-py3.6.egg/Adafruit_PureIO/smbus.py in write_bytes(self, addr, buf)
    307         self._select_device(addr)
--> 308         self._device.write(buf)
    309 

TimeoutError: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/adafruit_circuitpython_busdevice-5.0.6-py3.6.egg/adafruit_bus_device/i2c_device.py in __probe_for_device(self)
    159                 result = bytearray(1)
--> 160                 self.i2c.readfrom_into(self.device_address, result)
    161             except OSError:

/usr/local/lib/python3.6/dist-packages/Adafruit_Blinka-6.10.0-py3.6.egg/busio.py in readfrom_into(self, address, buffer, start, end)
    147         stop = True  # remove for efficiency later
--> 148         return self._i2c.readfrom_into(address, buffer, stop=stop)
    149 

/usr/local/lib/python3.6/dist-packages/Adafruit_Blinka-6.10.0-py3.6.egg/adafruit_blinka/microcontroller/generic_linux/i2c.py in readfrom_into(self, address, buffer, start, end, stop)
     55 
---> 56         readin = self._i2c_bus.read_bytes(address, end - start)
     57         for i in range(end - start):

/usr/local/lib/python3.6/dist-packages/Adafruit_PureIO-1.1.8-py3.6.egg/Adafruit_PureIO/smbus.py in read_bytes(self, addr, number)
    178         self._select_device(addr)
--> 179         return self._device.read(number)
    180 

OSError: [Errno 121] Remote I/O error

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-3-2b783e4b0c4f> in <module>
      1 from jetracer.nvidia_racecar import NvidiaRacecar
      2 
----> 3 car = NvidiaRacecar()

/usr/local/lib/python3.6/dist-packages/jetracer-0.0.0-py3.6.egg/jetracer/nvidia_racecar.py in __init__(self, *args, **kwargs)
     16         super(NvidiaRacecar, self).__init__(*args, **kwargs)
     17         self.kit = ServoKit(channels=16, address=self.i2c_address1)
---> 18         self.motor = ServoKit(channels=16, address=self.i2c_address2)
     19         self.motor._pca.frequency = 1600
     20         self.steering_motor = self.kit.continuous_servo[self.steering_channel]

/usr/local/lib/python3.6/dist-packages/adafruit_circuitpython_servokit-1.3.4-py3.6.egg/adafruit_servokit.py in __init__(self, channels, i2c, address, reference_clock_speed, frequency)
     79             i2c = board.I2C()
     80         self._pca = PCA9685(
---> 81             i2c, address=address, reference_clock_speed=reference_clock_speed
     82         )
     83         self._pca.frequency = frequency

/usr/local/lib/python3.6/dist-packages/adafruit_circuitpython_pca9685-3.3.6-py3.6.egg/adafruit_pca9685.py in __init__(self, i2c_bus, address, reference_clock_speed)
    118 
    119     def __init__(self, i2c_bus, *, address=0x40, reference_clock_speed=25000000):
--> 120         self.i2c_device = i2c_device.I2CDevice(i2c_bus, address)
    121         self.channels = PCAChannels(self)
    122         """Sequence of 16 `PWMChannel` objects. One for each channel."""

/usr/local/lib/python3.6/dist-packages/adafruit_circuitpython_busdevice-5.0.6-py3.6.egg/adafruit_bus_device/i2c_device.py in __init__(self, i2c, device_address, probe)
     48 
     49         if probe:
---> 50             self.__probe_for_device()
     51 
     52     def readinto(self, buf, *, start=0, end=None):

/usr/local/lib/python3.6/dist-packages/adafruit_circuitpython_busdevice-5.0.6-py3.6.egg/adafruit_bus_device/i2c_device.py in __probe_for_device(self)
    161             except OSError:
    162                 # pylint: disable=raise-missing-from
--> 163                 raise ValueError("No I2C device at address: 0x%x" % self.device_address)
    164                 # pylint: enable=raise-missing-from
    165         finally:

ValueError: No I2C device at address: 0x60
chentyra commented 4 months ago

Similar issue, cannot find I2C device 0x60 (Using Pre-build image based on Jetpack4.5 from waveshare wiki)

from jetracer.nvidia_racecar import NvidiaRacecar

car = NvidiaRacecar()

results in Timeout Error

---------------------------------------------------------------------------
TimeoutError                              Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/adafruit_circuitpython_busdevice-5.0.6-py3.6.egg/adafruit_bus_device/i2c_device.py in __probe_for_device(self)
    153         try:
--> 154             self.i2c.writeto(self.device_address, b"")
    155         except OSError:

/usr/local/lib/python3.6/dist-packages/Adafruit_Blinka-6.10.0-py3.6.egg/busio.py in writeto(self, address, buffer, start, end, stop)
    157             return self._i2c.writeto(address, memoryview(buffer)[start:end], stop=stop)
--> 158         return self._i2c.writeto(address, buffer, stop=stop)
    159 

/usr/local/lib/python3.6/dist-packages/Adafruit_Blinka-6.10.0-py3.6.egg/adafruit_blinka/microcontroller/generic_linux/i2c.py in writeto(self, address, buffer, start, end, stop)
     48             end = len(buffer)
---> 49         self._i2c_bus.write_bytes(address, buffer[start:end])
     50 

/usr/local/lib/python3.6/dist-packages/Adafruit_PureIO-1.1.8-py3.6.egg/Adafruit_PureIO/smbus.py in write_bytes(self, addr, buf)
    307         self._select_device(addr)
--> 308         self._device.write(buf)
    309 

TimeoutError: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/adafruit_circuitpython_busdevice-5.0.6-py3.6.egg/adafruit_bus_device/i2c_device.py in __probe_for_device(self)
    159                 result = bytearray(1)
--> 160                 self.i2c.readfrom_into(self.device_address, result)
    161             except OSError:

/usr/local/lib/python3.6/dist-packages/Adafruit_Blinka-6.10.0-py3.6.egg/busio.py in readfrom_into(self, address, buffer, start, end)
    147         stop = True  # remove for efficiency later
--> 148         return self._i2c.readfrom_into(address, buffer, stop=stop)
    149 

/usr/local/lib/python3.6/dist-packages/Adafruit_Blinka-6.10.0-py3.6.egg/adafruit_blinka/microcontroller/generic_linux/i2c.py in readfrom_into(self, address, buffer, start, end, stop)
     55 
---> 56         readin = self._i2c_bus.read_bytes(address, end - start)
     57         for i in range(end - start):

/usr/local/lib/python3.6/dist-packages/Adafruit_PureIO-1.1.8-py3.6.egg/Adafruit_PureIO/smbus.py in read_bytes(self, addr, number)
    178         self._select_device(addr)
--> 179         return self._device.read(number)
    180 

OSError: [Errno 121] Remote I/O error

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-3-2b783e4b0c4f> in <module>
      1 from jetracer.nvidia_racecar import NvidiaRacecar
      2 
----> 3 car = NvidiaRacecar()

/usr/local/lib/python3.6/dist-packages/jetracer-0.0.0-py3.6.egg/jetracer/nvidia_racecar.py in __init__(self, *args, **kwargs)
     16         super(NvidiaRacecar, self).__init__(*args, **kwargs)
     17         self.kit = ServoKit(channels=16, address=self.i2c_address1)
---> 18         self.motor = ServoKit(channels=16, address=self.i2c_address2)
     19         self.motor._pca.frequency = 1600
     20         self.steering_motor = self.kit.continuous_servo[self.steering_channel]

/usr/local/lib/python3.6/dist-packages/adafruit_circuitpython_servokit-1.3.4-py3.6.egg/adafruit_servokit.py in __init__(self, channels, i2c, address, reference_clock_speed, frequency)
     79             i2c = board.I2C()
     80         self._pca = PCA9685(
---> 81             i2c, address=address, reference_clock_speed=reference_clock_speed
     82         )
     83         self._pca.frequency = frequency

/usr/local/lib/python3.6/dist-packages/adafruit_circuitpython_pca9685-3.3.6-py3.6.egg/adafruit_pca9685.py in __init__(self, i2c_bus, address, reference_clock_speed)
    118 
    119     def __init__(self, i2c_bus, *, address=0x40, reference_clock_speed=25000000):
--> 120         self.i2c_device = i2c_device.I2CDevice(i2c_bus, address)
    121         self.channels = PCAChannels(self)
    122         """Sequence of 16 `PWMChannel` objects. One for each channel."""

/usr/local/lib/python3.6/dist-packages/adafruit_circuitpython_busdevice-5.0.6-py3.6.egg/adafruit_bus_device/i2c_device.py in __init__(self, i2c, device_address, probe)
     48 
     49         if probe:
---> 50             self.__probe_for_device()
     51 
     52     def readinto(self, buf, *, start=0, end=None):

/usr/local/lib/python3.6/dist-packages/adafruit_circuitpython_busdevice-5.0.6-py3.6.egg/adafruit_bus_device/i2c_device.py in __probe_for_device(self)
    161             except OSError:
    162                 # pylint: disable=raise-missing-from
--> 163                 raise ValueError("No I2C device at address: 0x%x" % self.device_address)
    164                 # pylint: enable=raise-missing-from
    165         finally:

ValueError: No I2C device at address: 0x60

I got the same error before updating the pre-installed repository. I have updated the package following the instructions on the Waveshare website:

cd jetracer
git checkout ws/pro
sudo python3 setup.py install

After these steps the problem no longer occurred.