I2C regression, last working commit is 43998c82a7e88df284b7aa30221b2d0d21b2b86a

VigibotDev commented 3 years ago

Hello,

With last raspbian apt update I get an I2C kernel problem on every platform (all RPI version) with :

RTIMULib IMU reads become unreliable (random corruption and FIFO overflow) on all RPI robotics platform >= kernel 5.10.14 RTIMULib IMU reads work perfectly on all RPI robotics platform <= kernel 5.10.11

Last working commit is 43998c82a7e88df284b7aa30221b2d0d21b2b86a (kernel: spi: bcm2835: Workaround/fix for zero-length transfers) Now we use this firmware commit ID on our official system image.

I2C regression can be inside :

kernel: Bump to 5.10.14
kernel: media: i2c: imx290: Support V4L2_CID_ANALOGUE_GAIN as well as V4L2_CID_GAIN See: raspberrypi/linux#4119
kernel: i2c: bcm2835: Handle untimely DONE signal See: raspberrypi/linux#3064

pelwell commented 3 years ago

Can you confirm that between working and non-working systems vcgencmd version returns the same result?

pelwell commented 3 years ago

Or to word it better, can you confirm that the same version of the firmware can be found in the last working and first non-working system?

VigibotDev commented 3 years ago

Last Raspbian stable, I2C NOT working : Our fresh continuous integration image (with apt update / upgrade, rpi-update never used)

(root|~) uname -a
Linux raspberrypi 5.10.17-v7+ #1403 SMP Mon Feb 22 11:29:51 GMT 2021 armv7l GNU/Linux
(root|~) vcgencmd version
Feb 25 2021 12:13:12
Copyright (c) 2012 Broadcom
version 564e5f9b852b23a330b1764bcf0b2d022a20afd0 (clean) (release) (start_x)

OLDEST version with I2C NOT WORKING : rpi-update 115e3a5f77488d9ee30a33bcb5ac31eb587f60a8

(root|~) uname -a
Linux raspberrypi 5.10.14-v7+ #1401 SMP Mon Feb 8 14:23:24 GMT 2021 armv7l GNU/Linux
(root|~) vcgencmd version
Feb  8 2021 14:35:41
Copyright (c) 2012 Broadcom
version 73b3cad64181954e67f6e9fe6d275378d3eefa10 (clean) (release) (start_x)

LAST version with I2C working : rpi-update 43998c82a7e88df284b7aa30221b2d0d21b2b86a

(root|~) uname -a
Linux raspberrypi 5.10.11-v7l+ #1399 SMP Thu Jan 28 12:09:48 GMT 2021 armv7l GNU/Linux
(root|~) vcgencmd version
Jan 27 2021 22:20:57
Copyright (c) 2012 Broadcom
version 99d9a48302e4553cff3688692bb7e9ac760a03fa (clean) (release) (start_x)

pelwell commented 3 years ago

Thanks. I'd dearly love it to not be my "untimely DONE signal" patch...

If you don't mind a bit of experimentation, starting with the last working commit (https://github.com/Hexxeh/rpi-firmware/commit/43998c82a7e88df284b7aa30221b2d0d21b2b86a), do a sudo SKIP_KERNEL=1 rpi-update 115e3a5 to just update the firmware.

VigibotDev commented 3 years ago

Hi

The next firmware version with the old kernel work, it's a success for you but please check if I understand all (my english is aproximative) :

From the last-working & complete version on one robot; I returned to the oldest non-working version + kernel-update skip as you want, please check the complete trace with some "vcgencmd version" proof of firmware changing without kernel change :

(root|~) vcgencmd version
Jan 27 2021 22:27:56
Copyright (c) 2012 Broadcom
version 99d9a48302e4553cff3688692bb7e9ac760a03fa (clean) (release) (start_x)
(root|~) SKIP_KERNEL=1 rpi-update 115e3a5f77488d9ee30a33bcb5ac31eb587f60a8
 *** Raspberry Pi firmware updater by Hexxeh, enhanced by AndrewS and Dom
 *** Performing self-update
 *** Relaunching after update
 *** Raspberry Pi firmware updater by Hexxeh, enhanced by AndrewS and Dom
#############################################################
WARNING: This update bumps to rpi-5.10.y linux tree
See: https://www.raspberrypi.org/forums/viewtopic.php?f=29&t=288234
'rpi-update' should only be used if there is a specific
reason to do so - for example, a request by a Raspberry Pi
engineer or if you want to help the testing effort
and are comfortable with restoring if there are regressions.

DO NOT use 'rpi-update' as part of a regular update process.

##############################################################
Would you like to proceed? (y/N)
 *** Downloading specific firmware revision (this will take a few minutes)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   168  100   168    0     0   1976      0 --:--:-- --:--:-- --:--:--  2000
100  119M    0  119M    0     0  3957k      0 --:--:--  0:00:31 --:--:-- 4602k
 *** Updating firmware
 *** As requested, not updating kernel
 *** As requested, not updating kernel modules
 *** Updating VideoCore libraries
 *** Using HardFP libraries
 *** Updating SDK
 *** Running ldconfig
 *** Storing current firmware revision
 *** Deleting downloaded files
 *** Syncing changes to disk
 *** If no errors appeared, your firmware was successfully updated to 115e3a5f77488d9ee30a33bcb5ac31eb587f60a8
 *** A reboot is needed to activate the new firmware
(root|~) uname -a
Linux raspberrypi 5.10.11-v7+ #1399 SMP Thu Jan 28 12:06:05 GMT 2021 armv7l GNU/Linux
(root|~) reboot
Connection to 192.168.0.28 closed by remote host.
Connection to 192.168.0.28 closed.
(root|~) ssh 192.168.0.28
root@192.168.0.28's password:
Linux raspberrypi 5.10.11-v7+ #1399 SMP Thu Jan 28 12:06:05 GMT 2021 armv7l

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Wed Mar 17 08:49:42 2021 from 192.168.0.2
(root|~) uname -a
Linux raspberrypi 5.10.11-v7+ #1399 SMP Thu Jan 28 12:06:05 GMT 2021 armv7l GNU/Linux
(root|~) vcgencmd version
Feb  8 2021 14:35:41
Copyright (c) 2012 Broadcom
version 73b3cad64181954e67f6e9fe6d275378d3eefa10 (clean) (release) (start_x)
(root|~)

It's good ? Now can we update to the last master or stable firmware + old kernel to get the last up to date and best working ?

VigibotDev commented 3 years ago

Or just ask for more tests... I have a small fleet of 8 RPI robots dedicated to stability testing. All 24/7 run, with H.264 low latency realtime video/audio with OpenCV graph overlay + multiples I2C sensors + UART LIDAR on the PI main UART @ 230K bauds + HPWM local audio + DMA PWM generators (PIGPIO) to run motors with mosfets.

pelwell commented 3 years ago

Thank you for doing that - you've narrowed the problem down to a kernel change in the 5.10.14 update. As you thought, sudo SKIP_KERNEL=1 rpi-update stable will install the latest stable firmware but preserve the 5.10.11 kernel.

I'm tempted to revert that one commit I'm suspicious about, but I want to discuss with colleagues if there is a better way to distribute trial releases rather than change it for everybody.

pelwell commented 3 years ago

What is the IMU device(s) you are using? It would be useful to get some for testing. Also, do you a very simple test script that demonstrates the problem?

VigibotDev commented 3 years ago

Now I try to make the bug visible out of our Vigibot.com software stack :

On Vigibot we read the I2C bus from two simultaneous process :

The main Node.js websocket client process (for battery gauge / voltage read) no I2C problem appear with it.
And the optional fast C++ process for LIDAR (SLAM) and IMU I2C read. Where the IMU corrupt I2C frame problem appear "I2C read error from 104, 116 - Failed to read fifo data" and some time "mpu-9250 fifo has overflowed".

On Vigibot we use RTIMULib official binaries from Raspbian repo.

I found the matching source code here : https://github.com/RPi-Distro/RTIMULib
With some easy to build and run IMU sample :

(root|~/RTIMULib/Linux/RTIMULibDrive) make
g++ -c -o objects/RTIMULibDrive.o RTIMULibDrive.cpp -pipe -O2 -Wall -W  -I. -I../../RTIMULib
g++ -c -o objects/RTMath.o ../../RTIMULib/RTMath.cpp -pipe -O2 -Wall -W  -I. -I../../RTIMULib
g++ -c -o objects/RTIMUHal.o ../../RTIMULib/RTIMUHal.cpp -pipe -O2 -Wall -W  -I. -I../../RTIMULib
g++ -c -o objects/RTFusion.o ../../RTIMULib/RTFusion.cpp -pipe -O2 -Wall -W  -I. -I../../RTIMULib
g++ -c -o objects/RTFusionKalman4.o ../../RTIMULib/RTFusionKalman4.cpp -pipe -O2 -Wall -W  -I. -I../../RTIMULib
g++ -c -o objects/RTFusionRTQF.o ../../RTIMULib/RTFusionRTQF.cpp -pipe -O2 -Wall -W  -I. -I../../RTIMULib
g++ -c -o objects/RTIMUSettings.o ../../RTIMULib/RTIMUSettings.cpp -pipe -O2 -Wall -W  -I. -I../../RTIMULib
g++ -c -o objects/RTIMUAccelCal.o ../../RTIMULib/RTIMUAccelCal.cpp -pipe -O2 -Wall -W  -I. -I../../RTIMULib
g++ -c -o objects/RTIMUMagCal.o ../../RTIMULib/RTIMUMagCal.cpp -pipe -O2 -Wall -W  -I. -I../../RTIMULib
g++ -c -o objects/RTIMU.o ../../RTIMULib/IMUDrivers/RTIMU.cpp -pipe -O2 -Wall -W  -I. -I../../RTIMULib
g++ -c -o objects/RTIMUNull.o ../../RTIMULib/IMUDrivers/RTIMUNull.cpp -pipe -O2 -Wall -W  -I. -I../../RTIMULib
g++ -c -o objects/RTIMUMPU9150.o ../../RTIMULib/IMUDrivers/RTIMUMPU9150.cpp -pipe -O2 -Wall -W  -I. -I../../RTIMULib
g++ -c -o objects/RTIMUMPU9250.o ../../RTIMULib/IMUDrivers/RTIMUMPU9250.cpp -pipe -O2 -Wall -W  -I. -I../../RTIMULib
g++ -c -o objects/RTIMUGD20HM303D.o ../../RTIMULib/IMUDrivers/RTIMUGD20HM303D.cpp -pipe -O2 -Wall -W  -I. -I../../RTIMULib
g++ -c -o objects/RTIMUGD20M303DLHC.o ../../RTIMULib/IMUDrivers/RTIMUGD20M303DLHC.cpp -pipe -O2 -Wall -W  -I. -I../../RTIMULib
g++ -c -o objects/RTIMUGD20HM303DLHC.o ../../RTIMULib/IMUDrivers/RTIMUGD20HM303DLHC.cpp -pipe -O2 -Wall -W  -I. -I../../RTIMULib
g++ -c -o objects/RTIMULSM9DS0.o ../../RTIMULib/IMUDrivers/RTIMULSM9DS0.cpp -pipe -O2 -Wall -W  -I. -I../../RTIMULib
g++ -c -o objects/RTIMULSM9DS1.o ../../RTIMULib/IMUDrivers/RTIMULSM9DS1.cpp -pipe -O2 -Wall -W  -I. -I../../RTIMULib
g++ -c -o objects/RTIMUBMX055.o ../../RTIMULib/IMUDrivers/RTIMUBMX055.cpp -pipe -O2 -Wall -W  -I. -I../../RTIMULib
g++ -c -o objects/RTIMUBNO055.o ../../RTIMULib/IMUDrivers/RTIMUBNO055.cpp -pipe -O2 -Wall -W  -I. -I../../RTIMULib
g++ -c -o objects/RTPressure.o ../../RTIMULib/IMUDrivers/RTPressure.cpp -pipe -O2 -Wall -W  -I. -I../../RTIMULib
g++ -c -o objects/RTPressureBMP180.o ../../RTIMULib/IMUDrivers/RTPressureBMP180.cpp -pipe -O2 -Wall -W  -I. -I../../RTIMULib
g++ -c -o objects/RTPressureLPS25H.o ../../RTIMULib/IMUDrivers/RTPressureLPS25H.cpp -pipe -O2 -Wall -W  -I. -I../../RTIMULib
g++ -c -o objects/RTPressureMS5611.o ../../RTIMULib/IMUDrivers/RTPressureMS5611.cpp -pipe -O2 -Wall -W  -I. -I../../RTIMULib
g++ -c -o objects/RTPressureMS5637.o ../../RTIMULib/IMUDrivers/RTPressureMS5637.cpp -pipe -O2 -Wall -W  -I. -I../../RTIMULib
g++ -Wl,-O1 -o Output/RTIMULibDrive objects/RTIMULibDrive.o objects/RTMath.o objects/RTIMUHal.o objects/RTFusion.o objects/RTFusionKalman4.o objects/RTFusionRTQF.o objects/RTIMUSettings.o objects/RTIMUAccelCal.o objects/RTIMUMa
gCal.o objects/RTIMU.o objects/RTIMUNull.o objects/RTIMUMPU9150.o objects/RTIMUMPU9250.o objects/RTIMUGD20HM303D.o objects/RTIMUGD20M303DLHC.o objects/RTIMUGD20HM303DLHC.o objects/RTIMULSM9DS0.o objects/RTIMULSM9DS1.o objects/R
TIMUBMX055.o objects/RTIMUBNO055.o objects/RTPressure.o objects/RTPressureBMP180.o objects/RTPressureLPS25H.o objects/RTPressureMS5611.o objects/RTPressureMS5637.o  -L/usr/lib/arm-linux-gnueabihf
(root|~/RTIMULib/Linux/RTIMULibDrive) cd Output/
(root|~/RTIMULib/Linux/RTIMULibDrive/Output) ls
RTIMULibDrive

And easy to run

(root|~/RTIMULib/Linux/RTIMULibDrive/Output) ./RTIMULibDrive
Settings file not found. Using defaults and creating settings file
Detected MPU9250 at standard address
Using fusion algorithm RTQF
min/max compass calibration not in use
Ellipsoid compass calibration not in use
Accel calibration not in use
MPU-9250 init complete
^Cmple rate 84: : roll:169.933659, pitch:-4.159935, yaw:140.757385

The process need CTRL + C to exit.

I found strings of the problem inside C++ code "I2C read error from 104, 116 - Failed to read fifo data"

(root|~/RTIMULib) git grep "I2C read error from" && git grep "Failed to read fifo data"
RTIMULib/RTIMUHal.cpp:                    HAL_ERROR3("I2C read error from %d, %d - %s\n", slaveAddr, regAddr, errorMsg);
RTIMULib/RTIMUHal.cpp:                    HAL_ERROR2("I2C read error from %d - %s\n", slaveAddr, errorMsg);
RTIMULib/IMUDrivers/RTIMUMPU9150.cpp:        if (!m_settings->HALRead(m_slaveAddr, MPU9150_FIFO_R_W, MPU9150_FIFO_CHUNK_SIZE, fifoData, "Failed to read fifo data"))
RTIMULib/IMUDrivers/RTIMUMPU9150.cpp:                                m_cache[m_cacheIn].data, "Failed to read fifo data"))
RTIMULib/IMUDrivers/RTIMUMPU9150.cpp:            if (!m_settings->HALRead(m_slaveAddr, MPU9150_FIFO_R_W, MPU9150_FIFO_CHUNK_SIZE, fifoData, "Failed to read fifo data"))
RTIMULib/IMUDrivers/RTIMUMPU9150.cpp:    if (!m_settings->HALRead(m_slaveAddr, MPU9150_FIFO_R_W, MPU9150_FIFO_CHUNK_SIZE, fifoData, "Failed to read fifo data"))
RTIMULib/IMUDrivers/RTIMUMPU9250.cpp:        if (!m_settings->HALRead(m_slaveAddr, MPU9250_FIFO_R_W, MPU9250_FIFO_CHUNK_SIZE, fifoData, "Failed to read fifo data"))
RTIMULib/IMUDrivers/RTIMUMPU9250.cpp:                    m_cache[m_cacheIn].data, "Failed to read fifo data"))
RTIMULib/IMUDrivers/RTIMUMPU9250.cpp:            if (!m_settings->HALRead(m_slaveAddr, MPU9250_FIFO_R_W, MPU9250_FIFO_CHUNK_SIZE, fifoData, "Failed to read fifo data"))
RTIMULib/IMUDrivers/RTIMUMPU9250.cpp:    if (!m_settings->HALRead(m_slaveAddr, MPU9250_FIFO_R_W, MPU9250_FIFO_CHUNK_SIZE, fifoData, "Failed to read fifo data"))

Now I need to return to a bad Kernel and reproduce the problem with the RTIMULibDrive process.

VigibotDev commented 3 years ago

You need a mpu-9250 breakout board :

https://www.amazon.fr/gp/product/B07HMQZ7N9

Don't buy the blue PCB version without pressure sensor : A lot of Non-working fake are on the market !!! We can't find working version of the no-pressure / blue PCB (tested aliexpress and amazon are all fake !)

VigibotDev commented 3 years ago

Win ! You can use the compiled binary to get the problem, (I used rpi-update without parameter to jump on the last firmware/kernel)

The error is more frequent with the Node.js client I2C slow pooling for battery monitoring. + the binary for testing :

(root|~/RTIMULib/Linux/RTIMULibDrive/Output) ls
RTIMULibDrive  RTIMULib.ini
(root|~/RTIMULib/Linux/RTIMULibDrive/Output) ./RTIMULibDrive
Settings file RTIMULib.ini loaded
Using fusion algorithm RTQF
min/max compass calibration not in use
Ellipsoid compass calibration not in use
Accel calibration not in use
MPU-9250 init complete
I2C read error from 104, 116 - Failed to read fifo data:140.464862
I2C read error from 104, 116 - Failed to read fifo data:140.428143ch:-4.165171, yaw:140.463510
^Cmple rate 84: : roll:170.000417, pitch:-4.202972, yaw:140.246870ch:-4.229920, yaw:140.305159

I stop the vigibot client

(root|~/RTIMULib/Linux/RTIMULibDrive/Output) systemctl stop vigiclient

But error still here, but very rare need a long run :

(root|~/RTIMULib/Linux/RTIMULibDrive/Output) ./RTIMULibDrive
Settings file RTIMULib.ini loaded
Using fusion algorithm RTQF
min/max compass calibration not in use
Ellipsoid compass calibration not in use
Accel calibration not in use
MPU-9250 init complete
I2C read error from 104, 116 - Failed to read fifo data:140.810606
^Cmple rate 84: : roll:169.980568, pitch:-4.166409, yaw:140.560061ch:-4.223783, yaw:140.817723

The raw trace from our C++ process received on the website is : I2C error appear way faster because Node.js pool battery fast + the C++ process high bandwidth on IMU :

16/03 15:15:40:972 |              | Client | Diffusion | 7334 | stderr | I2C read error from 104, 116 - Failed to read fifo data
16/03 15:15:42:057 |              | Client | Diffusion | 7334 | stderr | I2C read error from 104, 116 - Failed to read fifo data
16/03 15:15:44:778 |              | Client | Diffusion | 7334 | stderr | I2C read error from 104, 116 - Failed to read fifo data
16/03 15:15:46:529 |              | Client | Diffusion | 7334 | stderr | I2C read error from 104, 116 - Failed to read fifo data
16/03 15:15:48:257 |              | Client | Diffusion | 7334 | stderr | I2C read error from 104, 116 - Failed to read fifo data
16/03 15:15:49:456 |              | Client | Diffusion | 7334 | stderr | I2C read error from 104, 116 - Failed to read fifo data
16/03 15:15:52:156 |              | Client | Diffusion | 7334 | stderr | I2C read error from 104, 116 - Failed to read fifo data
16/03 15:15:53:582 |              | Client | Diffusion | 7334 | stderr | I2C read error from 104, 116 - Failed to read fifo data
16/03 15:15:56:034 |              | Client | Diffusion | 7334 | stderr | I2C read error from 104, 116 - Failed to read fifo data
16/03 15:15:58:032 |              | Client | Diffusion | 7334 | stderr | I2C read error from 104, 116 - Failed to read fifo data
16/03 15:15:59:556 |              | Client | Diffusion | 7334 | stderr | I2C read error from 104, 116 - Failed to read fifo data

or

16/03 15:57:51:622 |              | Client | Diffusion | 8554 | stderr | I2C read error from 104, 116 - Failed to read fifo data
16/03 15:57:53:837 |              | Client | Diffusion | 8554 | stderr | MPU-9250 fifo has overflowedI2C read error from 104, 116 - Failed to read fifo data
16/03 15:57:57:473 |              | Client | Diffusion | 8554 | stderr | MPU-9250 fifo has overflowedI2C read error from 104, 116 - Failed to read fifo data
16/03 15:57:59:208 |              | Client | Diffusion | 8554 | stderr | MPU-9250 fifo has overflowedI2C read error from 104, 116 - Failed to read fifo data

VigibotDev commented 3 years ago

I remembrer we use 400K I2C bus rate :

# Uncomment some or all of these to enable the optional hardware interfaces
dtparam=i2c_arm=on,i2c_arm_baudrate=400000

I just tested @ 100K and it's harder to get the problem ! we must run @ 400K !

pelwell commented 3 years ago

Thanks - I have one on order, but it might take a month to arrive.

VigibotDev commented 3 years ago

Important you need 400KHz I2C clock and 1KHz IMU sample rate to get multiples I2C errors every minutes.

I launched binary once to get the RTIMULib.ini file + set "MPU9250GyroAccelSampleRate=" to 1000 for 1 KHz sample rate
I also re-tested this minimal setup on the old Kernel + stable firmware without any error in 30 minutes.

Few minute testing on the last Master (kernel + firmware) :

(root|~/RTIMULib/Linux/RTIMULibDrive/Output) systemctl stop vigiclient
(root|~/RTIMULib/Linux/RTIMULibDrive/Output) grep MPU9250GyroAccelSampleRate RTIMULib.ini
MPU9250GyroAccelSampleRate=1000
(root|~/RTIMULib/Linux/RTIMULibDrive/Output) ./RTIMULibDrive
Settings file RTIMULib.ini loaded
Using fusion algorithm RTQF
min/max compass calibration not in use
Ellipsoid compass calibration not in use
Accel calibration not in use
MPU-9250 init complete
I2C read error from 104, 116 - Failed to read fifo dataaw:141.097747
I2C read error from 104, 116 - Failed to read fifo dataaw:140.320786ch:-4.169592, yaw:139.758907
I2C read error from 104, 116 - Failed to read fifo dataaw:141.043283h:-4.156291, yaw:140.894522
I2C read error from 104, 116 - Failed to read fifo dataaw:139.960753ch:-4.190503, yaw:140.338340
I2C read error from 104, 116 - Failed to read fifo dataw:139.932872tch:-4.166954, yaw:140.754107
I2C read error from 104, 116 - Failed to read fifo dataaw:141.235307h:-4.188396, yaw:140.192857
I2C read error from 104, 116 - Failed to read fifo dataw:140.302590tch:-4.195192, yaw:139.979973
^Cmple rate 1010: : roll:169.974927, pitch:-4.183485, yaw:141.555575h:-4.148631, yaw:140.939587
(root|~/RTIMULib/Linux/RTIMULibDrive/Output)

Also I can give you a root SSH access to a fresh robot with FTTH internet access if you want to try things before the package to arrive (you can crash it no problem, they are from my automatic integration).

pelwell commented 3 years ago

That's a good idea. Email me - phil@raspberrypi.com - with the details. My plan would be to push a trial kernel to it to confirm that it solves the problem before I make the change public.

VigibotDev commented 3 years ago

Now you can ssh to my domain with the email port + password.

It run on the last working version, all is ready to work or fail depend on the Kernel version. (400KHz I2C 1KHz IMU sample rate).

pelwell commented 3 years ago

Thanks - expect a visit soon.

VigibotDev commented 3 years ago

You can reboot as you want... I keep open access for multiples days if you need....

Pascal https://www.vigibot.com https://www.serveurperso.com

Le 17/03/2021 15:44, Phil Elwell a écrit :

Thanks - expect a visit soon.

-- You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub [1], or unsubscribe [2].

Links:

[1] https://github.com/Hexxeh/rpi-firmware/issues/261#issuecomment-801141744 [2] https://github.com/notifications/unsubscribe-auth/ABFF3UCP7W3S3HCOOFPXVATTEC53HANCNFSM4ZI7Y4KQ

VigibotDev commented 3 years ago

You can look https://www.youtube.com/watch?v=NmrKH4Qx6io how Vigibot perform on a smartphone :

Raspberry PI 3 A+
I2C IMU MPU9250
100$ 4500 measure/sec Mini ToF lidar
Small box (https://github.com/vigibot/vigicad/) with 4 Pololu N20 HP motors and two H bridge

Everything @ 30FPS and 0.1/0.2 second end to end / 4G to 4G latency. Vigibot.com is a non-profit-making organisation (1901 french law)

pelwell commented 3 years ago

Your target Pi is now running the latest kernel and firmware but with the suspect commit reverted, and I've not seen any I2C errors in a few minutes - with the current rpi-update kernel it saw errors after about 10 seconds.

On the basis of that I'll push the reversion to our current branches, then revisit the patch once the MPU9250 arrives.

pelwell commented 3 years ago

If you want to try the kernel on other Pi 3s, download install.tgz from here (https://drive.google.com/file/d/1ItRLCyvgba8cQeU6qgtloxu_xgDsG-df/view?usp=sharing), or use:

$ wget -O install.tgz "https://drive.google.com/uc?export=download&id=1ItRLCyvgba8cQeU6qgtloxu_xgDsG-df"

directly on the command line. Install with:

$ tar --no-same-owner --keep-directory-symlink -zxf install.tgz -C /

(You will need to use sudo unless you run as root).

VigibotDev commented 3 years ago

I confirm you the complete system work on the kernel (vigibot.com robot name is Lidar) (root|~) uname -a Linux raspberrypi 5.10.23-v7+ #1083 SMP Wed Mar 17 15:30:57 GMT 2021 armv7l GNU/Linux

If you can give me a procedure I can deploy on the 10 testing robots

pelwell commented 3 years ago

N.B. The test kernel is only for a Pi 3B, 3B+ or 3A+ or 2B.

VigibotDev commented 3 years ago

LOL I see the answer few ms. before I clicked the "Comment" button. Test success on the PI 3 A+ and B+ (No 4 as your N.B. And no 2B because is too slow for the IMU+Lidar Autopilot, 2B OK only for low latency basic FPV piloting)

you can check my executing your procedure here :

(root|~) wget -O install.tgz "https://drive.google.com/uc?export=download&id=1ItRLCyvgba8cQeU6qgtloxu_xgDsG-df"
--2021-03-17 15:52:30--  https://drive.google.com/uc?export=download&id=1ItRLCyvgba8cQeU6qgtloxu_xgDsG-df
Resolving drive.google.com (drive.google.com)... 142.250.179.78, 2a00:1450:4007:813::200e
Connecting to drive.google.com (drive.google.com)|142.250.179.78|:443... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: https://doc-0c-9o-docs.googleusercontent.com/docs/securesc/ha0ro937gcuc7l7deffksulhg5h7mbp1/k3bsbojtnacgq8d047hti204gme0mbp0/1615996350000/18441922891                              031522895/*/1ItRLCyvgba8cQeU6qgtloxu_xgDsG-df?e=download [following]
Warning: wildcards not supported in HTTP.
--2021-03-17 15:52:36--  https://doc-0c-9o-docs.googleusercontent.com/docs/securesc/ha0ro937gcuc7l7deffksulhg5h7mbp1/k3bsbojtnacgq8d047hti204gme0mbp0/1615996350                              000/18441922891031522895/*/1ItRLCyvgba8cQeU6qgtloxu_xgDsG-df?e=download
Resolving doc-0c-9o-docs.googleusercontent.com (doc-0c-9o-docs.googleusercontent.com)... 216.58.206.225, 2a00:1450:4007:817::2001
Connecting to doc-0c-9o-docs.googleusercontent.com (doc-0c-9o-docs.googleusercontent.com)|216.58.206.225|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/x-compressed]
Saving to: ‘install.tgz’

install.tgz                                 [                                    <=>                                         ]  25.38M  3.52MB/s    in 7.1s

2021-03-17 15:52:43 (3.57 MB/s) - ‘install.tgz’ saved [26614529]

(root|~) ls
install.tgz
(root|~) tar --no-same-owner --keep-directory-symlink -zxf install.tgz -C /
(root|~) uname -a
Linux raspberrypi 5.10.11-v7+ #1399 SMP Thu Jan 28 12:06:05 GMT 2021 armv7l GNU/Linux
(root|~) reboot
Connection to 192.168.0.25 closed by remote host.
Connection to 192.168.0.25 closed.
(root|~) ssh 192.168.0.25
root@192.168.0.25's password:
Linux raspberrypi 5.10.23-v7+ #1083 SMP Wed Mar 17 15:30:57 GMT 2021 armv7l

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Wed Mar 17 15:51:31 2021 from 192.168.0.2
(root|~) uname -a
Linux raspberrypi 5.10.23-v7+ #1083 SMP Wed Mar 17 15:30:57 GMT 2021 armv7l GNU/Linux

pelwell commented 3 years ago

Yes - that looks correct.

VigibotDev commented 3 years ago

My today testing :

Fresh image + apt update / upgrade on nine robots with Raspberry PI 3 A+ / B+ and 4 4Go
9 * robots with IMU do not work (as expected)
rpi-update on all (without parameters)
9 * robots with IMU all working OK

The master branch firmware (and kernel) are OK !

pelwell commented 3 years ago

Please close the issue,

Hexxeh / rpi-firmware

I2C regression, last working commit is 43998c82a7e88df284b7aa30221b2d0d21b2b86a #261

Links: