search

FS-AI / FS-AI_2019

Software library for the FS-AI ADS-DV for 2019.
2 stars 0 forks source link

CAN comms fault issues with Innodisk CAN hardware. #4

Open IanMurphy-Rockfort opened 5 years ago

IanMurphy-Rockfort commented 5 years ago

CAN comms faults have been reported during software testing with the ADS-DV.

The issue seems to be related to the particular combination of the Edinburgh / Brookes Linux configuration and the Innodisk CAN hardware. The evidence is as follows:

  1. Software functions correctly with the Coventry Linux config plus Innodisk hardware (tested at Bruntingthorpe and MIRA).
  2. Software functions correctly with the Edinburgh / Brookes Linux config plus PCAN-USB hardware (tested at Brookes).
  3. Software functions correctly with a laptop plus PCAN-USB hardware (tested at Brookes).
  4. Software does not function correctly with the Edinburgh / Brookes Linux config plus Innodisk hardware (tested at Brookes).

The majority of tests were carried out using ‘fs-ai_api_console’ as uploaded to https://github.com/FS-AI/FS-AI_2019 25/06/2019. Tests were also run using Brookes experimental code and also the ‘fs-ai_api_rc’ software used to drive the car remotely at Bruntingthorpe. The default timing scheme was a ‘usleep(5000);’ call in the send thread.

Using a fork of ‘candump’ with extra stats (see https://github.com/IanMurphy-Rockfort/can-utils ), running ‘candump can0,510:7ff -t s’ (to view the stats of AI2VCU_Status frame ID), the following can be observed:

  1. With PCAN-USB hardware (either host) the stats are consistent with respect to transmission rate.
  2. With PCAN-USB hardware (either host) the handshake bits can be seen to toggle correctly once in AS_Driving.
  3. With Innodisk hardware (Edinburgh / Brookes host) the stats are not consistent with respect to transmission rate, there is much more variation in frame timing.
  4. With Innodisk hardware (Edinburgh / Brookes host) the handshake bits are not seen to toggle correctly once in AS_Driving.

Based on experiments with CAN timing using the ‘t [delay]’ command in ‘fs-ai_api_console’ the vehicle ECU is tolerant of timing errors up to the 50ms threshold. Therefore the cause of the instant ‘comms lost’ error on the transition to AS_Driving is likely to be (4) above.

The reason is still not understood, but is likely related to frame buffering and / or dropped frames. More likely on the receive side but this is not proven yet.

Further work can be done but the recommended workaround is to use PCAN-USB CAN hardware until the issue is understood.

As additional info is uncovered it will be appended to this issue.

imgeorgiev commented 5 years ago

The PCAN hardware is initialised with:

sudo ip link set can0 type can bitrate 500000
sudo ip link set can0 up

and the Innodisk one is initialised with:

sudo sudo ip link set can0 up qlen 1000

Might it be worth adding bitrate 500000 to the innodisk initialisation as well?

ianrmurphy commented 5 years ago

The Innodisk hardware is a UART to CAN bridge and is configured with it's own utility program 'emucd_64'. It's this utility that sets the bit rate using a command line argument -sN where N sets the bit rate.

The Innodisk driver is supplied with a shell script 'start.sh' which contains the following lines:

sudo ./emucd_64 -s9 ttyACM0 can0 can1
sudo sudo ip link set can0 up qlen 1000
sudo sudo ip link set can1 up qlen 1000

In this case -s9 sets a bit rate of 1Mbps. Based on what was explained to me by Jacob Culley at Brookes, there is some uncertainty as to whether -s6 or -s7 is correct for 500kbps. The source code for the Innodisk driver says that -s7 is correct for 500kbps, however during the static workshop day hosted at Hypermotive I believe -s6 was actually found to work.

This is straightforward to test using a separate CAN adapter (e.g. a PCAN-USB) and the cansend and candump utilities.

If a definitive guide to the correct configuration of the Innodisk hardware could be written and posted here, it would be appreciated.