Can bus support for controller?

[sandervandegeijn]

A lot of inverters (Goodwe, Growatt, SMA, Sungrow, etc) have the capability to communicate with a battery over CAN. I even suspect they refuse to function without communication being present. Maybe it would be a nice addition to support communication over CAN replicating the LG RESU message protocol (best supported in the industry it seems).

[stuartpittaway]

Hello, I agree, I've been working on a new controller design and that includes CAN BUS support.

Do you have any good references to the LG RESU protocols?

[sandervandegeijn]

Cool :) I did see this project: https://github.com/jens18/lgresu/tree/master/lgresustatus

It contains the PID's and messages and it also simulates the LG battery in order to test itself, see lgresustatus_test.go For the CAN side, an ESP32 with a can transceiver is quite simple to use. ESP8266 with CAN controller can also work though. I implemented something similar to connect to a CAN capable charger for my motorcycle.

The RESU batteries also come in high voltage units delivering over 400V (100S I presume). Don't thing the diybms scales to 100s right? :)

[atanisoft]

I'd suggest a plugin approach for the CAN interface as there may not be a consistent protocol between various inverters and/or charge controllers. As an example EPEver (or EPSolar, whichever it may be) supports a simple protocol to monitor the charge controller (there may be support for multiple charge controllers on the bus each using a unique ID but I haven't explored it).

I was working on an ESP32 update of the controller code to add CAN and shunt support but I know @stuartpittaway has this in the works already though so I've stopped working on that work (it included a new controller board which is also likely not necessary now). Once the new board / controller code is available I can adapt it to work with the EPEver charge controller I have now and when I get another I can test the multiple charge controllers on the same bus support (if it exists).

[stuartpittaway]

The new controller will definately be an ESP32 - DEVKITC style.

@atanisoft more than happy for you to help with the CAN stuff, I've not done much work on that yet.

I decided on the SN65HVD230DR chip for the hardware element http://www.ti.com/lit/ds/symlink/sn65hvd230.pdf as they are simple to use and available via JLCPCB, although hand soldering of it is a pain!

@atanisoft you have probably noticed the new branch for the controller and module code, lots of changes going on to improve the solution.

[atanisoft]

I decided on the SN65HVD230DR chip for the hardware element http://www.ti.com/lit/ds/symlink/sn65hvd230.pdf as they are simple to use and available via JLCPCB, although hand soldering of it is a pain!

The SN65HVD230DR works great for 3v3 BUS speed, but may have some issues with 5v CAN bus voltages. I'd suggest using TJA1051T/3 (C38695) which offers split voltage support (3v3 for MCU IO and 5v for CAN H/L). I've also used a few other transceivers in various projects. If you are hand soldering it is around the same difficulty as the ATTiny, but since it is available as a basic part on JLCPCB I see no reason to hand solder it.

I've not done much work on that yet.

I've been working on the ESP32 CAN (soon to be renamed TWAI) peripheral for a while now. Note that there is ZERO support in arduino-esp32 as you see for other peripherals but that is not a problem as you can use the ESP-IDF functions to interact with the peripheral without issues. You could also use https://github.com/collin80/esp32_can which offers a few options but I've found that using the ESP-IDF drivers directly works best. Here is a driver I wrote using ESP-IDF functions to get you started if you opt for that route.

[sandervandegeijn]

There definitely differences between the different battery manufacturers regarding CAN message structure. There doesn't seem to be an industry standard everyone uses. So a abstract plugin type implementation would give the flexibility to emulate whatever battery the inverter wants to use. Batrium also does it this way as far as I can tell.

I'm about to pull the trigger on a Sungrow inverter for my DIY powerwall. Possibly I could help out testing it against that inverter. I'm afraid my C++ isn't good enough to help out here... So I'm stuck providing feedback and ideas.

[sandervandegeijn]

Btw, I was looking into the Daly BMS'es, I did get their protocol. Not the most used, not aware of any inverters that support it, but maybe it can be used for inspiration:

Daly CAN Protocol.docx

[atanisoft]

Possibly I could help out testing it against that inverter. I'm afraid my C++ isn't good enough to help out here.

Don't be afraid to submit C code for it, it can be converted to C++ code if needed. Almost all ESP32 code is done with C (ESP-IDF) and most C++ code is just thin wrappers around the lower level C code.

[stuartpittaway]

I'd suggest using TJA1051T/3 (C38695) which offers split voltage support (3v3 for MCU IO and 5v for CAN H/L).

Thats annoying, I skipped that part as the JLC description says 4.5 to 5v and I thought that was the input voltage! Should have read the datasheet!

[stuartpittaway]

Don't be afraid to submit C code for it, it can be converted to C++ code if needed.

Agreed, my C and C++ programming are terrible, so you won't do any worse than that :-)

[atanisoft]

Thats annoying, I skipped that part as the JLC description says 4.5 to 5v and I thought that was the input voltage! Should have read the datasheet!

Yeah, sometimes their descriptions are not the most helpful... I'd also suggest adding one of these as a safety measure. Another CAN transceiver I've been using on a couple designs is this one, it is pin compatible with TJA1051T/3.

[sandervandegeijn]

Stuart, when you have a new design could you post it here? I'll order it and see if I can help out with some coding/testing. Another good option is to incorporate a connector for a CT current clamp so we can monitor what's going through the battery. https://github.com/stuartpittaway/diyBMSv4/issues/53

[stuartpittaway]

I'm also building an external current shunt to provide the DC current and voltage monitoring, which will connect back to the controller over RS485.

[stuartpittaway]

Thats annoying, I skipped that part as the JLC description says 4.5 to 5v and I thought that was the input voltage! Should have read the datasheet!

Yeah, sometimes their descriptions are not the most helpful... I'd also suggest adding one of these as a safety measure. Another CAN transceiver I've been using on a couple designs is this one, it is pin compatible with TJA1051T/3.

Hi @atanisoft finally got around to trying out the CAN interface on my new controller board.

For test purposes, I've got an Arduino and MCP2515 and MCP2551 combination (note the MCP2551 is 5V) and the SN65HVD230DR on the controller/ESP32 side.

After getting over the initial problem of running a 5V part at 3.3V :-) I finally managed to get the Arduino to transmit a few packets to the ESP32.

What I'm finding is the first 3 or 4 packets get through, then the ESP32 stops receiving and timeout. I'm using the standard ESP-IDF libs.

I also noticed that some packets are scrambled - I'm just sending 8 byte data of 1,2,3,4,5,6,7,8 and sometimes I'll get 5,6,7,8,1,2,3,4 so it seems to be mixing up two packets.

Once the ESP has stopped picking up packets, if I reboot it, it starts again for another 3 or 4 packets and then stops.

The ESP32 code is a copy and paste from the examples.

I've also found that unless I set both clkout_io and buf_off_io to a real pin (instead of-1) then nothing is received.

  can_general_config_t g_config;

  g_config.mode = CAN_MODE_NORMAL;
  g_config.tx_io = gpio_num_t::GPIO_NUM_22;
  g_config.rx_io = gpio_num_t::GPIO_NUM_21;
  //These are fake just for testing should be -1
  g_config.clkout_io = gpio_num_t::GPIO_NUM_25;// (gpio_num_t)-1;
  //These are fake just for testing should be -1
  g_config.bus_off_io = gpio_num_t::GPIO_NUM_15; //(gpio_num_t)-1;
  g_config.tx_queue_len = 5;
  g_config.rx_queue_len = 5;
  g_config.alerts_enabled = CAN_ALERT_NONE;
  g_config.clkout_divider = 0;

  can_timing_config_t t_config = CAN_TIMING_CONFIG_125KBITS();
  can_filter_config_t f_config = CAN_FILTER_CONFIG_ACCEPT_ALL();

  //Install CAN driver
  if (can_driver_install(&g_config, &t_config, &f_config) == ESP_OK)
  {
    SERIAL_DEBUG.printf("Driver installed\n");
  }
  else
  {
    SERIAL_DEBUG.printf("Failed to install driver\n");
  }

  //Start CAN driver
  if (can_start() == ESP_OK)
  {
    SERIAL_DEBUG.println("Driver started\n");
  }
  else
  {
    SERIAL_DEBUG.println("Failed to start driver\n");
  }

  while (1)
  {

    //Wait for message to be received
    can_message_t message;
    if (can_receive(&message, pdMS_TO_TICKS(10000)) == ESP_OK)
    {
      SERIAL_DEBUG.println("Message received\n");

      //Process received message
      if (message.flags & CAN_MSG_FLAG_EXTD)
      {
        SERIAL_DEBUG.printf("Message is in Extended Format\n");
      }
      else
      {
        SERIAL_DEBUG.printf("Message is in Standard Format\n");
      }
      SERIAL_DEBUG.printf("ID is %d", message.identifier);
      if (!(message.flags & CAN_MSG_FLAG_RTR))
      {
        for (int i = 0; i < message.data_length_code; i++)
        {
          dumpByte(message.data[i]);
        }
      }
      SERIAL_DEBUG.println();
    }
    else
    {
      SERIAL_DEBUG.printf("Failed to receive message\n");
    }

    delay(500);
  }

[atanisoft]

overall your code looks OK. I'd suggest using:

can_general_config_t g_config = CAN_GENERAL_CONFIG_DEFAULT(GPIO_NUM_22, GPIO_NUM_21, CAN_MODE_NORMAL);

rather than initializing the fields yourself. This should be available in IDF v3.2+ (including arduino 1.0.5 RC). If it is not available here is the definition.

I also noticed that some packets are scrambled - I'm just sending 8 byte data of 1,2,3,4,5,6,7,8 and sometimes I'll get 5,6,7,8,1,2,3,4 so it seems to be mixing up two packets.

Check the endianess of the packets, it looks like it is being sent by one side as little-endian and the other as big-endian. I'd suggest force the endianess to network order (use htonl and nltoh or similar)

Once the ESP has stopped picking up packets, if I reboot it, it starts again for another 3 or 4 packets and then stops.

You likely are in a bus-off condition due to errors. try this:

static const char *ESP32_CAN_STATUS_STRINGS[] = {
    "STOPPED",               // CAN_STATE_STOPPED
    "RUNNING",               // CAN_STATE_RUNNING
    "OFF / RECOVERY NEEDED", // CAN_STATE_BUS_OFF
    "RECOVERY UNDERWAY"      // CAN_STATE_RECOVERING
};

can_message_t message;
esp_err_t res = can_receive(&message, pdMS_TO_TICKS(10000));
if (res == ESP_OK)
{
 /// process it.
}
else if (res == ESP_ERR_TIMEOUT)
{
 /// ignore the timeout or do something
}
else
{
  // check the health of the bus
  can_status_info_t status;
  can_get_status_info(&status);
  printf("rx-q:%d, tx-q:%d, rx-err:%d, tx-err:%d, arb-lost:%d, bus-err:%d, state: %s",
    status.msgs_to_rx, status.msgs_to_tx,status.rx_error_counter, status.tx_error_counter, status.arb_lost_count,
    status.bus_error_count, ESP32_CAN_STATUS_STRINGS[status.state]);
  if (status.state == CAN_STATE_BUS_OFF)
  {
    // When the bus is OFF we need to initiate recovery, transmit is
    // not possible when in this state.
    printf("ESP32-CAN: initiating recovery");
    can_initiate_recovery();
  }
  else if (status.state == CAN_STATE_RECOVERING)
  {
    // when the bus is in recovery mode transmit is not possible.
    delay(TX_DEFAULT_DELAY);
  }
}

If the bus status is not RUNNING then you won't receive anything, similarly you can't transmit.

Also make sure you have a pair of 120ohm resistors on the H/L lines between the devices (one on each end as a terminator)

[atanisoft]

Note that in my case I have a background task that runs every 10-15sec to dump the bus status and also initiates the recovery when needed.

[stuartpittaway]

Mike, I couldn't initialize the can controller using the parameters as it won't allow me to use -1 for the two ports I don't want/need. Any ideas?

[atanisoft]

What error are you seeing with the -1?

[stuartpittaway]

It can't be cast to the pin enumeration. Think there is an old bug open for it.

[atanisoft]

use (gpio_num_t)-1 instead.

[stuartpittaway]

I've done that, it compiled but I never get any packets received when set like that. Strange!

[atanisoft]

Very strange. It sounds like a bug in the version of the code you are using. I'm guessing PIO is not picking up the latest ESP-IDF code.

[stuartpittaway]

Still not making much progress with this.

The platformio toolchain is pulling this version of the XTENSA32 framework.

"name": "toolchain-xtensa32", "version": "2.50200.80", "description": "GCC Toolchain for Xtensa32 processor", "url": "https://github.com/espressif/esp-idf"

along with

"name": "framework-arduinoespressif32", "version": "3.10004.201016", "description": "Arduino Wiring-based Framework for Espressif ESP32 microcontrollers",

If I use the code below to configure CAN_GENERAL_CONFIG_DEFAULT

  can_general_config_t g_config = CAN_GENERAL_CONFIG_DEFAULT(gpio_num_t::GPIO_NUM_22, gpio_num_t::GPIO_NUM_21, CAN_MODE_NORMAL);

then I get errors in the "can.h"

....packages/framework-arduinoespressif32/tools/sdk/include/driver/driver/can.h:39:131: error: invalid conversion from 'int' to 'gpio_num_t' [-fpermissive]

This is caused by these defaults trying to use CAN_IO_UNUSED ...

#define CAN_GENERAL_CONFIG_DEFAULT(tx_io_num, rx_io_num, op_mode) {.mode = op_mode, .tx_io = tx_io_num, .rx_io = rx_io_num,       \
                                                                   .clkout_io = CAN_IO_UNUSED, .bus_off_io = CAN_IO_UNUSED,       \
                                                                   .tx_queue_len = 5, .rx_queue_len = 5,                          \
                                                                   .alerts_enabled = CAN_ALERT_NONE,  .clkout_divider = 0,        }

[stuartpittaway]

I've added this to platform.io

; Use my github version of the framework
platform_packages = framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git

and its pulled down the master branch of the Arduino ESP32 framework, which seems to fix the previous compile error.

This now has

#define CAN_IO_UNUSED                   ((gpio_num_t) -1)   /**< Marks GPIO as unused in CAN configuration */

I've definitely got 120ohm resistors on both ends of the can bus, and I don't see any canbus errors just ESP_ERR_TIMEOUT

[atanisoft]

"name": "framework-arduinoespressif32", "version": "3.10004.201016", "description": "Arduino Wiring-based Framework for Espressif ESP32 microcontrollers",

ok this would map to arduino-esp32 1.0.4 which had an earlier version of ESP-IDF v3.2 in it.

I've added this to platform.io

; Use my github version of the framework
platform_packages = framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git

This is effectively arduino-esp32 1.0.5-rc4 which picks up the most recent ESP-IDF v3.3 release. This is a good thing!

I don't see any canbus errors just ESP_ERR_TIMEOUT

I'd suggest add a background task that monitors the CAN bus health and reacts to the changes on the bus. Something like this might work:

void can_mon()
{
  static const char *STATUS_STRINGS[] =
  {
    "STOPPED",               // CAN_STATE_STOPPED
    "RUNNING",               // CAN_STATE_RUNNING
    "OFF / RECOVERY NEEDED", // CAN_STATE_BUS_OFF
    "RECOVERY UNDERWAY"      // CAN_STATE_RECOVERING
  };
  while(true)
  {
    can_status_info_t status;
    can_get_status_info(&status);
    if (status.state == CAN_STATE_BUS_OFF)
    {
      // When the bus is OFF we need to initiate recovery, transmit is
      // not possible when in this state.
      printf("ESP32-CAN: initiating recovery\n");
      can_initiate_recovery();
    }
    printf("ESP32-CAN: rx-q:%d, tx-q:%d, rx-err:%d, tx-err:%d, arb-lost:%d, bus-err:%d, state: %s\n",
             status.msgs_to_rx, status.msgs_to_tx, status.rx_error_counter, status.tx_error_counter,
             status.arb_lost_count,status.bus_error_count, STATUS_STRINGS[status.state]);
    // sleep for 10sec with option to wake up early
    ulTaskNotifyTake(pdTRUE, pdMS_TO_TICKS(10000));
  }
}

void setup()
{
  ... start CAN peripheral ...

  // start CAN monitor task at priority 2 with 2kb stack.
  xTaskCreate(can_mon, "CAN MON", 2048, nullptr, 2, nullptr);
}

void loop()
{
  can_message_t msg;
  bzero(&msg, sizeof(can_message_t));
  esp_err_t res = can_receive(&message, pdMS_TO_TICKS(250));
  if (res != ESP_OK)
  {
    if (res != ESP_ERR_TIMEOUT)
    {
      printf("CAN-RX: Error: %d (%s)\n", res, esp_err_to_name(res));
    }
    return;
  }
  printf("CAN-RX id:%08x, flags:%04x, dlc:%02d, data:%02x%02x%02x%02x%02x%02x%02x%02x\n",
           msg.identifier, msg.flags, msg.data_length_code,
           msg.data[0], msg.data[1], msg.data[2], msg.data[3],
           msg.data[4], msg.data[5], msg.data[6], msg.data[7]);
}

The CAN MON task is set to priority 2 which is one higher than loopTask (setup and loop run from this task) so it will have priority when the ~10sec timer runs out. When the bus is OFF or in Recovery state you won't be able to use the CAN bus.

The above example is a very simplified version of my CAN driver which I wrote a while ago (arduino-esp32 1.0.0 timeframe). A more recent version is here which is a lower level driver for higher performance and more direct integration in the OpenMRN stack.

[sandervandegeijn]

On a esp32 you can also put of the tasks on the second core if you would be worried about performance

[atanisoft]

By using xTaskCreate it will float between cores. The loopTask runs pinned to core 1 (APP_CPU), so you could use xTaskCreatePinnedToCore(...., PRO_CPU_NUM) have the monitoring task run on core 0.

[stuartpittaway]

I think this may be hardware related, as I'm not receiving anything on the ESP side any more. I don't think the ESP side is transmitting ACK to the CAN bus when data is sent from the Arduino, as the bus is always busy with data, even though I have a 5 second gap between transmits.

I also can't get a packet received when sent ESP to Arduino.

Bit more digging.

On the ESP, it reports sending 1 packet, then a timeout whilst trying to receive and then the bus is off/recovery needed, eventually goes into STOPPED.

Transmit ok      
Reply Timeout
rx-q:0, tx-q:1, rx-err:0, tx-err:128, arb-lost:0, bus-err:26, state: OFF / RECOVERY NEEDEDESP32-CAN: initiating recovery103Transmit failed
Timeout
rx-q:0, tx-q:0, rx-err:0, tx-err:0, arb-lost:0, bus-err:26, state: STOPPED103Transmit failed
Timeout
rx-q:0, tx-q:0, rx-err:0, tx-err:0, arb-lost:0, bus-err:26, state: STOPPED103Transmit failed
Timeout
rx-q:0, tx-q:0, rx-err:0, tx-err:0, arb-lost:0, bus-err:26, state: STOPPED103Transmit failed

[atanisoft]

Double check you don't have TX/RX swapped somehow. I've done that a few times in my test setups on the workbench...

[stuartpittaway]

Finally making some progress, looks like a broken PCB track on one of the data lines!

I'm now reliably receiving frames from the Arduino into the ESP, however it doesn't seem to be ack the frames.

I'm sending a message every second, and on the ESP I can see high receive queue values and the bus-err slowly creeps up.

I seem to be receiving 3 or 4 duplicate messages for every transmit.

On the Arduino, every 4th message fails and I have to reset the MCP chip to keep it alive/sending.

Do you think this could be down to the 3.3volt vs 5volt components?

[atanisoft]

Do you think this could be down to the 3.3volt vs 5volt components?

It could be. I've used a split voltage transceiver so that I can power the bus side independent of the IO side.

I'm now reliably receiving frames from the Arduino into the ESP, however it doesn't seem to be ack the frames.

There shouldn't need to be any sort of explicit ACK in your code unless the protocol you are working with requires it.

I seem to be receiving 3 or 4 duplicate messages for every transmit.

That may be due to the bus errors which the sender is encountering and trying to kick out the same packet again thinking that it didn't get sent. It may be a bug in the library or your code for sending.

[stuartpittaway]

Okay, got to the bottom of it.

I directly connected the TX/RX lines of the MCP2515 to the ESP32 CAN pins - comms works flawlessly.

When I introduce the "real" CAN BUS using SN65HVD230D (3.3volt) talking to MCP2551 (5volt), data from the MCP to SN65 works and packets are received, however the reverse is not true - I only get 1 way comms, so the ESP cannot acknowledge receipt of the packets and the MCP sends duplicates of the data, until it gives up/times out.

The SN65HVD230D operates at 3.3volt and supports the ISO11898-2 standards, using a CANL/H voltages between -4 and +16V.

Reading through this CAN Bus Transceivers Operating from 3.3V it suggests that at 3.3V the output CAN voltages will be 2V to 3V (specified by ISO 11898-2).

However the MCP2551 is an older device, supporting only ISO11898 standards, not ISO11898-2. My assumption is that this device cannot detect the received signal.

Either way, all I was trying to do was test that my PCB circuit works - it does :-)

So shall I swap to using the TJA1051T/3 chip? And where does the TVS diode go for the protection? Between the CANL/H pins?

[atanisoft]

So shall I swap to using the TJA1051T/3 chip?

I would yes, you can add a jumper config option for the user to select 3v3 or 5v bus voltage.

And where does the TVS diode go for the protection? Between the CANL/H pins?

Yes, the connections would be L/H and GND:

[stuartpittaway]

Hi @atanisoft

Quick question on ESP32 - how can I use the I2C_MUTEX_LOCK and SPI mutex locks? In platformio they seem to be unavailable as CONFIG_DISABLE_HAL_LOCKS is set ?

[atanisoft]

@stuartpittaway I believe those are internal driver usage only. App code needs to implement its own locking if the same I2C or SPI instance is used across multiple tasks.

[stuartpittaway]

@stuartpittaway I believe those are internal driver usage only. App code needs to implement its own locking if the same I2C or SPI instance is used across multiple tasks.

Thanks Mike, another question while you are here! Can I use the ESP_LOG functions from inside a task?

[atanisoft]

Can I use the ESP_LOG functions from inside a task?

Yes, that should work just fine. One problem though is since you are using Arduino-esp32 the ESP_LOGX(...) macros are replaced with log_X(...) which is controlled via the core debug level. The TAG parameter for ESP_LOGX(...) is dropped entirely. The only way to work around this would be not to include Arduino.h in every file and instead include esp_log.h only where necessary. If you include Arduino.h it will in turn include esp32-hal-log.h which replaces the macros.

[stuartpittaway]

I found that out yesterday, why they felt the need to change those I don't know!

[chron0]

I am currently building a powerwall as well and want to give diybms a go, especially with a Sunny Island and the needed CAN communication. @stuartpittaway great work with the new controller to have this HW available, I think the code should be somehwat ez to adapt from: https://github.com/Tom-evnut/TeslaBMSV2/issues/4

Unfortunately I dont have diybms controller and balancer modules atm, so I cant really dig into trying to adapt the code into diybms. attiny841 doesnt seem to be available in 2021 :/