How to use softwareserial?

[marcus2208]

Installation specifics

• Heating type: Vitoladens 300-T with Vitotronic 200 KW1
• Protocol: KW
• Board: ESP8266 (Wemos D1 mini clone)
• Hardware: The one in your example, but connected to (RX) GPIO4 and (TX) GPIO5
• Modification: Softwareserial for optolink and telnet for output

Symptom

What happens? Timeout on all datapoints
When does it happen? Always

The softwareserial and the hardware connected to is confirmed working, because I can communicate via telnet to Vitotronic, without vitowifi, using a different sketch. I just can´t find out, how to pass the softwareserial to your library, so vitowifi is working with softwareserial. When I try, at least the IR-LED is flashing, so someting is working already. But all datapoint's results are timeouts. My guess is, vitowifi is using the softwareserial as hardwareserial, resulting in SERIAL_8E2 instead of SWSERIAL_8E2. But since I´m not a high-level-skilled C programmer, I might be wrong.

My code

#include <ESP8266WiFi.h>
#include "SoftwareSerial.h"
const int sRX = 4; // software RX on D2(GPIO 4)
const int sTX = 5; // software TX on D1(GPIO 5)
EspSoftwareSerial::UART Optolink(sRX, sTX);

#include <WiFiManager.h>
#include "BaseOTA.h"
#include <VitoWiFi.h>

#define MAX_SRV_CLIENTS 1
WiFiManager wifiManager;
WiFiServer server(23);
WiFiClient serverClient;
const char deviceName[] = "Vitotronic";

VitoWiFi::VitoWiFi<VitoWiFi::VS1> vitoWiFi(&Optolink);
bool readValues = false;
uint8_t datapointIndex = 0;
unsigned long lastReceived = 0;

VitoWiFi::Datapoint datapoints[] = {
  VitoWiFi::Datapoint("outsidetemp", 0x5525, 2, VitoWiFi::div10),
  VitoWiFi::Datapoint("boilertemp", 0x0810, 2, VitoWiFi::div10),
  VitoWiFi::Datapoint("watertemp", 0x0812, 2, VitoWiFi::div10),
  VitoWiFi::Datapoint("heatpump", 0x2906, 1, VitoWiFi::noconv),
  VitoWiFi::Datapoint("loadpump", 0x0845, 1, VitoWiFi::noconv),
  VitoWiFi::Datapoint("circpump", 0x0846, 1, VitoWiFi::noconv),
  VitoWiFi::Datapoint("burnerrun", 0x0842, 1, VitoWiFi::noconv),
  VitoWiFi::Datapoint("burnererror", 0x0883, 1, VitoWiFi::noconv)
};

void onResponse(const uint8_t* data, uint8_t length, const VitoWiFi::Datapoint& request) {
  // raw data can be accessed through the 'response' argument
  serverClient.print("Raw data received:");
  for (uint8_t i = 0; i < length; ++i) {
    serverClient.printf(" %02x", data[i]);
  }
  serverClient.print("\n");

  // the raw data can be decoded using the datapoint. Be sure to use the correct type
  serverClient.printf("%s: ", request.name());
  if (request.converter() == VitoWiFi::div10) {
    float value = request.decode(data, length);
    serverClient.printf("%.1f\n", value);
  } else if (request.converter() == VitoWiFi::noconv) {
    // in this example, the response is one byte
    serverClient.printf("%s\n", (data[0] > 0) ? "ON" : "OFF");
  }
}

void onError(VitoWiFi::OptolinkResult error, const VitoWiFi::Datapoint& request) {
  serverClient.printf("Datapoint \"%s\" error: ", request.name());
  if (error == VitoWiFi::OptolinkResult::TIMEOUT) {
    serverClient.print("timeout\n");
  } else if (error == VitoWiFi::OptolinkResult::LENGTH) {
    serverClient.print("length\n");
  } else if (error == VitoWiFi::OptolinkResult::NACK) {
    serverClient.print("nack\n");
  } else if (error == VitoWiFi::OptolinkResult::CRC) {
    serverClient.print("crc\n");
  } else if (error == VitoWiFi::OptolinkResult::ERROR) {
    serverClient.print("error\n");
  }
}

void setup() {
  pinMode(LED_BUILTIN, OUTPUT);
  digitalWrite(LED_BUILTIN, HIGH);

  delay(5000); //BOOT WAIT

  WiFi.setSleepMode(WIFI_NONE_SLEEP);     // disable WiFi sleep for more performance
  WiFi.setOutputPower(10);                // 0 to 20.5 [dBm] inclusive, 0.25 [dBm] steps
  WiFi.hostname(deviceName);

  wifiManager.setDebugOutput(false);
  wifiManager.autoConnect(deviceName);

  OTAinit();

    server.begin();
    server.setNoDelay(true);

  vitoWiFi.onResponse(onResponse);
  vitoWiFi.onError(onError);
  vitoWiFi.begin();
}

void loop() {
  ArduinoOTA.handle();

    if (server.hasClient()) {
        AcceptConnection();
    }

  if (millis() - lastReceived > 60000UL) {  // read all values every 60 seconds
    lastReceived = millis();
    readValues = true;
    datapointIndex = 0;
  }

  if (readValues) {
    if (vitoWiFi.read(datapoints[datapointIndex])) {
      ++datapointIndex;
    }

    if (datapointIndex >= 8) {
      readValues = false;
    }
  }

  vitoWiFi.loop();
}

void AcceptConnection() {
    if (serverClient && serverClient.connected()) {
        serverClient.stop();
    }

    serverClient = server.accept();
}

The essential parts

#include "SoftwareSerial.h"
const int sRX = 4; // software RX on D2(GPIO 4)
const int sTX = 5; // software TX on D1(GPIO 5)
EspSoftwareSerial::UART Optolink(sRX, sTX);
VitoWiFi::VitoWiFi<VitoWiFi::VS1> vitoWiFi(&Optolink);

I´ve read all documentation, bug reports and your source code, but can´t find out how to use softwareserial in that case. It compiles fine, IR-LED is flashing, but all datapoints return timeouts. Maybe you can have a look please?

Thanks in advance!

Edit: I'm using V3-development

[bertmelis]

To be honest, I didn't test softwareserial with v3. At first glance I don't see anything wrong with your code.

The softwareserial library might have changed their API. We might need to specify pins in the begin call instead of the constructor. I'll prepare a possible fix to test.

[marcus2208]

Wow, that was a really fast response!

I don't think the pin assignment is a problem. The IR-LED is flashing, so the pin for TX at least should be right. But as I said, I'm a beginner in C and might be wrong.

[bertmelis]

I actually don't have a clue at this point.

Would it be possible to enable debug logging? Are you using Arduino IDE or Platformio?

[marcus2208]

I'm using Arduino IDE, latest 1.8.x Debug logging... Either I have to take my notebook down to the basement, or I have to try it over telnet. First one seems easier, last one more comfortable. Right now I'm on the way to a customer, will try that at the weekend. Thank you for having a look into this.

[bertmelis]

Could you test with this branch: https://github.com/bertmelis/VitoWiFi/tree/fix-softwareserial

edit1: just a minute, cleaning up dirty work edit2: ready to test

[marcus2208]

I'll try the fix-softwareserial as soon as I'm back home. Saturday or Sunday I think.

[marcus2208]

I´ve just tried the fix-softwareserial branch. Sadly nothing changed, still all timeouts. Now, for the debugging I need some instructions please. As far as I understand, I have to "#define DEBUG_VITOWIFI" and choose my debug port, e.g. "serial" in Arduino IDE? What about debug level in IDE? Leave it at "None"?

EDIT: Tried to debug, but I didn´t get any output from vitowifi. Something I´ve done wrong? EDIT2: Just to be clear, the debugging from the ESP itself is working as expected, just no output from vitowifi.

[bertmelis]

Debug port in IDE has to be set to Serial I assume, yes. This port you also use for other output?

Then, edit the config and add #define DEBUG_VITOWIFI just after #pragma once.

[marcus2208]

Okay, I had #define DEBUG_VITOWIFI in my sketch. That was the fault I made. Debugging is working now.

16:49:03.029 -> reading packet OK
16:49:03.176 -> state 0 --> 2
16:49:03.176 -> state 2 --> 4
16:49:03.176 -> state 4 --> 5
16:49:07.042 -> state 5 --> 0
16:49:07.042 -> reading packet OK
16:49:07.390 -> state 0 --> 2
16:49:07.390 -> state 2 --> 4
16:49:07.390 -> state 4 --> 5
16:49:11.068 -> state 5 --> 0
16:49:11.068 -> reading packet OK
16:49:11.670 -> state 0 --> 2
16:49:11.670 -> state 2 --> 4
16:49:11.670 -> state 4 --> 5
16:49:15.028 -> state 5 --> 0
16:49:15.028 -> reading packet OK
16:49:15.885 -> state 0 --> 2
16:49:15.885 -> state 2 --> 4
16:49:15.885 -> state 4 --> 5
16:49:19.042 -> state 5 --> 0
16:49:19.042 -> reading packet OK
16:49:20.098 -> state 0 --> 2
16:49:20.098 -> state 2 --> 4
16:49:20.098 -> state 4 --> 5
16:49:23.055 -> state 5 --> 0
16:49:23.055 -> reading packet OK
16:49:24.359 -> state 0 --> 2
16:49:24.359 -> state 2 --> 4
16:49:24.359 -> state 4 --> 5
16:49:27.022 -> state 5 --> 0
16:49:27.022 -> reading packet OK
16:49:28.574 -> state 0 --> 2
16:49:28.574 -> state 2 --> 4
16:49:28.574 -> state 4 --> 5
16:49:31.029 -> state 5 --> 0
16:49:31.029 -> reading packet OK

EDIT: Serial is for debugging only, nothing else is using it.

[bertmelis]

There might be a problem with the VS1 protocol implementation and not with your software serial.

Are you sure your heater doesn't work with VS2?

[marcus2208]

I'm very sure it's using VS1 aka KW only. It's quite old. The boiler was installed in 2009, but the control unit was recycled from the old boiler, because it's compatible. I don't know exactly how old it is, because I bought the house in 2009. Vitosoft, the OEM software is speaking KW to this control unit.

[bertmelis]

I fixed a mistake in the VS1 protocol. I think it should be better now. I pushed to the same branch. I was skipping state 1 (INIT_ACK).

[marcus2208]

Sounds promising. Will try it later.

[marcus2208]

Still no data

Datapoint "outsidetemp" error: timeout
reading packet OK
state 0 --> 1
state 1 --> 2
state 2 --> 4
state 4 --> 5
state 5 --> 0

Is state 3 missing also?

EDIT: No, state 3 isn´t missing. When vitowifi is idle, I can see state 3 also:

state 2 --> 3
state 3 --> 2
state 2 --> 3
state 3 --> 2
state 2 --> 3
state 3 --> 2

EDIT2: Maybe it´s helpful to have a more complete log: EDIT3: Added timestamps:

20:50:16.024 -> state 2 --> 3
20:50:16.071 -> state 3 --> 2
20:50:16.544 -> state 2 --> 3
20:50:16.544 -> state 3 --> 2
20:50:17.062 -> state 2 --> 3
20:50:17.062 -> state 3 --> 2
20:50:17.537 -> state 2 --> 3
20:50:17.537 -> state 3 --> 2
20:50:18.053 -> state 2 --> 3
20:50:18.053 -> state 3 --> 2
20:50:18.523 -> reading packet OK
20:50:18.523 -> state 2 --> 4
20:50:18.523 -> state 4 --> 5
20:50:22.540 -> state 5 --> 0
20:50:22.540 -> Datapoint "outsidetemp" error: timeout
20:50:22.540 -> reading packet OK
20:50:23.013 -> state 0 --> 1
20:50:23.013 -> state 1 --> 2
20:50:23.013 -> state 2 --> 4
20:50:23.013 -> state 4 --> 5
20:50:26.552 -> state 5 --> 0
20:50:26.552 -> Datapoint "boilertemp" error: timeout
20:50:26.552 -> reading packet OK
20:50:27.492 -> state 0 --> 1
20:50:27.492 -> state 1 --> 2
20:50:27.492 -> state 2 --> 4
20:50:27.539 -> state 4 --> 5
20:50:30.517 -> state 5 --> 0
20:50:30.517 -> Datapoint "watertemp" error: timeout
20:50:30.517 -> reading packet OK
20:50:31.976 -> state 0 --> 1
20:50:31.976 -> state 1 --> 2
20:50:31.976 -> state 2 --> 4
20:50:32.024 -> state 4 --> 5
20:50:34.528 -> state 5 --> 0
20:50:34.528 -> Datapoint "heatpump" error: timeout
20:50:34.528 -> reading packet OK
20:50:36.467 -> state 0 --> 1
20:50:36.467 -> state 1 --> 2
20:50:36.467 -> state 2 --> 4
20:50:36.514 -> state 4 --> 5
20:50:38.540 -> state 5 --> 0
20:50:38.540 -> Datapoint "loadpump" error: timeout
20:50:38.540 -> reading packet OK
20:50:38.728 -> state 0 --> 1
20:50:38.728 -> state 1 --> 2
20:50:38.728 -> state 2 --> 4
20:50:38.775 -> state 4 --> 5
20:50:42.556 -> state 5 --> 0
20:50:42.556 -> Datapoint "circpump" error: timeout
20:50:42.556 -> reading packet OK
20:50:43.974 -> state 0 --> 1
20:50:43.974 -> state 1 --> 2
20:50:43.974 -> state 2 --> 4
20:50:43.974 -> state 4 --> 5
20:50:46.521 -> state 5 --> 0
20:50:46.521 -> Datapoint "burnerrun" error: timeout
20:50:48.461 -> state 0 --> 1
20:50:48.461 -> state 1 --> 2
20:50:48.979 -> state 2 --> 3
20:50:48.979 -> state 3 --> 2
20:50:49.502 -> state 2 --> 3
20:50:49.502 -> state 3 --> 2
20:50:49.974 -> state 2 --> 3
20:50:50.021 -> state 3 --> 2
20:50:50.493 -> state 2 --> 3
20:50:50.493 -> state 3 --> 2
20:50:50.966 -> state 2 --> 3

[bertmelis]

Sorry, I'll analyze again tomorrow. I already turned off my computer.

For easier navigation: here is the link to the protocol: https://github.com/openv/openv/wiki/Protokoll-KW https://github.com/sarnau/InsideViessmannVitosoft/blob/main/VitosoftCommunication.md

[marcus2208]

Of course. Have a good night!

[bertmelis]

Did some reading. The timing is off with a factor 10 (500ms instead of 50). I will fix tomorrow.

[marcus2208]

I tried to fix myself, but failed. In VS1.cpp, line 233:

} else if (_currentMillis - _lastMillis > 50) {

Sadly, still the same error. But probably thats not the only change is nessesary? Will go to sleep now. Thanks for your support so far. Tomorrow we´re going to fix it, for sure. ;-)

[bertmelis]

The weekend was busier than expected... I simplified the VS1 protocol implementation. You're the chosen one to test.

[marcus2208]

Don't worry, weekend is weekend. Noone expected anything. I'm happy to test later, after daily business.

EDIT: I´ve tested latest changes, sadly still the same. Timeout.

15:46:27.154 -> state 0 --> 1
15:46:27.154 -> state 1 --> 2
15:46:27.201 -> state 2 --> 0
15:46:30.121 -> state 0 --> 1
15:46:30.121 -> state 1 --> 2
15:46:30.168 -> state 2 --> 0
15:46:33.129 -> state 0 --> 1
15:46:33.129 -> state 1 --> 2
15:46:33.176 -> state 2 --> 0
15:46:35.859 -> reading packet OK
15:46:36.143 -> state 0 --> 1
15:46:36.143 -> state 1 --> 2
15:46:36.143 -> state 2 --> 3
15:46:36.143 -> state 3 --> 4
15:46:39.851 -> state 4 --> 0
15:46:39.851 -> Datapoint "outsidetemp" error: timeout
15:46:39.851 -> reading packet OK
15:46:41.358 -> state 0 --> 1
15:46:41.358 -> state 1 --> 2
15:46:41.358 -> state 2 --> 3
15:46:41.358 -> state 3 --> 4
15:46:43.890 -> state 4 --> 0
15:46:43.890 -> Datapoint "boilertemp" error: timeout
15:46:43.890 -> reading packet OK
15:46:45.863 -> state 0 --> 1
15:46:45.863 -> state 1 --> 2
15:46:45.863 -> state 2 --> 3
15:46:45.863 -> state 3 --> 4
15:46:47.887 -> state 4 --> 0
15:46:47.887 -> Datapoint "watertemp" error: timeout

[bertmelis]

Strange. You go through the whole loop so you must be reading the ACK from the connection.

I'm not at the computer right now. I'm sorry this takes so long.

Are you sure your send IR-led is working? The heating is receiving the signal?

[marcus2208]

No need to apologize. I really hope it's not a stupid mistake on my side, wasting your time.

I have a different sketch running on this hardware. All it's doing is a telnet so softwareserial bridge. This way I can connect vitosoft wirelessly to my boiler. On Windows side I have hw virtual serial port, a software translating from telnet to a virtual serial port, vitosoft can work with. Works flawlessly. So the hardware is okay. It's exactly the one in your readme file. Tomorrow I can post this sketch also.

I'm not sure the softwareserial is working with vitowifi the way it should.

[bertmelis]

I reworked the protocol again.

Let's assume softwareserial isn't the problem.

[marcus2208]

Nope, still the same. Here is the log, state 2 is missing now, I don´t know if it´s by intention:

15:48:47.559 -> state 0 --> 1
15:48:47.606 -> state 1 --> 0
15:48:49.768 -> state 0 --> 1
15:48:49.815 -> state 1 --> 0
15:48:52.015 -> state 0 --> 1
15:48:52.062 -> state 1 --> 0
15:48:54.269 -> state 0 --> 1
15:48:54.316 -> state 1 --> 0
15:48:56.521 -> state 0 --> 1
15:48:56.567 -> state 1 --> 0
15:48:57.272 -> reading packet OK
15:48:58.732 -> state 0 --> 1
15:48:58.732 -> state 1 --> 3
15:48:58.778 -> state 3 --> 4
15:49:01.270 -> state 4 --> 0
15:49:01.270 -> Datapoint "outsidetemp" error: timeout
15:49:01.270 -> reading packet OK
15:49:01.786 -> state 0 --> 1
15:49:01.786 -> state 1 --> 3
15:49:01.786 -> state 3 --> 4
15:49:05.267 -> state 4 --> 0
15:49:05.267 -> Datapoint "boilertemp" error: timeout
15:49:05.267 -> reading packet OK
15:49:06.254 -> state 0 --> 1
15:49:06.254 -> state 1 --> 3
15:49:06.254 -> state 3 --> 4
15:49:09.268 -> state 4 --> 0
15:49:09.268 -> Datapoint "watertemp" error: timeout

Do you have a Viessmann boiler? I think I´ve read it somewhere. Even if it´s VS2 protocol, it should talk VS1 also. This way we can confirm, the protocol is or isn´t the problem. And... Take your time. I´m not in hurry.

EDIT: Forgot to post the optolink <-> telnet sketch:

#include <ESP8266WiFi.h>
#include "SoftwareSerial.h"
#include <WiFiManager.h>

#define logSer Serial
#define logBaud 74880
#define optoMode SWSERIAL_8E2
#define MAX_SRV_CLIENTS 1

#include "BaseOTA.h"

const char deviceName[] = "Vitotronic";
const int sRX = 4, // software RX on D2(GPIO 4)
          sTX = 5; // software TX on D1(GPIO 5)
bool telnetConnected = false;

WiFiManager wifiManager;
WiFiServer telnetServer(23);
WiFiClient serverClient;
EspSoftwareSerial::UART Optolink(sRX, sTX);

void setup() {
  logSer.begin(logBaud);
  logSer.printf("\033[H\033[J");          // clear terminal
  logSer.println("Setting up Vitotronic Interface...");

  pinMode(LED_BUILTIN, OUTPUT);
  digitalWrite(LED_BUILTIN, HIGH);

  delay(5000);                            // BOOT WAIT

  WiFi.setSleepMode(WIFI_NONE_SLEEP);     // disable WiFi sleep for more performance
  WiFi.setOutputPower(10);                // 0 to 20.5 [dBm] inclusive, 0.25 [dBm] steps
  WiFi.hostname(deviceName);

  wifiManager.setDebugOutput(false);
  wifiManager.autoConnect(deviceName);

  OTAinit();

    telnetServer.begin();
    telnetServer.setNoDelay(true);

  logSer.println("Setup done, Vitotronic ready!");
  logSer.println();
}

void loop() {
  ArduinoOTA.handle();

    if (telnetServer.hasClient()) {
        AcceptConnection();

    } else if (serverClient && serverClient.connected()) {
    ManageConnected();

    } else if (telnetConnected){
    CloseConnection();
  }
}

void AcceptConnection() {
  logSer.println("Telnet connected");
  telnetConnected = true;
  digitalWrite(LED_BUILTIN, LOW);
  Optolink.begin(4800, optoMode);
    if (serverClient && serverClient.connected()) {
        serverClient.stop();
    }

    serverClient = telnetServer.accept();
}

void ManageConnected() {
  size_t rxlen = serverClient.available();
  if (rxlen > 0) {
        uint8_t sbuf[rxlen];
        serverClient.readBytes(sbuf, rxlen);
        Optolink.write(sbuf, rxlen);
    }

  size_t txlen = Optolink.available();
  if (txlen > 0) {
        uint8_t sbuf[txlen];
        Optolink.readBytes(sbuf, txlen);
        serverClient.write(sbuf, txlen);
    }
}

void CloseConnection() {
  telnetConnected = false;
  Optolink.end();
  digitalWrite(LED_BUILTIN, HIGH);
  logSer.println("Telnet disconnected");
}

On Windows side use hw virtual serial port and Vitosoft 300 SID1. The single edition of hw virtual serial port is for free.

Above sketch is working on my hardware, but Vitosoft has some connection problems, resetting the virtual serial port too fast. It´s just not made for that usecase. But once it´s connected, it runs fine.

[bertmelis]

Yes, state 2 is only when you read a second datapoint right after the first.

I can install my raspberry pi again. It's a bit of a chore, but this trial and error has to stop. Next commit is a working protocol.

[marcus2208]

I´m still not sure it´s the protocol. But let´s see. Please remember, it´s low priority. Again, thank you for your support!

[bertmelis]

No worries. It sparked my interest. I'm flashing an SD for my raspberry pi to test.

[bertmelis]

I think I got it working. Some issues with the serial interface on the raspberry pi though, but I don't think it propagates to the esp.

[marcus2208]

Great job! 🥇 It´s reading data now. No more timeouts! Thank you very much!

[bertmelis]

another weight from my shoulders.

The branch is a bit messy now with all different changes. I'll clean up and merge soon.

[bertmelis]

merged updates into main.