MQTT SSL Fingerprints causes compilation error

animal6496 commented 4 years ago

Operating environment/Installation (Hass.io/Docker/pip/etc.):

Pip on Ubuntu 19.10. Using the docker environment gives the same compiler error.

ESP (ESP32/ESP8266, Board/Sonoff):

I've tried with D1-mini and generic Sonoff.

Affected component:

Affects mqtt component https://esphome.io/components/mqtt.html

Description of problem: I'm trying to set up a link to my MQTT broker with TLS using fingerprint validation. When adding the ssl_fingerprints directive to the yaml file the compiler raises an error on ESPAsyncTCP.cpp.

Problem-relevant YAML-configuration entries:

PASTE YAML FILE HERE
esphome:
  name: test
  platform: ESP8266
  board: d1_mini

  # Use the recommended  version
  arduino_version: recommended

wifi:
  ssid: "some_network"
  password: "mysecret"

# Enable logging
logger:

# MQTT Client
mqtt:
  broker: my-broker.local
  port: 8883
  username: test
  password: moresecrets
  discovery: false
  discovery_retain: false
  ssl_fingerprints:
    - c88ddac678f02b05e79f56c40096b9ada29a5b59
  topic_prefix: tele/test/inhouse

Logs (if applicable):

PASTE DEBUG LOG HERE
esphome test.yaml run
INFO Reading configuration test.yaml...
INFO Generating C++ source...
INFO Core config or version changed, cleaning build files...
INFO Deleting test/.pioenvs
INFO Deleting test/.piolibdeps
INFO Compiling app...
INFO Running:  platformio run -d test
Processing test (board: d1_mini; framework: arduino; platform: espressif8266@2.2.3)
------------------------------------------------------------------------------------------------------------------------------------------------------------
HARDWARE: ESP8266 80MHz, 80KB RAM, 4MB Flash
PACKAGES: 
 - framework-arduinoespressif8266 2.20502.0 (2.5.2) 
 - tool-esptool 1.413.0 (4.13) 
 - tool-esptoolpy 1.20600.0 (2.6.0) 
 - toolchain-xtensa 2.40802.200502 (4.8.2)
Looking for ESPAsyncTCP-esphome library in registry
LibraryManager: Installing id=6757 @ 1.2.2
ESPAsyncTCP-esphome @ 1.2.2 has been successfully installed!
Looking for AsyncMqttClient-esphome library in registry
LibraryManager: Installing id=6759 @ 0.8.4
AsyncMqttClient-esphome @ 0.8.4 has been successfully installed!
LibraryManager: Installing id=6757 @ ^1.2.2
Looking for AsyncTCP-esphome library in registry
LibraryManager: Installing id=6798 @ ^1.1.1
AsyncTCP-esphome @ 1.1.1 has been successfully installed!
Looking for ArduinoJson-esphomelib library in registry
LibraryManager: Installing id=3837 @ 5.13.3
ArduinoJson-esphomelib @ 5.13.3 has been successfully installed!
Dependency Graph
|-- <ESPAsyncTCP-esphome> 1.2.2
|   |-- <ESP8266WiFi> 1.0
|-- <ESP8266WiFi> 1.0
|-- <ESP8266mDNS> 1.2
|   |-- <ESP8266WiFi> 1.0
|-- <AsyncMqttClient-esphome> 0.8.4
|   |-- <ESPAsyncTCP-esphome> 1.2.2
|   |   |-- <ESP8266WiFi> 1.0
|-- <ArduinoJson-esphomelib> 5.13.3
Building in release mode
Compiling .pioenvs/test/src/esphome/components/json/json_util.cpp.o
Compiling .pioenvs/test/src/esphome/components/logger/logger.cpp.o
Compiling .pioenvs/test/src/esphome/components/mqtt/custom_mqtt_device.cpp.o
Compiling .pioenvs/test/src/esphome/components/mqtt/mqtt_binary_sensor.cpp.o
Compiling .pioenvs/test/src/esphome/components/mqtt/mqtt_client.cpp.o
Compiling .pioenvs/test/src/esphome/components/mqtt/mqtt_climate.cpp.o
Compiling .pioenvs/test/src/esphome/components/mqtt/mqtt_component.cpp.o
Compiling .pioenvs/test/src/esphome/components/mqtt/mqtt_cover.cpp.o
Compiling .pioenvs/test/src/esphome/components/mqtt/mqtt_fan.cpp.o
Compiling .pioenvs/test/src/esphome/components/mqtt/mqtt_light.cpp.o
Compiling .pioenvs/test/src/esphome/components/mqtt/mqtt_sensor.cpp.o
Compiling .pioenvs/test/src/esphome/components/mqtt/mqtt_switch.cpp.o
Compiling .pioenvs/test/src/esphome/components/mqtt/mqtt_text_sensor.cpp.o
Compiling .pioenvs/test/src/esphome/components/wifi/wifi_component.cpp.o
Compiling .pioenvs/test/src/esphome/components/wifi/wifi_component_esp32.cpp.o
Compiling .pioenvs/test/src/esphome/components/wifi/wifi_component_esp8266.cpp.o
Compiling .pioenvs/test/src/esphome/core/application.cpp.o
Compiling .pioenvs/test/src/esphome/core/component.cpp.o
Compiling .pioenvs/test/src/esphome/core/controller.cpp.o
Compiling .pioenvs/test/src/esphome/core/esphal.cpp.o
Compiling .pioenvs/test/src/esphome/core/helpers.cpp.o
Compiling .pioenvs/test/src/esphome/core/log.cpp.o
Compiling .pioenvs/test/src/esphome/core/preferences.cpp.o
Compiling .pioenvs/test/src/esphome/core/scheduler.cpp.o
Compiling .pioenvs/test/src/esphome/core/util.cpp.o
Compiling .pioenvs/test/src/main.cpp.o
Generating LD script .pioenvs/test/ld/local.eagle.app.v6.common.ld
Compiling .pioenvs/test/lib075/ESP8266WiFi/BearSSLHelpers.cpp.o
Compiling .pioenvs/test/lib075/ESP8266WiFi/CertStoreBearSSL.cpp.o
Compiling .pioenvs/test/lib075/ESP8266WiFi/ESP8266WiFi.cpp.o
Compiling .pioenvs/test/lib075/ESP8266WiFi/ESP8266WiFiAP.cpp.o
Compiling .pioenvs/test/lib075/ESP8266WiFi/ESP8266WiFiGeneric.cpp.o
Compiling .pioenvs/test/lib075/ESP8266WiFi/ESP8266WiFiMulti.cpp.o
Compiling .pioenvs/test/lib075/ESP8266WiFi/ESP8266WiFiSTA-WPS.cpp.o
Compiling .pioenvs/test/lib075/ESP8266WiFi/ESP8266WiFiSTA.cpp.o
Compiling .pioenvs/test/lib075/ESP8266WiFi/ESP8266WiFiScan.cpp.o
Compiling .pioenvs/test/lib075/ESP8266WiFi/WiFiClient.cpp.o
Compiling .pioenvs/test/lib075/ESP8266WiFi/WiFiClientSecureAxTLS.cpp.o
Compiling .pioenvs/test/lib075/ESP8266WiFi/WiFiClientSecureBearSSL.cpp.o
Compiling .pioenvs/test/lib075/ESP8266WiFi/WiFiServer.cpp.o
Compiling .pioenvs/test/lib075/ESP8266WiFi/WiFiServerSecureAxTLS.cpp.o
Compiling .pioenvs/test/lib075/ESP8266WiFi/WiFiServerSecureBearSSL.cpp.o
Compiling .pioenvs/test/lib075/ESP8266WiFi/WiFiUdp.cpp.o
Compiling .pioenvs/test/lib3ff/ESPAsyncTCP-esphome_ID6757/AsyncPrinter.cpp.o
Compiling .pioenvs/test/lib3ff/ESPAsyncTCP-esphome_ID6757/ESPAsyncTCP.cpp.o
Compiling .pioenvs/test/lib3ff/ESPAsyncTCP-esphome_ID6757/ESPAsyncTCPbuffer.cpp.o
Compiling .pioenvs/test/lib3ff/ESPAsyncTCP-esphome_ID6757/SyncClient.cpp.o
Compiling .pioenvs/test/lib3ff/ESPAsyncTCP-esphome_ID6757/tcp_axtls.c.o
Compiling .pioenvs/test/lib909/ESP8266mDNS/ESP8266mDNS.cpp.o
Compiling .pioenvs/test/lib909/ESP8266mDNS/ESP8266mDNS_Legacy.cpp.o
.piolibdeps/test/ESPAsyncTCP-esphome_ID6757/src/ESPAsyncTCP.cpp: In member function 'err_t AsyncServer::_poll(tcp_pcb*)':
.piolibdeps/test/ESPAsyncTCP-esphome_ID6757/src/ESPAsyncTCP.cpp:1324:31: error: no matching function for call to 'AsyncClient::_recv(tcp_pcb*&, pbuf*&, int)'
         c->_recv(pcb, p->pb, 0);
                               ^
.piolibdeps/test/ESPAsyncTCP-esphome_ID6757/src/ESPAsyncTCP.cpp:1324:31: note: candidate is:
.piolibdeps/test/ESPAsyncTCP-esphome_ID6757/src/ESPAsyncTCP.cpp:565:6: note: void AsyncClient::_recv(std::shared_ptr<ACErrorTracker>&, tcp_pcb*, pbuf*, err_t)
 void AsyncClient::_recv(std::shared_ptr<ACErrorTracker>& errorTracker, tcp_pcb* pcb, pbuf* pb, err_t err) {
      ^
.piolibdeps/test/ESPAsyncTCP-esphome_ID6757/src/ESPAsyncTCP.cpp:565:6: note:   candidate expects 4 arguments, 3 provided
*** [.pioenvs/test/lib3ff/ESPAsyncTCP-esphome_ID6757/ESPAsyncTCP.cpp.o] Error 1
================================================================ [FAILED] Took 7.19 seconds ================================================================

I 'm using esphome 1.14.3 and tried to use other Arduino libraries (recommended, latest) but all resulted in same error message. Additional information and things you've tried:

As stated above I've tried different boards, esphome (via pip) and dockerized environment and I've tried different Arduino Library versions including latest. I did some searches on the web and it looks the problem is reported as solved Oct 2019 in version 1.2.2. but I've still the same compiler errors. Here on the esphome the discussion is closed and only open for developers since summer 2019. I've no idea how to tackle this myself any help appreciated.

trinitor commented 4 years ago

I can confirm the bug. Verified with NodeMCU and docker (latest and dev).

The build process will work when the ssl_fingerprints section in the config file is removed. The device will connect to the mqtt server on the tls port 8883, but the connection attempt will fail. I cannot get a encrypted connection to a mqtt server. Regardless if add the certificate fingerprint to the config or if it is not present in the config.

Here are my config and the needed docker commands to reproduce the issue.

root@linux:/tmp# cat test.yaml

esphome:
  name: test
  build_path: ./builds/test
  platform: ESP8266
  board: nodemcuv2

wifi:
  ssid: "myssid"
  password: "mypassword"

mqtt:
  broker: "mqtt.example.com"
  username: "esphome"
  password: "bar"
  port: "8883"
  ssl_fingerprints: 
    - 1234567890123458789012345678901234567890

root@linux:/tmp# docker run -it --rm --device=/dev/ttyUSB0 -v "$PWD":/config esphome/esphome:dev /config/test.yaml mqtt-fingerprint

INFO Reading configuration /config/test.yaml...
INFO Getting fingerprint from mqtt.example.com:8883
SHA1 Fingerprint: 1234567890123458789012345678901234567890
Copy the string above into mqtt.ssl_fingerprints section of /config/test.yaml

root@linux:/tmp# docker run -it --rm --device=/dev/ttyUSB0 -v "$PWD":/config esphome/esphome:dev /config/test.yaml run

[...]
PlatformIO has been successfully upgraded to 4.1.0!
[...]
Processing neopixel_livingroom (board: nodemcuv2; framework: arduino; platform: espressif8266@2.2.3)
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
HARDWARE: ESP8266 80MHz, 80KB RAM, 4MB Flash
Looking for ESPAsyncTCP-esphome library in registry
LibraryManager: Installing id=6757 @ 1.2.2
ESPAsyncTCP-esphome @ 1.2.2 has been successfully installed!
Looking for AsyncMqttClient-esphome library in registry
LibraryManager: Installing id=6759 @ 0.8.4
AsyncMqttClient-esphome @ 0.8.4 has been successfully installed!
LibraryManager: Installing id=6757 @ ^1.2.2
Looking for AsyncTCP-esphome library in registry
LibraryManager: Installing id=6798 @ ^1.1.1
AsyncTCP-esphome @ 1.1.1 has been successfully installed!
Looking for ArduinoJson-esphomelib library in registry
LibraryManager: Installing id=3837 @ 5.13.3
ArduinoJson-esphomelib @ 5.13.3 has been successfully installed!
Dependency Graph
|-- <ESPAsyncTCP-esphome> 1.2.2
|   |-- <ESP8266WiFi> 1.0
|-- <ESP8266WiFi> 1.0
|-- <ESP8266mDNS> 1.2
|   |-- <ESP8266WiFi> 1.0
|-- <AsyncMqttClient-esphome> 0.8.4
|   |-- <ESPAsyncTCP-esphome> 1.2.2
|   |   |-- <ESP8266WiFi> 1.0
|-- <ArduinoJson-esphomelib> 5.13.3
[...]
Compiling .pioenvs/neopixel_livingroom/lib813/ESPAsyncTCP-esphome_ID6757/SyncClient.cpp.o
Compiling .pioenvs/neopixel_livingroom/lib813/ESPAsyncTCP-esphome_ID6757/tcp_axtls.c.o
Compiling .pioenvs/neopixel_livingroom/lib0d3/ESP8266mDNS/ESP8266mDNS.cpp.o
.piolibdeps/neopixel_livingroom/ESPAsyncTCP-esphome_ID6757/src/ESPAsyncTCP.cpp: In member function 'err_t AsyncServer::_poll(tcp_pcb*)':
.piolibdeps/neopixel_livingroom/ESPAsyncTCP-esphome_ID6757/src/ESPAsyncTCP.cpp:1324:31: error: no matching function for call to 'AsyncClient::_recv(tcp_pcb*&, pbuf*&, int)'                   
         c->_recv(pcb, p->pb, 0);                                                                                                                                                              
                               ^                                                                                                                                                               
.piolibdeps/neopixel_livingroom/ESPAsyncTCP-esphome_ID6757/src/ESPAsyncTCP.cpp:1324:31: note: candidate is:                                                                                    
.piolibdeps/neopixel_livingroom/ESPAsyncTCP-esphome_ID6757/src/ESPAsyncTCP.cpp:565:6: note: void AsyncClient::_recv(std::shared_ptr<ACErrorTracker>&, tcp_pcb*, pbuf*, err_t)                  
 void AsyncClient::_recv(std::shared_ptr<ACErrorTracker>& errorTracker, tcp_pcb* pcb, pbuf* pb, err_t err) {                                                                                   
      ^                                                                                                                                                                                        
.piolibdeps/neopixel_livingroom/ESPAsyncTCP-esphome_ID6757/src/ESPAsyncTCP.cpp:565:6: note:   candidate expects 4 arguments, 3 provided                                                        
*** [.pioenvs/neopixel_livingroom/lib813/ESPAsyncTCP-esphome_ID6757/ESPAsyncTCP.cpp.o] Error 1                                                                                                 
================================================================================= [FAILED] Took 44.81 seconds =================================================================================

Nephiel commented 4 years ago

For what it's worth, I tried the changes described here: https://github.com/esphome/issues/issues/786#issuecomment-549071645

After patching .piolibdeps/<device>/ESPAsyncTCP-esphome_ID6757/, compilation is successful but the resulting firmware never completes a MQTT connection.
After patching .piolibdeps/<device>/AsyncMqttClient-esphome_ID6759/ as well, MQTT connection is established, but is very unstable: protocol errors, timeouts, and garbage in message contents.

Nephiel commented 3 years ago

Looks like the problem lies deeper in the async-mqtt-client library, see https://github.com/marvinroger/async-mqtt-client/issues/196

IMHO moving from async-mqtt-client to PangolinMQTT could fix this and many other MQTT-related issues. Author kindly provided a quickstart for this.

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

michaelpayne02 commented 1 year ago

Any indication of this getting fixed?

michaelpayne02 commented 1 year ago

EasyasyncTCP has not been updated since 2019. I think it might be wise to move on to a smaller SSL library like BearSSL.

Some useful starting points: https://github.com/esp8266/Arduino/blob/master/libraries/ESP8266WebServer/examples/HelloServerBearSSL/HelloServerBearSSL.ino

https://arduino-esp8266.readthedocs.io/en/latest/esp8266wifi/bearssl-client-secure-class.html

esphome / issues

MQTT SSL Fingerprints causes compilation error #1253