Integrate fingerprint and gateway

[rospogrigio]

Hi @formatBCE and Merry Xmas! I need your expertise for something I'd like to accomplish. I wanted to integrate a fingerprint scanner (specifically the R503, see here: https://esphome.io/components/fingerprint_grow.html) and the BLE gateway within the same ESP32 using ESPHome. So far I could make it run using the provided example code, but when I tried to introduce the gateway features the firmware fails to compile. In detail, when I add:

  includes:
    - airbnk-gateway.h
  libraries:
    - "h2zero/NimBLE-Arduino"

I get lots of compilation errors (see next post). So, here are 2 questions for you: 1) do you think this is feasible, first of all? 2) do you know why I get these compilation errors, and how I can fix these?

Thank you in advance for your support! rospogrigio

[rospogrigio]

...and here are the errors I get when compling:

Compiling /data/fingerprint-reader/.pioenvs/fingerprint-reader/lib523/NimBLE-Arduino/NimBLE2904.cpp.o
Compiling /data/fingerprint-reader/.pioenvs/fingerprint-reader/lib523/NimBLE-Arduino/NimBLEAddress.cpp.o
Compiling /data/fingerprint-reader/.pioenvs/fingerprint-reader/lib523/NimBLE-Arduino/NimBLEAdvertisedDevice.cpp.o
Compiling /data/fingerprint-reader/.pioenvs/fingerprint-reader/lib523/NimBLE-Arduino/NimBLEAdvertising.cpp.o
In file included from src/main.cpp:76:
src/airbnk-gateway.h:110:78: error: expected class-name before '{' token
 class AirbnkGatewayNodeComponent : public Component, public CustomMQTTDevice {
                                                                              ^
src/airbnk-gateway.h:152:25: error: 'JsonObject' has not been declared
     void sendBlePayload(JsonObject &root) {
                         ^~~~~~~~~~
src/airbnk-gateway.h:237:21: error: 'JsonObject' has not been declared
     void on_command(JsonObject root) {
                     ^~~~~~~~~~
src/airbnk-gateway.h: In member function 'bool AirbnkGatewayNodeComponent::reportDevice(NimBLEAdvertisedDevice&)':
src/airbnk-gateway.h:125:48: error: 'JsonObject' has not been declared
         publish_json(advert_topic.c_str(), [=](JsonObject root) {
                                                ^~~~~~~~~~
src/airbnk-gateway.h: In lambda function:
src/airbnk-gateway.h:126:27: error: assignment of read-only location '"mac"[root]'
             root["mac"] = mac_address;
                           ^~~~~~~~~~~
src/airbnk-gateway.h:127:28: error: assignment of read-only location '"rssi"[root]'
             root["rssi"] = RSSI;
                            ^~~~
src/airbnk-gateway.h:128:28: error: assignment of read-only location '"data"[root]'
             root["data"] = pHex;
                            ^~~~
src/airbnk-gateway.h: In member function 'bool AirbnkGatewayNodeComponent::reportDevice(NimBLEAdvertisedDevice&)':
src/airbnk-gateway.h:125:9: error: 'publish_json' was not declared in this scope
         publish_json(advert_topic.c_str(), [=](JsonObject root) {
         ^~~~~~~~~~~~
src/airbnk-gateway.h: In member function 'void AirbnkGatewayNodeComponent::sendCommandResult(boolean, std::__cxx11::string, int, std::__cxx11::string)':
src/airbnk-gateway.h:135:17: error: 'is_connected' was not declared in this scope
         while (!is_connected()) { // WiFi most likely was disconnected during command write. Meh.
                 ^~~~~~~~~~~~
src/airbnk-gateway.h:135:17: note: suggested alternative:
In file included from src/esphome.h:25,
                 from src/main.cpp:3:
src/esphome/components/network/util.h:10:6: note:   'esphome::network::is_connected'
 bool is_connected();
      ^~~~~~~~~~~~
In file included from src/main.cpp:76:
src/airbnk-gateway.h:138:56: error: 'JsonObject' has not been declared
         publish_json(command_result_topic.c_str(), [=](JsonObject root) {
                                                        ^~~~~~~~~~
src/airbnk-gateway.h: In lambda function:
src/airbnk-gateway.h:139:31: error: assignment of read-only location '"success"[root]'
             root["success"] = success;
                               ^~~~~~~
src/airbnk-gateway.h:140:41: error: assignment of read-only location '"error"[root]'
             root["error"] = error.c_str();
                                         ^
src/airbnk-gateway.h:141:28: error: assignment of read-only location '"sign"[root]'
             root["sign"] = sign;
                            ^~~~
src/airbnk-gateway.h:142:27: error: assignment of read-only location '"mac"[root]'
             root["mac"] = lock_mac;
                           ^~~~~~~~
src/airbnk-gateway.h:143:47: error: assignment of read-only location '"lockStatus"[root]'
             root["lockStatus"] = status.c_str();
                                               ^
src/airbnk-gateway.h: In member function 'void AirbnkGatewayNodeComponent::sendCommandResult(boolean, std::__cxx11::string, int, std::__cxx11::string)':
src/airbnk-gateway.h:138:9: error: 'publish_json' was not declared in this scope
         publish_json(command_result_topic.c_str(), [=](JsonObject root) {
         ^~~~~~~~~~~~
src/airbnk-gateway.h: In member function 'void AirbnkGatewayNodeComponent::sendBlePayload(int&)':
src/airbnk-gateway.h:163:44: error: invalid conversion from 'char' to 'const char*' [-fpermissive]
         const char* cmnd1 = root["command1"];
                             ~~~~~~~~~~~~~~~^
src/airbnk-gateway.h:164:44: error: invalid conversion from 'char' to 'const char*' [-fpermissive]
         const char* cmnd2 = root["command2"];
                             ~~~~~~~~~~~~~~~^
src/airbnk-gateway.h: In member function 'virtual void AirbnkGatewayNodeComponent::setup()':
src/airbnk-gateway.h:280:9: error: 'subscribe_json' was not declared in this scope
         subscribe_json(command_topic, &AirbnkGatewayNodeComponent::on_command);
         ^~~~~~~~~~~~~~
Compiling /data/fingerprint-reader/.pioenvs/fingerprint-reader/lib523/NimBLE-Arduino/NimBLEBeacon.cpp.o
Compiling /data/fingerprint-reader/.pioenvs/fingerprint-reader/lib523/NimBLE-Arduino/NimBLECharacteristic.cpp.o
Compiling /data/fingerprint-reader/.pioenvs/fingerprint-reader/lib523/NimBLE-Arduino/NimBLEClient.cpp.o
*** [/data/fingerprint-reader/.pioenvs/fingerprint-reader/src/main.cpp.o] Error 1

[rospogrigio]

FYI, my YAML file is something like this:

esphome:
  name: fingerprint-reader

esp32:
  board: esp32dev
  framework:
    type: arduino

# Enable logging
logger:

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "Airbnk-Gateway Fallback Hotspot"
    password: "***HIDDEN***"

ota:
  password: "***HIDDEN***"

captive_portal:

mqtt:
  broker: ***HIDDEN***
  username: !secret mqtt_user # TODO replace with your MQTT user or remove
  password: !secret mqtt_password # TODO replace with your MQTT password or remove
  discovery: false

uart:
  rx_pin: GPIO17
  tx_pin: GPIO22
  baud_rate: 57600

fingerprint_grow:
  sensing_pin: GPIO16
  on_finger_scan_matched:
    - homeassistant.event:
        event: esphome.test_node_finger_scan_matched
        data:
          finger_id: !lambda 'return finger_id;'
          confidence: !lambda 'return confidence;'
  on_finger_scan_unmatched:
    - homeassistant.event:
        event: esphome.test_node_finger_scan_unmatched
  on_enrollment_scan:
    - homeassistant.event:
        event: esphome.test_node_enrollment_scan
        data:
          finger_id: !lambda 'return finger_id;'
          scan_num: !lambda 'return scan_num;'
  on_enrollment_done:
    - homeassistant.event:
        event: esphome.test_node_enrollment_done
        data:
          finger_id: !lambda 'return finger_id;'
  on_enrollment_failed:
    - fingerprint_grow.aura_led_control:
        state: FLASHING
        speed: 25
        color: RED
        count: 4
    - homeassistant.event:
        event: esphome.test_node_enrollment_failed
        data:
          finger_id: !lambda 'return finger_id;'

api:
  encryption:
    key: "***HIDDEN***"
  services:
  - service: enroll
    variables:
      finger_id: int
      num_scans: int
    then:
      - fingerprint_grow.enroll:
          finger_id: !lambda 'return finger_id;'
          num_scans: !lambda 'return num_scans;'
  - service: cancel_enroll
    then:
      - fingerprint_grow.cancel_enroll:
 [...]

[rospogrigio]

Nevermind, now I did it and it works! Probably it was a matter of order of the YAML sections... cool! Thank you anyway!

[formatBCE]

Great job! Yes, order matters, all libraries should go to the top section.

Does scanner also use BLE, or it's something different? I see you are just sending events to HA. Is it something inbuilt into ESPHome?

P.S. ah I see, it's wired to ESP32. Nice!

[rospogrigio]

Yes, it gets wired to the ESP32. My plan is to install it outside the door and do something like "left thumb opens the door, right thumb closes the door" and go keyless 😉 Yes, the events are sent to HA so you can handle them as desired using integrations or Node-red. Thank you for the attention!

[rospogrigio]

Hi guys (@formatBCE , but also @ofirsnb maybe can be of any help). I finally installed the fingerprint reader outside my door, and I'm having issues. Basically, everytime the ESP32 sends a command to the lock, it disconnects from the wifi. Here is a snippet:

[16:44:29][D][airbnk_mqtt:123]: Sending adv
[16:44:31][D][airbnk_mqtt:105]: BLE scan heartbeat
[16:44:36][D][airbnk_mqtt:105]: BLE scan heartbeat
[16:44:40][D][fingerprint_grow:092]: Scan and match
[16:44:40][D][fingerprint_grow:123]: Getting image 1
[16:44:40][D][fingerprint_grow:144]: Processing image 1
[16:44:41][D][airbnk_mqtt:123]: Sending adv
[16:44:41][I][fingerprint_grow:148]: Processed image 1
[16:44:41][D][fingerprint_grow:101]: Fingerprint matched
[16:44:41][D][sensor:127]: 'Last finger ID': Sending state 0.00000  with 0 decimals of accuracy
[16:44:41][D][sensor:127]: 'Confidence of last fingerprint read': Sending state 98.00000  with 0 decimals of accuracy
[16:44:41][D][airbnk_mqtt:105]: BLE scan heartbeat
[16:44:41][D][airbnk_mqtt:238]: Got command
[16:44:41][D][airbnk_mqtt:169]: Sending to:
[16:44:41][D][airbnk_mqtt:170]: 58:xx:xx:xx:xx:3a
[16:44:42][D][airbnk_mqtt:178]: Connected to lock.
[16:44:42][D][fingerprint_grow:305]: Setting Aura LED
[16:44:42][D][fingerprint_grow:309]: Aura LED set
INFO 192.168.1.37: Error while reading incoming messages: Error while reading data: [Errno 104] Connection reset by peer
INFO 192.168.1.37: Ping Failed: Error while reading data: [Errno 104] Connection reset by peer
INFO Disconnected from ESPHome API for 192.168.1.37
WARNING Disconnected from API
INFO Successfully connected to 192.168.1.37
[16:44:51][D][fingerprint_grow:092]: Scan and match
[16:44:51][D][fingerprint_grow:123]: Getting image 1
[16:44:51][D][fingerprint_grow:144]: Processing image 1
[16:44:52][I][fingerprint_grow:148]: Processed image 1
[16:44:52][D][fingerprint_grow:101]: Fingerprint matched
[16:44:52][D][sensor:127]: 'Last finger ID': Sending state 0.00000  with 0 decimals of accuracy
[16:44:52][D][sensor:127]: 'Confidence of last fingerprint read': Sending state 73.00000  with 0 decimals of accuracy
[16:44:52][D][fingerprint_grow:305]: Setting Aura LED
[16:44:52][D][fingerprint_grow:309]: Aura LED set
[16:44:52][D][text_sensor:067]: 'Fingerprint State': Sending state 'Authorized finger 0, confidence 73'
[16:44:52][D][api:102]: Accepted ::FFFF:192.168.1.30
[16:44:52][D][fingerprint_grow:305]: Setting Aura LED
[16:44:52][D][fingerprint_grow:309]: Aura LED set
[16:44:52][D][api.connection:918]: ESPHome Logs 2022.12.8 (::FFFF:192.168.1.30): Connected successfully
[16:44:54][D][airbnk_mqtt:105]: BLE scan heartbeat
[16:44:56][D][airbnk_mqtt:123]: Sending adv
[16:44:59][D][airbnk_mqtt:105]: BLE scan heartbeat
[16:45:00][D][fingerprint_grow:092]: Scan and match
[16:45:00][D][fingerprint_grow:123]: Getting image 1
[16:45:01][D][fingerprint_grow:144]: Processing image 1
[16:45:01][I][fingerprint_grow:148]: Processed image 1
[16:45:01][D][fingerprint_grow:101]: Fingerprint matched
[16:45:01][D][sensor:127]: 'Last finger ID': Sending state 0.00000  with 0 decimals of accuracy
[16:45:01][D][sensor:127]: 'Confidence of last fingerprint read': Sending state 71.00000  with 0 decimals of accuracy
[16:45:01][D][airbnk_mqtt:238]: Got command
[16:45:01][D][airbnk_mqtt:169]: Sending to:
[16:45:01][D][airbnk_mqtt:170]: 58:xx:xx:xx:xx:3a
[16:45:02][D][fingerprint_grow:305]: Setting Aura LED
[16:45:02][D][fingerprint_grow:309]: Aura LED set
[16:45:04][D][airbnk_mqtt:178]: Connected to lock.
INFO 192.168.1.37: Error while reading incoming messages: Error while reading data: [Errno 104] Connection reset by peer
INFO 192.168.1.37: Ping Failed: Error while reading data: [Errno 104] Connection reset by peer
INFO Disconnected from ESPHome API for 192.168.1.37
WARNING Disconnected from API
INFO Successfully connected to 192.168.1.37
[16:45:14][D][api:102]: Accepted ::FFFF:192.168.1.30
[16:45:15][D][api.connection:918]: ESPHome Logs 2022.12.8 (::FFFF:192.168.1.30): Connected successfully
[16:45:15][D][api:102]: Accepted ::FFFF:192.168.1.30
[16:45:15][D][api.connection:918]: Home Assistant 2023.2.1 (::FFFF:192.168.1.30): Connected successfully
[16:45:16][D][airbnk_mqtt:105]: BLE scan heartbeat
[16:45:20][D][airbnk_mqtt:123]: Sending adv
[16:45:21][D][airbnk_mqtt:105]: BLE scan heartbeat
[16:45:26][D][airbnk_mqtt:105]: BLE scan heartbeat

Now, the fact is that I'm using the old version of airbnk-gateway.h (the one that doesn't try to read the FFF3 characteristic but only relies on ADV messages), but still I'm getting these disconnections (which I didn't notice in my previous tests). Maybe it's a bit too much for the ESP32 to handle? I mean, wifi + BT + serial with the fingerprint? Am I pushing it beyond its limits or is there some parameter tweaking I can try? Or there's nothing else to do, and I shall use one ESP for the fingerprint and another one as gateway to the lock? Thank you in advance for your help

[rospogrigio]

Wait @formatBCE , I am looking at your last commit and I can't find much of a difference: the FFF3 characteristic was being read also before that commit, am I wrong? The only difference I can notice is a delay changed from 700 to 100 ms when retrying to read the characteristic, seems a speedup. Am I missing something?

[formatBCE]

@rospogrigio that non-fff3 version was in separate file, which is deleted. Check this commit: https://github.com/formatBCE/Airbnk-MQTTOpenGateway/commit/0ee260d8bb62b0673351aa5fc541063b7eb1e8b2

After it, i deleted that file.

Disconnects will persist, i believe. Next week my ethernet ESPs will arrive, will check, if it's any better.

[rospogrigio]

I'm a bit confused, can you explain better? In the meantime, I have removed the gateway part from the fingerprint ESP32 and it is now reading all fingerprint without ever disconnecting. Now I'll try the gateway part with the other ESP32 I own and see what happens...

[formatBCE]

@rospogrigio so when I made changes to bypass FFF3, I created another .h file there instead of changing main one. Then, when I (supposedly) made main version better, I deleted that file. So in order to find it, you'll want to go to repository state before that deletion commit. Gimme sec, I will paste the link to deleted file directly.

[formatBCE]

https://github.com/formatBCE/Airbnk-MQTTOpenGateway/blob/0ee260d8bb62b0673351aa5fc541063b7eb1e8b2/src/esphome/airbnk-gateway-nostatus.h

[rospogrigio]

OK now I see. Can I ask why some logs are sent with the "airbnk_mqtt_scanner" tag and other with the "airbnk_mqtt" tag? Is it a mistake or is it supposed to be like this?

[formatBCE]

Yes, sorry, it's just copy-paste mistake.

[ofirsnb]

Hey @rospogrigio , sorry for the late reply. I actually use the same fingerprint (R503), on an esp32 with ESPhome's bluetooth-proxy component together. Although I currently use the bluetooth-proxy as Passive BLE and of course it's not as heavy as Active BLE, I don't believe it's an "overwhelming" situation like you described (wifi+bt+serial). I don't have any disconnections, or panics or such. But of course I could be wrong:/ Our setups are a bit different at the moment. Once I'll get the lock I can flash with your exact setup, send commands to the lock, and see if that happens to me as well.

[rospogrigio]

Thank you @ofirsnb ! Actually I believe that the critical part is what @formatBCE already explained. i.e. wifi and BT sharing the antenna time so that when BT is reading FFF3 characteristic the wifi connection drops. If you could find a solution for this I believe everybody would benefit from it. Keep us posted please, thank you!