Implement MeaterLink transmission

[nathanfaber]

If we can publish MeaterLink format broadcast packets then the standard app will be able to receive these and work normally. This will also allow the real Meater app to publish to the cloud so folks can use a hybrid of the system to publish the data how they want but also use the Meater app/cloud for secondary monitoring.

It is likely that we will need to change the MAC to something different than the real probe so the app won't try to connect directly to the probes and break meaterble.

[nathanfaber]

I've replayed the UDP broadcast packets using tcpreplay and it looks like a pretty simple format/synchronization, the app recognizes the replay. It isn't clear that I will be able to get the app to publish to the cloud for us, it might rely on the sender to do this as well since it needs wifi in order to broadcast in the first place. We also may need to pretend to be a Meater block for discover to work easily.

In theory, we might be able to also publish to the cloud directly but this needs to be investigated.

If we have a full implementation of MeaterLink (both reading and writing) and cloud support, then we could basically have an open source Meater Block and we this would not require adjusting the MACs.

[Austin299792]

Hello @nathanfaber, I'm very interested in doing this with an ESP32. I'm a pretty inexperienced coder, but I'd like to help, if I'm able to. Can I do anything to help?

[andrelind]

Also interested to get this going on an ESP32 (I have a Adafruit Huzzah32) I'll gladly help out also 😁

[Austin299792]

@nathanfaber, would you be interested in publishing your source code for the replay?

[nathanfaber]

Hi,

I haven't written code to generate the MeaterLink format packets. I simply captured the UDP broadcasts using tcpdump from the official app and then replayed them using tcpreplay. This was basically to test out that there is no time stamping or sequence number and it is a half duplex communication (at least for the updates). I haven't decoded the wire format yet either but this can be aided by decompiling the Android app.

I've been a bit busy lately to work on this and I currently have the master code functioning enough to publish all my probes to MQTT. I haven't used the official apps in weeks and have done a number of successful cooks without it.

[Austin299792]

Hello @nathanfaber. I'm able to do tcpdump and save the pcap file, but I'm stuck at trying to tcpreplay it. I'm getting an error when trying to replay it over wlan0.

Unable to process unsupported DLT type: Bluetooth HCI UART transport layer plus pseudo-header

Or when I try to relay it over the bluetooth, I get this. I can see the device with hciconfig (it's up) and hcitool dev

Fatal Error: failed to open device bluetooth0: ioctl: No such device

Not asking you for help troubleshooting, just if you broadcast it over wifi, wired, or bluetooth, and if wifi/wired, did you have to modify the pcap file?

[nathanfaber]

I captured using the wired ethernet interface and injected using wired. Give me a few days and I will try to reproduce it. I did not modify the pcap file but I did need to do some rewrite parameters for tcpreplay (IP src and MAC iirc).

[Austin299792]

@andrelind - I've switched back to just trying to receive and rebroadcast the probe data on my ESP32. Just sounds like such a fun project, and cool to have something so small I can put next to my grill to extend the probe BLE range (like the Meater+ block, I guess).

Anyways, I'm trying to work in Arduino IDE. The ESP32 library has BLE support and examples, and I'm looking at this tutorial. I'll keep you updated on what I learn, or post a link to my git repo, if I can get something working.

[haklein]

@Austin299792 I've created a proof of concept sketch to read out the Meater with an ESP32 and android:

/**
 * A BLE client example that is rich in capabilities.
 * There is a lot new capabilities implemented.
 * author unknown
 * updated by chegewara
 */

#include "BLEDevice.h"
//#include "BLEScan.h"

// The remote service we wish to connect to.
static BLEUUID serviceUUID("a75cc7fc-c956-488f-ac2a-2dbc08b63a04"); 
// The characteristic of the remote service we are interested in.
static BLEUUID    charUUID("7edda774-045e-4bbf-909b-45d1991a2876");

static boolean doConnect = false;
static boolean connected = false;
static boolean doScan = false;
static BLERemoteCharacteristic* pRemoteCharacteristic;
static BLEAdvertisedDevice* myDevice;

static void notifyCallback(
  BLERemoteCharacteristic* pBLERemoteCharacteristic,
  uint8_t* pData,
  size_t length,
  bool isNotify) {
    Serial.print("Notify callback for characteristic ");
    Serial.print(pBLERemoteCharacteristic->getUUID().toString().c_str());
    Serial.print(" of data length ");
    Serial.println(length);
    /*
     * meat_raw, ambient_raw, offset, whatever = struct.unpack('<hhhh', value)
    meat = meat_raw / 16
    # 6,33 ist ein anhand mehrerer Wertepaare üer Excel ermittelter Faktor, k.A. wo dieser herkommt....
    ambient = (meat_raw + max(0, (ambient_raw - offset)) * 6.33) / 16

     */
    int meat_raw = ( (pData[1] << 8) + (pData[0])) ;
    float temp_int = float(meat_raw) / 16;
    int ambient_raw= ( (pData[3] << 8) + (pData[2]));
    int offset = ( (pData[5] << 8) + (pData[4]));
    float ambient = float(meat_raw + max(0, (ambient_raw - offset)) * 6.33) / 16;
    Serial.print("data: ");
    Serial.println((char*)pData);
    Serial.print("temp int: ");
    Serial.println(temp_int);
     Serial.print("ambient: ");
    Serial.println(ambient);
}

class MyClientCallback : public BLEClientCallbacks {
  void onConnect(BLEClient* pclient) {
  }

  void onDisconnect(BLEClient* pclient) {
    connected = false;
    Serial.println("onDisconnect");
  }
};

bool connectToServer() {
    Serial.print("Forming a connection to ");
    Serial.println(myDevice->getAddress().toString().c_str());

    BLEClient*  pClient  = BLEDevice::createClient();
    Serial.println(" - Created client");

    pClient->setClientCallbacks(new MyClientCallback());

    // Connect to the remove BLE Server.
    pClient->connect(myDevice);  // if you pass BLEAdvertisedDevice instead of address, it will be recognized type of peer device address (public or private)
    Serial.println(" - Connected to server");

    // Obtain a reference to the service we are after in the remote BLE server.
    BLERemoteService* pRemoteService = pClient->getService(serviceUUID);
    if (pRemoteService == nullptr) {
      Serial.print("Failed to find our service UUID: ");
      Serial.println(serviceUUID.toString().c_str());
      pClient->disconnect();
      return false;
    }
    Serial.println(" - Found our service");

    // Obtain a reference to the characteristic in the service of the remote BLE server.
    pRemoteCharacteristic = pRemoteService->getCharacteristic(charUUID);
    if (pRemoteCharacteristic == nullptr) {
      Serial.print("Failed to find our characteristic UUID: ");
      Serial.println(charUUID.toString().c_str());
      pClient->disconnect();
      return false;
    }
    Serial.println(" - Found our characteristic");

    // Read the value of the characteristic.
    if(pRemoteCharacteristic->canRead()) {
      std::string value = pRemoteCharacteristic->readValue();
      Serial.print("The characteristic value was: ");
      Serial.println(value.c_str());
    }

    if(pRemoteCharacteristic->canNotify())
      pRemoteCharacteristic->registerForNotify(notifyCallback);

    connected = true;
    return true;
}
/**
 * Scan for BLE servers and find the first one that advertises the service we are looking for.
 */
class MyAdvertisedDeviceCallbacks: public BLEAdvertisedDeviceCallbacks {
 /**
   * Called for each advertising BLE server.
   */
  void onResult(BLEAdvertisedDevice advertisedDevice) {
    Serial.print("BLE Advertised Device found: ");
    Serial.println(advertisedDevice.toString().c_str());

    // We have found a device, let us now see if it contains the service we are looking for.
    if (advertisedDevice.haveServiceUUID() && advertisedDevice.isAdvertisingService(serviceUUID)) {

      BLEDevice::getScan()->stop();
      myDevice = new BLEAdvertisedDevice(advertisedDevice);
      doConnect = true;
      doScan = true;

    } // Found our server
  } // onResult
}; // MyAdvertisedDeviceCallbacks

void setup() {
  Serial.begin(115200);
  Serial.println("Starting Arduino BLE Client application...");
  BLEDevice::init("");

  // Retrieve a Scanner and set the callback we want to use to be informed when we
  // have detected a new device.  Specify that we want active scanning and start the
  // scan to run for 5 seconds.
  BLEScan* pBLEScan = BLEDevice::getScan();
  pBLEScan->setAdvertisedDeviceCallbacks(new MyAdvertisedDeviceCallbacks());
  pBLEScan->setInterval(1349);
  pBLEScan->setWindow(449);
  pBLEScan->setActiveScan(true);
  pBLEScan->start(5, false);
} // End of setup.

// This is the Arduino main loop function.
void loop() {

  // If the flag "doConnect" is true then we have scanned for and found the desired
  // BLE Server with which we wish to connect.  Now we connect to it.  Once we are 
  // connected we set the connected flag to be true.
  if (doConnect == true) {
    if (connectToServer()) {
      Serial.println("We are now connected to the BLE Server.");
    } else {
      Serial.println("We have failed to connect to the server; there is nothin more we will do.");
    }
    doConnect = false;
  }

  // If we are connected to a peer BLE Server, update the characteristic each time we are reached
  // with the current time since boot.
  if (connected) {
    String newValue = "Time since boot: " + String(millis()/1000);
    Serial.println("Setting new characteristic value to \"" + newValue + "\"");

    // Set the characteristic's value to be the array of bytes that is actually a string.
    pRemoteCharacteristic->writeValue(newValue.c_str(), newValue.length());
  }else if(doScan){
    BLEDevice::getScan()->start(0);  // this is just eample to start scan after disconnect, most likely there is better way to do it in arduino
  }

  delay(1000); // Delay a second between loops.
} // End of loop

[Austin299792]

This looks great! Hopefully I’ll be able to dig into and try this soon. Do you have a git repo where you’re keeping this?

[nathanfaber]

I captured using the wired ethernet interface and injected using wired. Give me a few days and I will try to reproduce it. I did not modify the pcap file but I did need to do some rewrite parameters for tcpreplay (IP src and MAC iirc).

This capture works capturing from the MeaterBlock and replaying to the app. It does not work app to app, needs more testing/reverse engineering for that.

0. Set bash variable (used below) iface=eth0 (assuming capturing on eth0).
1. Take one probe off charger so it starts broadcasting to the app
2. Start capture: tcpdump -v -i $iface -n 'port 7878' -s0 -w meaterlink.pcap
3. Turn on your MeaterBlock - you will see the capture above incrementing packets. Capture like 5 before ctrl-c.
4. Rewrite for replay: tcprewrite --srcipmap=0.0.0.0/0:$(ip -4 -o addr show $iface | awk '{print $4}' | cut -d '/' -f1) --enet-smac=$(cat /sys/class/net/$iface/address) --infile=meaterlink.pcap --outfile=meaterlink-rewrite.pcap
5. Put probe back on the charger so it stops broadcasting, the app will gray out the probe.
6. Replay tcpreplay -l10 --intf1 $iface meaterlink-rewrite.pcap
7. Turn off the block. Run the Meater app on your phone. You should see the probes replaying the old data.

[nathanfaber]

Interestingly enough, replaying the MeaterBlock data to the app seems to cause the app to issue requests to cloud.meater.com:8883. I wonder if the Meater App actually publishes everything it sees on MeaterLink. If this was true, it would mean there is likely multiple publishing to the cloud for redundancy (ie: from the block and from the app via MeaterLink) and then it is deduped on the cloud side. This effectively means that if we implement the MeaterLink protocol correctly such that the app recognizes it, then a running app would publish to the cloud and everything might Just Work.

[codahq]

https://www.youtube.com/watch?v=nBaksvm8NDY

[Austin299792]

Hello @haklein. I uploaded the code you provided and it works to receive and print the probe data to the serial console. :) Unfortunately, my c++ skills aren't great and I'm having a hard time broadcasting the raw probe data out of a BLE server I set up on the ESP32 following the BLE_server example in Arduino. Have you tried to do this to see if the MEATER app sees data from the ESP?

[haklein]

@Austin299792 unfortunately I don't own a meater, it was just borrowed from a friend for a few days..

[jcallaghan]

It would be really impressive to get this working on ESP32 with ESPHome (> Home Assistant). That was my intention at least. Watching this thread carefully.

[andrelind]

Does anyone have a UDP recording to share? Haven’t got my meater yet and want to try to get the temp via WiFi in the meantime

[nathanfaber]

Does anyone have a UDP recording to share? Haven’t got my meater yet and want to try to get the temp via WiFi in the meantime

The Meater captures contain the MAC of the Meater probes. This means that once you replay the captures and add them to your app you will have that probe "claimed" and the person that captured will be effectively sharing their probes with you via the cloud and cooks could be modified. This is why I haven't published any. It could be chaotic to publish actual captures with real probe MACs.

I don't want to publish them globally but I'd be willing to give you a capture directly, if that would suffice for your testing.

[andrelind]

Does anyone have a UDP recording to share? Haven’t got my meater yet and want to try to get the temp via WiFi in the meantime

The Meater captures contain the MAC of the Meater probes. This means that once you replay the captures and add them to your app you will have that probe "claimed" and the person that captured will be effectively sharing their probes with you via the cloud and cooks could be modified. This is why I haven't published any. It could be chaotic to publish actual captures with real probe MACs.

I don't want to publish them globally but I'd be willing to give you a capture directly, if that would suffice for your testing.

Thanks, that would be really helpful! 😊 How will we proceed?

[nathanfaber]

Does anyone have a UDP recording to share? Haven’t got my meater yet and want to try to get the temp via WiFi in the meantime

The Meater captures contain the MAC of the Meater probes. This means that once you replay the captures and add them to your app you will have that probe "claimed" and the person that captured will be effectively sharing their probes with you via the cloud and cooks could be modified. This is why I haven't published any. It could be chaotic to publish actual captures with real probe MACs. I don't want to publish them globally but I'd be willing to give you a capture directly, if that would suffice for your testing.

Thanks, that would be really helpful! 😊 How will we proceed?

Shoot me an email at my github username at gmail.

[jaybinks]

MeterLink is using Google Protocol buffers over UDP port 7878.

I've got an almost working ESP32 solution, that connects via BT and communicates on MeterLink protocol to all devices on your WIFI (Meterlink uses Broadcast message to discover other devices on your WIFI)

I need to tidy up the code a bit before I put it public on github, but feel free to contact me with questions.

I've not tried to talk to their cloud service yet, as its probably TLS encrypted and Im not sure I can be bothered with that, however it would be cool none the less.

[andrelind]

MeterLink is using Google Protocol buffers over UDP port 7878.

I've got an almost working ESP32 solution, that connects via BT and communicates on MeterLink protocol to all devices on your WIFI (Meterlink uses Broadcast message to discover other devices on your WIFI)

I need to tidy up the code a bit before I put it public on github, but feel free to contact me with questions.

I've not tried to talk to their cloud service yet, as its probably TLS encrypted and Im not sure I can be bothered with that, however it would be cool none the less.

That’s seriously cool! 🎉 What platform and language are you using?

[jaybinks]

i'm using an ESP32 board, which has bluetooth and wifi built in for about $10.

I started doing this in the Arduino IDE pretty much all in C, but have been doing recent work in PlatformIO, which is almost the same as Arduino ide (same toolchain im sure), but a nicer IDE environment.

I have some prototype bits of it that I wrote in GoLang and PHP but they are nowhere near as developed.

[jcallaghan]

@jaybinks this is great. Are you able to share your work? I have just started to look at building a component for ESPHome.

[andrelind]

@jaybinks did you get anywhere with the protobuf format? Working on reverse engineering the UDP data packets but it seems to just be sending discovery packets as of now 🤔