General guidance on how to "get started" with the stack

Ocramius commented 3 years ago

Hey there! First of all, kudos for trying to emulate the whole Pet Hub.

Background

I'm trying to understand a couple details about this repository, mostly because the README.md seems to include multiple diverging approaches to how to get these devices to publish to MQTT (I understand that this is the result of a lot of try-and-fail).

I'm currently sitting here like a complete moron with a ConbeeII in my hands, and realizing that it's pretty much useless :D

Questions

[x] does anything in this repository provide some way to capture direct ZigBee / MiWi wireless signals from the feeder devices, in order to publish to a LAN MQTT broker?
[x] is the only way to capture MiWi signals (according to your findings) to either use a shield for arduino/ESP8622/RaspberryPI? Is there such a thing as a USB stick that understands MiWi? AFAIK it's very much a walled garden of proprietary stuff: https://en.wikipedia.org/wiki/MiWi
[x] is this repository mostly trying to use the original SurePetCare Hub, and to MITM outgoing traffic via DNS poisoning?
[x] assuming the SurePetCare Hub traffic is being hijacked, I suppose the Android/iOS apps for it will also become useless, right?

Thank you in advance for any guidance!

Breadcrumbs

Meanwhile, for others that are as lost as I am about avoiding the SurePetCare Hub, here are some references about stuff that brought me here:

https://github.com/alextoft/sureflap/issues/8 - people are trying to intercept traffic coming from the Zigbee/802.15.4 antenna within these SurePetCare devices, but with little success due to the Microchip MiWi protocol being typical "internet of shit" stuff
on https://github.com/Koenkk/zigbee2mqtt/issues/3261, findings seem to state that these devices do not use Zigbee, but rather MiWi
https://github.com/mretallack/catfeeder is a complete RaspberryPI + https://www.mikroe.com/pi-3-click-shield + https://www.mikroe.com/bee-click that uses the raw antenna signal plus MiWi libraries to capture network traffic and publish it to MQTT - the reason why I'm not jumping at it is that I'm trying to find a way to make all of this work over a USB stick (if even possible)

plambrechtsen commented 3 years ago

This project is 98% the docker compose stack that yes does DNS poisoning of the first API call the hub makes to download the official credentials from surepet then rewrite them to have an easier MQTT topic name then point to a MQTT instance that is also part of the compose that listens on 8883 and relays messages from that MQTT instance to your main Home Assistant instance where the last component pethub listens to the messages that the hub creates on your home assistant MQTT instance from the hub and home assistant and translates between the two. Last week I also figured out how to find the password for the certificate which is in the credentials file by connecting a serial console and performing a firmware update. Then home assistant replaces most of the functionality of the app and I am working on a cli to do the more infrequent commands but the messages are documented in pethubpacket.py

I thought the readme in the docker folder but will update the main readme as well to reflect that.

There is also zigsniff which is a stack to sniff the Zigbee traffic and decrypt it and the wemos hub but that was before I figured out how to find the certificate password and once I did that it was not worth putting any effort in as I could monitor all messages in both directions using PolarProxy.

Does that help and now I know the areas in my readme I need to update as I know documentation needs some work.

plambrechtsen commented 3 years ago

If you wanted to work on your own hub solution then Marks app works on a Pi or similar using the MRF radio as does my wemos code that works with the same radio but on a wemos/arduino rather than a Pi but I didn't finish that as I got the hub working and the messages the devices send is the same anyway so even if we did build a new hub you would still need pethubpacket to translate the messages and I didn't fully get to the bottom of the crc for the Zigbee/802.15.4 frame. Technically it's a 802.15.4 frame that has the data frame being XORed with a static key, and the beacon frames aren't encrypted.

Ocramius commented 3 years ago

Hey @plambrechtsen,

Thanks for the super fast response!

I think your answer pretty much covers what I needed to know 👍

Indeed, all what was kinda needed is to put some more emphasis on the first section (describing what you put in docker/), while describing that the rest is dev tooling.

I will probably try the MITM approach only after trying using the antennas directly: the CRC part seems to have been figured out too. Having a piece of equipment that cannot ever receive firmware updates (because locked out of WAN via DNS poisoning) seems a but scary long-term 😬

On Fri, May 7, 2021, 20:17 Peter Lambrechtsen @.***> wrote:

If you wanted to work on your own hub solution then Marks app works on a Pi or similar using the MRF radio as does my wemos code that works with the same radio but on a wemos/arduino rather than a Pi but I didn't finish that as I got the hub working and the messages the devices send is the same anyway so even if we did build a new hub you would still need pethubpacket to translate the messages and I didn't fully get to the bottom of the crc for the Zigbee/802.15.4 frame. Technically it's a 802.15.4 frame that has the data frame being XORed with a static key, and the beacon frames aren't encrypted.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/plambrechtsen/pethublocal/issues/4#issuecomment-834668916, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABFVED2WDDL37TOQJPOTY3TMQVCLANCNFSM44KK6GVA .

plambrechtsen commented 3 years ago

If you have a hub and setup the docker stack correctly it also downloads the current firmware for your hub by default so you can reapply it as many times as you want. I do recommend getting the certificate password via the console. And in /docs/Hub has photos of the internal hub and the pins to solder to if you have a TTL serial adapter. But I also mention if you brick your hub it's your own fault. :) but I know of a few folks who have done it without issue. In theory the CC2531 could be used as the transmitter but I couldn't find any easily reusable firmware for it that wasn't Zigbee so either I had to write my own custom firmware for the CC2531 and then driver software, or use the MRF24 which is SPI and has a good specification sheet available. Plus I liked the idea of a wifi enabled hub using the ESP32 so then I could have it plugged in close to the device, but haven't focused on that since I have the hub fully MITMed. Patches or PRs are always welcome. :)

Lenbok commented 3 years ago

Hopping in here as this counts as "getting started". I've just bought a cat flap connect and hub and plan to use it with this for a no-cloud solution. Thanks for your work on this @plambrechtsen !

It sounds like I need to initially set up an account on the cloud surepet API before proceeding with the mkpetlocaldb.py. Making that account immediately asks me to do things like add a hub, and set up pets and profiles - does that all need to be done too before mkpetlocaldb.py? Do I also need the docker compose running at that point in time?

plambrechtsen commented 3 years ago

Hopping in here as this counts as "getting started". I've just bought a cat flap connect and hub and plan to use it with this for a no-cloud solution. Thanks for your work on this @plambrechtsen !

It sounds like I need to initially set up an account on the cloud surepet API before proceeding with the mkpetlocaldb.py. Making that account immediately asks me to do things like add a hub, and set up pets and profiles - does that all need to be done too before mkpetlocaldb.py? Do I also need the docker compose running at that point in time?

"Technically" you don't need to connect to the cloud it is just it makes life much much easier to get yourself going if you use their cloud service. Depending on the hardware revision of the hub (ie a H010-xx or one of the older ones H008 or lower) then if you connect it first up to the docker stack you get the long_password which is the password for the SSL Certificate. And if it is a H008 or older then it needs to be firmware updated first, which is just easier to do using the cloud. If it is a H010 then you don't get the long_serial and it should just work to the docker stack.

The devices first need to be adopted to the hub, which you can use the command line tool: https://github.com/plambrechtsen/pethublocal/blob/main/docker/source/cli.py with something like python3 cli.py hub adoptenable Which generates one of these sorts of messages https://github.com/plambrechtsen/pethublocal/blob/main/docker/source/pethubpacket.py#L821 Then you will need to create the pethublocal.db sqlite3 database using this sql file. https://github.com/plambrechtsen/pethublocal/blob/main/docker/source/pethublocal.sql And then it should work, or might need a bit of tweaking depending on your devices.

Otherwise if you sign up to the cloud service, ideally using a desktop browser to https://surepetcare.io since then you can see all the APIs being called then the mkpetlocaldb.py will create the pethublocal.db using your current config.

If it is a older Hub (ie H001-H009) then I would be VERY interested to see what the initial boot is to the docker stack. Feel free to email me offline if you wanted some more help.

plambrechtsen commented 3 years ago

I also made major changes to the main readme @Ocramius so hopefully that helps you too.

plambrechtsen commented 3 years ago

I think everything has been covered, so closing this.

plambrechtsen / pethublocal