Open bwood8383 opened 5 years ago
Yep, I just spent some time updating the dependencies and ensuring it does all build on Ubuntu. But please keep in mind that this has been more of a proof of concept / tinker project. So I heavily recommend getting serial access to the device before loading on different firmware.
I'm using:
Steps:
sudo apt install python-pip
pip install -U platformio
export PATH=$PATH:$(python -m site --user-base)/bin
2. Build main firmware
```sh
git clone {repo}
cd {repo}/firmware
pio run
Then you can use PlatformIO to push firmware, or try vesync-hijack if you're feeling really adventureous.
Thanks for the work.
It was actually vesync-hijack that i was referring to in my first post. It appears that you only had it running on a Mac?
Ah, right. vesync-hijack only supports automatic retrieval of the WiFi BSSID on Mac. (I wasn't able to find a decent cross-platform node_module to do this).
However, you should still be able to use the hijacking-server on Linux by manually supplying the BSSID as a command line argument:
node ./index.js -p wifipassword -s yourssid -b yo:ur:_b:ss:id
I believe iwconfig
should give the bssid, but my VM doesn't have a Wireless adapter to test it with...
I'll give it a try.
Thanks again for all of the extra work.
Did you using python 3.6 with your Ubuntu install?
I've been using the system default 2.7 (both Ubuntu + Mac).
commit 96bc95178ce899e8b51d45b70c630d7947cdc51e fixes up the scripts for Python 3.6.8.
These are the two bin files I get when I compile the bootstrap firmware. -rw-rw-r-- 1 bwood bwood 34656 Sep 17 15:18 firmware.bin -rw-rw-r-- 1 bwood bwood 256239 Sep 17 15:18 firmware.bin.irom0text.bin
I'm guessing i can just rename them to user1 and user2 if the file sizes look ok to you and you can tell me which is which.
Hmmmm that's not right... Looks like those additional targets were being silently ignored, so c3c905ffe43daa91044e100338dc385e65c2c61c adds user1 & user2 as explicit targets.
You should now get:
$ ls -al .pio/build/app/
-rwxr-xr-x 1 adapt adapt 462886 Sep 17 20:51 firmware.elf
-rw-r--r-- 1 adapt adapt 290916 Sep 17 21:50 user1.bin
-rwxr-xr-x 1 adapt adapt 462886 Sep 17 21:50 user1.elf
-rw-r--r-- 1 adapt adapt 290916 Sep 17 21:50 user2.bin
-rwxr-xr-x 1 adapt adapt 462886 Sep 17 21:50 user2.elf
I'm unsure of how reproducible the builds are, but here's some SHA1 sums:
$ git rev-parse --short HEAD
c3c905f
$ shasum .pio/build/app/*
adbf092dc1b8a81e4347f1857d15bc6dbc503635 .pio/build/app/firmware.elf
231f356ccb3590e6ffc1a606ddcefbfed3e4557d .pio/build/app/user1.bin
d10c5ae7618c26e0e5f4f7d4e06544b1f538d6a8 .pio/build/app/user1.elf
4f88b2b2615ca39d0e3fe79b56c729203c16dcd6 .pio/build/app/user2.bin
eddb36f979983e86c8668e9cc07472d1fb49fd48 .pio/build/app/user2.elf
Reminder: grab the user1.bin
& user2.bin
from here, firmware.bin
from the main firmware build, and place them in the hijacking-server/assets folder.
I also suggest grabbing an image of the existing firmware before doing a bootstrap. I found esptool.py the easiest way to do this.
e.g., ./esptool.py --port /dev/tty.usbserial read_flash 0 0x400000 etekcity.bin
Greatly appreciate your persistence with this 😄
Got everything to compile.
Will report back with results of the hijack.
I'm a little confused about the web server: node ./index.js -p wifipassword -s yourssid -b yo:ur:_b:ss:id
"yourssid" - is this my existing home network ssid that i would like the device to eventually end up on once it is all configured?
Do I need to have the nic for the Ubuntu ws that i am running this node web server on connected to the same network that "yourssid" is on?
My current configuration is that the workstation is on a different vlan from the ssid I am using for "yourssid".
Yep, SSID/BSSID is the wireless network that the device needs to join so we can communicate with it. Everything needs to be on the same broadcast domain. It's not really going to be able to jump across VLANs. So yes, nic with the web server needs to be on the same network as "yourssid".
Typically I've been creating an adhoc wireless bridge (Internet sharing) so I can more easily monitor the traffic for this process. But any flat compatible Wireless network should work.
hijacking-server sends out periodic multicast packets with varying sizes per the Smart Config protocol. These packets need to end up being sent out over WiFi so that the Etekcity device can see them. Device will then use that configuration information to join the wireless network and contact our web server. Our web server will reply back to get the device to perform a couple of firmware upgrades. Final flow summary. (Which is why we need the same broadcast domain)
Note that the hijacking-server does need to know your machine's IP to be able to advertise it. It should figure out what the primary interface is, but if that doesn't work out there's an additional --ip
setting to force it.
Ok, I think I understand how things are supposed to work but so far Etekcity devices don't seem to be communicating with my hijack server. I'll describe what I have going with the hope you can see where I have something wrong.
I am using a Ubuntu 18.04.3 LTS machine with both a wired and a wireless nic. I have disabled the wired nic and configured the wireless nic to connect to my IOT SSID. I am running the hijack web server using the same IOT SSID and password along with BSSID that I obtained by running iwconfig.
The hijack server seems to be starting and working correctly: node ./index.js -p iotpass -s iotssid -b "mac addr from iwconfig Access Point:" Using wlp4s0 192.168.111.82 Using SSID "iotssid" (BSSID: "mac address", Local IP: 192.168.111.82) Listening for devices on UDP port 18266 Starting web server on TCP port 17273 Sending Smart Config...
These are the ports that are listening: sudo netstat -tulpn | grep node tcp6 0 0 :::17273 ::: LISTEN 2664/node udp 0 0 192.168.111.82:18266 0.0.0.0: 2664/node
I plug in the Etekcity device and hold down the button until i get a solid blue light which indicates the device is in "Smart Config". i can see the mac address of the device on my network but it doesn't ever get an ip address.
Are there any logs I can be checking to look for errors? I don't see any errors in /var/log/syslog.
That all sounds good. Although if you're seeing the MAC address appear on your network, it should also then be grabbing a DHCP lease. Assuming you have a DHCP server on your IOT network?
Maybe first try placing the device in Smart Config mode (with no hijack server). Confirm that the device is no longer on the IOT network, and then running the hijack server to verify that it at least gets the device to join your network. Which should help a little bit with the troubleshooting.
Unfortunately if it's not connecting, the only real logs to look at are on the Etekcity device itself...
It's also been a while since I've updated the stock firmware on my test device. So I'll take a look to see if the connection behavior has been changed.
Does it matter that the web server on the tcp port is ip6 instead of ip4? tcp6 0 0 :::17273 :::* LISTEN 2664/node
I believe that means that it is listening for both ipv4 + ipv6 connections.
--
Managed to upgrade my plug to the latest firmware 2.123 and the onboarding connection process has changed. Rather than the smart config; device is set to AP mode, a TCP connection provides the initial config, and the device uses that to connect to the provided AP + server.
Problem now is that the websocket data between device + server has switched to using binary frames. These frames look to be encrypted with some kind of constant block cipher. So still looking into that...
Thanks for all of your help Chris.
I'll let you know if I stumble onto anything new.
Alright, I've been very slowly nibbling away at this. Good news is that I figured out the binary WebSocket protocol, along with the needed changes to the hijack server. This includes changes to the bootstrap firmware to make it act more Etekcity-like. Along with updates to the main firmware to improve the provisioning process.
assets/
before trying againpio lib update
Bad news is that this does make the hijack process quite a bit more involved. Easiest way is to have a dedicated AP connected to one network interface, then have another wireless network interface for connecting to the device directly. Flow then becomes; connect to device's AP, send packets which cause the device to connect to your AP, device then makes network requests to your PC.
I have my Mac connected to an AP via Ethernet, then use the Wireless interface to connect to the device.
Process being:
Put the Etekcity device into AP mode by holding down the button for 5 seconds or so
Connect to the ESP_XXXX AP that shows up
Start hijack-server
Specifying the local AP's SSID, password, your Ethernet's IP (connected to local AP), and the device's Wireless IP (should always be 192.168.4.1)
node ./index.js --ssid ${AP_SSID} -p ${AP_PASSWORD} --ip ${LOCAL_INTERFACE} --device 192.168.4.1
Should see something like:
Connected to device 192.168.4.1
{
url: '/connectInfo',
...
firmVersion: '2.123',
...
}
Accepted WebSocket from ::ffff:192.168.7.240
{
account: '0',
...
}
Initiating device upgrade
::ffff:192.168.7.240 - - [03/Nov/2019:17:31:13 +0000] "HEAD /user1.bin HTTP/1.1"
::ffff:192.168.7.240 - - [03/Nov/2019:17:31:13 +0000] "HEAD /user1.bin HTTP/1.1" 200 332548 "-" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36 "
{ uri: '/restore', param: 'schedule' }
::ffff:192.168.7.240 - - [03/Nov/2019:17:31:18 +0000] "GET /user1.bin HTTP/1.1"
::ffff:192.168.7.240 - - [03/Nov/2019:17:31:19 +0000] "GET /user1.bin HTTP/1.1" 200 332548 "-" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36 "
{ uri: '/upgrade', newVersion: '2.00', error: 0 }
{
account: '0',
...
}
We connected to the device, told it to connect to our AP. Device connects back to our server, which then instructs it to initiate an upgrade.
Here we see it grabbed /user1.bin
as the current firmware is running from the second partition.
Control+C hijack-server
Re-connect to ESP_XXXX AP which should show up. Our bootstrap firmware automatically starts in AP mode on boot.
Start hijack-server back up
node ./index.js --ssid ${AP_SSID} -p ${AP_PASSWORD} --ip ${LOCAL_INTERFACE} --device 192.168.4.1
Now we should see slightly different output from the bootstrap firmware:
Starting web server on TCP port 17273
Attempting to connect to device 192.168.4.1
Connected to device 192.168.4.1
::ffff:192.168.7.240 - - [03/Nov/2019:17:31:53 +0000] "GET /firmware.bin HTTP/1.0"
::ffff:192.168.7.240 - - [03/Nov/2019:17:31:59 +0000] "GET /firmware.bin HTTP/1.0" 200 742160 "-" "-"
Here it retrieved /firmware.bin
which is the final payload.
Control+C hijack-server
Repeat previous two steps if the device requested /user1.bin
ESP SDK uses two partitions for upgrades, while the Arduino SDK treats the flash more monolithic. Meaning that we need the firmware to be uploaded to the second partition. Bootstrap firmware is aware of this, so will request the appropriate binary depending on which partition it is executing from.
Re-connect to ESP_XXXX AP which should show up
Use a web browser to connect to 192.168.4.1
For some reason WebSockets can take a little while before they start working? Not entirely sure what is going on there. Seems to happen more often in AP mode. Anyway, if the page loads, but says disconnected, give it 20 secs.
Go to Settings -> Network and enter your network details
If you need the device to go back into AP mode; hold down the button for five seconds.
Wow, thanks Chris.
I'll set it up and give it a go.
Hi Chris, Sorry it has taken me a while but i was having issues hunting down an ap.
I think that i have everything setup correctly but this is the output from my latest attempt. Using SSID "-s" (BSSID: 00:00:00:00:00:00, Local IP: 192.168.15.2) Starting web server on TCP port 17273 Attempting to connect to device 192.168.4.1 Connected to device 192.168.4.1 undefined
This is the command that i am using to run the node server. node vesync-hijack/hijacking-server/index.js -ssid "ssid for ap" -p "ssid pass" --ip 192.168.15.2 --device 192.168.4.1
Apologies, been busy traveling for work.
Are you able to confirm which firmware version is loaded on your device? I've been testing with 2.123, so first want to check that we're using the same thing.
Hmmm if we're seeing "undefined"; there was a response from the device, but there must have been an issue decoding it. I just pushed a change to hijacking-server to hexdump out the received response so we can verify the format. If that doesn't look like the below, I'll probably need to see a tcpdump/Wireshark capture of that TCP stream (Follow TCP stream). You can use a dummy ssid/pw for that session, as I just need to see the request/reply, content is less important.
Starting web server on TCP port 17273
Attempting to connect to device 192.168.4.1
Connected to device 192.168.4.1
0000 9f7b2275726c223a222f636f6e6e656374496e666f222c2273746570223a2253
0020 54455033222c226465736372697074696f6e223a224445565f52454345495645
0040 5f4150505f44415441222c226669726d56657273696f6e223a22322e31323322
0060 2c22636964223a2266613136663335652d313039362d343938662d396566342d
0080 643133306632336635333536222c22667265655f68656170223a33303631367d
00a0 00
{
url: '/connectInfo',
step: 'STEP3',
description: 'DEV_RECEIVE_APP_DATA',
firmVersion: '2.123',
cid: 'fa16f35e-1096-498f-9ef4-d130f23f5356',
free_heap: 30616
}
Thanks for the reply Chris.
I'll pull the change and run it again.
On Sat, Dec 7, 2019 at 11:30 AM Chris Byrne notifications@github.com wrote:
Apologies, been busy traveling for work.
Are you able to confirm which firmware version is loaded on your device? I've been testing with 2.123, so first want to check that we're using the same thing.
Hmmm if we're seeing "undefined"; there was a response from the device, but there must have been an issue decoding it. I just pushed a change to hijacking-server to hexdump out the received response so we can verify the format. If that doesn't look like the below, I'll probably need to see a tcpdump/Wireshark capture of that TCP stream (Follow TCP stream). You can use a dummy ssid/pw for that session, as I just need to see the request/reply, content is less important.
Starting web server on TCP port 17273 Attempting to connect to device 192.168.4.1 Connected to device 192.168.4.1 0000 9f7b2275726c223a222f636f6e6e656374496e666f222c2273746570223a2253 0020 54455033222c226465736372697074696f6e223a224445565f52454345495645 0040 5f4150505f44415441222c226669726d56657273696f6e223a22322e31323322 0060 2c22636964223a2266613136663335652d313039362d343938662d396566342d 0080 643133306632336635333536222c22667265655f68656170223a33303631367d 00a0 00 { url: '/connectInfo', step: 'STEP3', description: 'DEV_RECEIVE_APP_DATA', firmVersion: '2.123', cid: 'fa16f35e-1096-498f-9ef4-d130f23f5356', free_heap: 30616 }
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/adapt0/smartplug/issues/4?email_source=notifications&email_token=ANFIR727Y2MAAXNE26RRJZTQXP2UHA5CNFSM4IV2POO2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGGNYKY#issuecomment-562879531, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANFIR74AQ6RZRTPNEOUEGIDQXP2UHANCNFSM4IV2POOQ .
Still getting "undefined".
Maybe i should configure the device using vesync to ensure i have the current firmware?
Is there an easy way to tell what firmware version it is running?
Easiest way is with a serial connection, looking at the log when the device boots up:
Firm name:cosytek_firm_a
Firm version:2.123
Firm version code:123
Flash-Size-Map: FLASH_SIZE_32M_MAP_512_512
Alternatively with the VeSync app (the newer one):
The "new" version of these plugs that is currently being sold unfortunately does not react in the same way:
Starting web server on TCP port 17273
Attempting to connect to device 192.168.4.1
Connected to device 192.168.4.1
0000 437b22757269223a222f636f6e666967496e666f222c22636f6e6669674d6f64
0020 756c65223a223130414f75746c65745553222c2269735365637572697479223a
0040 2231227d
0000 498e3c762468c689c8e9793e1b040ad80385276ae4aa61eac22c3f98211fe3f3
0020 4fa4ab08199533880d0bc0d40a32f77f24234376048b0df0c81b237176ab26b2
0040 e844a15173a397d702bbed082f8e55624731e581028053704f9065c312dbd94f
0060 052185ad584a62bebd4680f5fa718fae946769685675742959dc84e6a79fa111
0080 a2d12d4df0cb2afdae67164b550e6aafc0fe4309ca41478e32713afcf7450c58
00a0 5e44349e1c4fb2821a50faf63b8202bb
{ uri: '/configInfo', configModule: '10AOutletUS', isSecurity: '1' }
The serial console displays the following meanwhile:
ESP8266 SDK version : 2.2.2-dev(2a5f9d5)
VeSync SDK version : 2.4.7
Flash-Size-Map: FLASH_SIZE_8M_MAP_512_512
User run area : user1
Device MAC : 60:01:94:e8:d7:e7
Device channel : 1
sleep disable
sleep type: 0.
Device type : 10AOutletUS
Firmware version : 1.1.27
System started ...
mode : sta(60:01:94:e8:d7:e7)
add if0
[E]<Vesync>Delete tcp error : -12 !
scandone
mode : sta(60:01:94:e8:d7:e7) + softAP(62:01:94:e8:d7:e7)
add if1
dhcp server start:(ip:192.168.4.1,mask:255.255.255.0,gw:192.168.4.1)
bcn 100
bcn 0
del if1
add if1
dhcp server start:(ip:192.168.4.1,mask:255.255.255.0,gw:192.168.4.1)
bcn 100
add 1
aid 1
station: a4:83:e7:64:b3:d0 join, AID = 1
[E]<Vesync>Parse cjson error !
Judging from the tcp stream that follows when doing a device setup (from the ios app), what follows is obfuscated or encrypted.
I tried the same firmwares with the "old" plugs (running 2.123 stock firmware). The upgrade is initiated, but the plug replies with an error:
Using SSID "DEAD" (BSSID: 00:00:00:00:00:00, Local IP: 192.168.1.238)
Starting web server on TCP port 17273
Attempting to connect to device 192.168.4.1
Connected to device 192.168.4.1
0000 9f7b2275726c223a222f636f6e6e656374496e666f222c2273746570223a2253
0020 54455033222c226465736372697074696f6e223a224445565f52454345495645
0040 5f4150505f44415441222c226669726d56657273696f6e223a22322e31323322
0060 2c22636964223a2235643135323766312d343261342d343763322d626164322d
0080 343238623136633164336263222c22667265655f68656170223a32393732387d
00a0 00
{
url: '/connectInfo',
step: 'STEP3',
description: 'DEV_RECEIVE_APP_DATA',
firmVersion: '2.123',
cid: '5d1527f1-42a4-47c2-bad2-428b16c1d3bc',
free_heap: 29728
}
Accepted WebSocket from ::ffff:192.168.1.243
{
account: '0',
id: '5d1527f1-42a4-47c2-bad2-428b16c1d3bc',
deviceName: 'vesync_wifi_outlet',
deviceVersion: '1.5',
deviceVersionCode: 5,
type: 'wifi-switch',
apptype: 'switch-measure',
firmName: 'cosytek_firm_a',
firmVersion: '2.123',
firmVersionCode: 123,
key: 2147483647,
relay: 'break',
rssi: -35,
mac: '5E:CF:7F:62:DB:E6',
OTP: 'off',
initState: 'ConfigNet'
}
Initiating device upgrade
::ffff:192.168.1.243 - - [23/Jan/2020:21:59:04 +0000] "HEAD /user2.bin HTTP/1.1"
::ffff:192.168.1.243 - - [23/Jan/2020:21:59:04 +0000] "HEAD /user2.bin HTTP/1.1" 200 332548 "-" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36 "
{ uri: '/restore', param: 'schedule' }
::ffff:192.168.1.243 - - [23/Jan/2020:21:59:06 +0000] "GET /user2.bin HTTP/1.1"
::ffff:192.168.1.243 - - [23/Jan/2020:21:59:07 +0000] "GET /user2.bin HTTP/1.1" 200 332548 "-" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36 "
{ uri: '/upgrade', newVersion: '2.00', error: 1 }
{
account: '0',
id: '5d1527f1-42a4-47c2-bad2-428b16c3e3bc',
deviceName: 'vesync_wifi_outlet',
deviceVersion: '1.5',
deviceVersionCode: 5,
type: 'wifi-switch',
apptype: 'switch-measure',
firmName: 'cosytek_firm_a',
firmVersion: '2.123',
firmVersionCode: 123,
key: 0,
relay: 'break',
rssi: -37,
mac: '5C:CF:7F:62:CE:E6',
OTP: 'off',
initState: 'Upgrade'
}
What happens on the plug meanwhile, output captured from the serial port:
[01-30 12:31:23] uri value:/upgrade
execute_upgrade_command
upgrade url:http://192.168.7.238:17273
upgrade newVersion:2.126
=======cs_firm_upgrade_begin!==========
download firm url: GET /user1.bin HTTP/1.1
Host: 192.168.7.238:17273
Connection: keep-alive
Cache-Control: no-cache
User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36
Accept: */*
Accept-Encoding: gzip,deflate,sdch
Accept-Language: zh-CN,zh;q=0.8
system_upgrade_start
upgrade_connect 29568
upgrade_connect_cb
pusrdata = HTTP/1.1 200 OK
Content-Length: 332548
Date: Mon, 27 Jan 2020 20:24:01 GMT
Connection: close
sumlength = 332548
sec_block 82
..................................................................................upgrade_get_sum_disconcb 27376
erase sector=1 ok
erase sector=2 ok
erase sector=3 ok
erase sector=4 ok
restore schedule: {"uri":"/restore","param":"schedule"}
buf length = 37
erase sector=5 ok
cs_switch_client_sent_cb
erase sector=6 ok
erase sector=7 ok
[..]
erase sector=40 ok
erase sector=41 ok
wifi_handle_event_cb 7
erase sector=42 ok
erase sector=43 ok
[..]
erase sector=77 ok
erase sector=78 ok
smartconfig restart tick stop.
==WIFI set STATION MODE
station: a4:83:e7:64:b3:d0 leave, AID = 1
rm 1
bcn 0
del if1
pm open,type:2 0
mode : sta(2c:3a:e8:48:3d:a9)
[01-30 12:31:26] wifi_handle_event_cb 6
APP TCP station: a4:83:e7:64:b3:d0 leave, AID = 1
wifi_handle_event_cb 8
erase sector=79 ok
erase sector=80 ok
erase sector=81 ok
erase sector=82 ok
ALL=82 sectors erase ok!
upgrade_connect_cb
sumlength = 332548
upgrade file download start.
totallen = 2816
totallen = 4276
[..]
totallen = 331316
totallen = 332548
upgrade file download finished.
flash_crc = 3818304285
img_crc = 476663011
upgrade_check
cs_firm_upgrade failed
upgrade reply:49, {"uri":"/upgrade","newVersion":"2.126","error":1}
buf length = 49
cs_switch_on_upgrade_completed, result:1
Looking at the final output lines flash_crc = 3818304285 img_crc = 476663011 upgrade_check cs_firm_upgrade failed
it looks like the firmware does a checksum that fails.
Hi Chris, I would like to get this running on an Ubuntu workstation and use it to load different firmware.
Could this be easily done?