search

garbled1 / homeassistant_ecowitt

Ecowitt Weather Station integration for homeassistant
Apache License 2.0
143 stars 70 forks source link

Compatability with Bresser weather station #88

Open ei-ke opened 2 years ago

ei-ke commented 2 years ago

First of all thank you for creating this plugin. Currently I'm trying to set it up with this Bresser weather station. The weather station seems to be manufactured by Fine Offset Electronics like the Ecowitt stations. The only difference is that I'm currently having issues to use WS view but I can use the stations web interface. It does offer custom server addresses to send data to but no port option. I'm not sure if "IP:PORT" works and thus I'm doing a port translation on my router from 80 to 4199. On the HA server I can see the traffic coming in with tshark:

root@gaia:~# tshark -i 1 -f "tcp port 4199"
Running as user "root" and group "root". This could be dangerous.
Capturing on 'ens18'
    1 0.000000000 192.168.4.10 → 192.168.2.225 TCP 58 49559 → 4199 [SYN] Seq=0 Win=5840 Len=0 MSS=1460
    2 0.000034047 192.168.2.225 → 192.168.4.10 TCP 58 4199 → 49559 [SYN, ACK] Seq=0 Ack=1 Win=64240 Len=0 MSS=1460
    3 0.001048919 192.168.4.10 → 192.168.2.225 TCP 54 49559 → 4199 [ACK] Seq=1 Ack=1 Win=5840 Len=0
    4 0.174793451 192.168.4.10 → 192.168.2.225 HTTP 402 GET /weatherstation/updateweatherstation.php?ID=&PASSWORD=&action=updateraww&realtime=1&rtfreq=5&dateutc=now&baromin=29.91&tempf=73.0&dewptf=51.6&humidity=47&windspeedmph=0.0&windgustmph=0.0&winddir=219&rainin=0.0&dailyrainin=0.01&solarradiation=0.0&UV=0.0&indoortempf=72.5&indoorhumidity=44 HTTP/1.1 
    5 0.174821133 192.168.2.225 → 192.168.4.10 TCP 54 4199 → 49559 [ACK] Seq=1 Ack=349 Win=63892 Len=0
    6 0.175260360 192.168.2.225 → 192.168.4.10 TCP 204 HTTP/1.1 200 OK  [TCP segment of a reassembled PDU]
    7 0.175336522 192.168.2.225 → 192.168.4.10 HTTP 56 HTTP/1.1 200 OK  (text/plain)
    8 0.176624373 192.168.4.10 → 192.168.2.225 TCP 54 49559 → 4199 [ACK] Seq=349 Ack=153 Win=5688 Len=0
    9 0.405282855 192.168.4.10 → 192.168.2.225 TCP 54 49559 → 4199 [FIN, ACK] Seq=349 Ack=153 Win=5688 Len=0
   10 0.405514444 192.168.2.225 → 192.168.4.10 TCP 54 4199 → 49559 [FIN, ACK] Seq=153 Ack=350 Win=63892 Len=0
   11 0.406758798 192.168.4.10 → 192.168.2.225 TCP 54 49559 → 4199 [ACK] Seq=350 Ack=154 Win=5687 Len=0
   12 12.062899882 192.168.4.10 → 192.168.2.225 TCP 58 49561 → 4199 [SYN] Seq=0 Win=5840 Len=0 MSS=1460
   13 12.062924736 192.168.2.225 → 192.168.4.10 TCP 58 4199 → 49561 [SYN, ACK] Seq=0 Ack=1 Win=64240 Len=0 MSS=1460
   14 12.064167905 192.168.4.10 → 192.168.2.225 TCP 54 49561 → 4199 [ACK] Seq=1 Ack=1 Win=5840 Len=0
   15 12.237803498 192.168.4.10 → 192.168.2.225 HTTP 402 GET /weatherstation/updateweatherstation.php?ID=&PASSWORD=&action=updateraww&realtime=1&rtfreq=5&dateutc=now&baromin=29.91&tempf=73.0&dewptf=51.9&humidity=48&windspeedmph=0.0&windgustmph=0.0&winddir=219&rainin=0.0&dailyrainin=0.01&solarradiation=0.0&UV=0.0&indoortempf=72.5&indoorhumidity=44 HTTP/1.1 
   16 12.237830320 192.168.2.225 → 192.168.4.10 TCP 54 4199 → 49561 [ACK] Seq=1 Ack=349 Win=63892 Len=0
   17 12.238285460 192.168.2.225 → 192.168.4.10 TCP 204 HTTP/1.1 200 OK  [TCP segment of a reassembled PDU]
   18 12.238329253 192.168.2.225 → 192.168.4.10 HTTP 56 HTTP/1.1 200 OK  (text/plain)
   19 12.239571448 192.168.4.10 → 192.168.2.225 TCP 54 49561 → 4199 [ACK] Seq=349 Ack=153 Win=5688 Len=0
   20 12.469111674 192.168.4.10 → 192.168.2.225 TCP 54 49561 → 4199 [FIN, ACK] Seq=349 Ack=153 Win=5688 Len=0
   21 12.469331287 192.168.2.225 → 192.168.4.10 TCP 54 4199 → 49561 [FIN, ACK] Seq=153 Ack=350 Win=63892 Len=0
   22 12.470518289 192.168.4.10 → 192.168.2.225 TCP 54 49561 → 4199 [ACK] Seq=350 Ack=154 Win=5687 Len=0
22 packets captured

But the plugin is not creating any kind of sensors. I've also set the default logging of HA to verbose but grepping for ecowitt doesn't show any useful information to me. The only result is:

2021-12-13 20:26:39 DEBUG (MainThread) [homeassistant.components.websocket_api.http.connection] [139973481939056] Sending {"id": 27, "type": "result", "success": true, "result": [{"domain": "mobile_app", "name": "Mobile App", "config_flow": true, "documentation": "https://www.home-assistant.io/integrations/mobile_app", "requirements": ["PyNaCl==1.4.0", "emoji==1.5.0"], "dependencies": ["http", "webhook", "person", "tag", "websocket_api"], "after_dependencies": ["cloud", "camera", "notify"], "codeowners": ["@robbiet480"], "quality_scale": "internal", "iot_class": "local_push", "is_built_in": true}, {"domain": "network", "name": "Network Configuration", "documentation": "https://www.home-assistant.io/integrations/network", "requirements": ["ifaddr==0.1.7"], "codeowners": [], "dependencies": ["websocket_api"], "quality_scale": "internal", "iot_class": "local_push", "is_built_in": true}, {"domain": "lovelace", "name": "Lovelace", "documentation": "https://www.home-assistant.io/integrations/lovelace", "codeowners": ["@home-assistant/frontend"], "is_built_in": true}, {"domain": "ecowitt", "name": "Ecowitt Weather Station", "version": "0.7", "config_flow": true, "documentation": "https://www.home-assistant.io/integrations/ecowitt", "requirements": ["pyecowitt==0.14"], "ssdp": [], "zeroconf": [], "homekit": {}, "dependencies": [], "codeowners": ["@garbled1"], "is_built_in": false}, {"domain": "persistent_notification", "name": "Persistent Notification", "documentation": "https://www.home-assistant.io/integrations/persistent_notification", "codeowners": ["@home-assistant/core"], "quality_scale": "internal", "iot_class": "local_push", "is_built_in": true}, {"domain": "zeroconf", "name": "Zero-configuration networking (zeroconf)", "documentation": "https://www.home-assistant.io/integrations/zeroconf", "requirements": ["zeroconf==0.37.0"], "dependencies": ["network", "api"], "codeowners": ["@bdraco"], "quality_scale": "internal", "iot_class": "local_push", "is_built_in": true}, {"domain": "logger", "name": "Logger", "documentation": "https://www.home-assistant.io/integrations/logger", "codeowners": ["@home-assistant/core"], "quality_scale": "internal", "is_built_in": true}, {"domain": "cloud", "name": "Home Assistant Cloud", "documentation": "https://www.home-assistant.io/integrations/cloud", "requirements": ["hass-nabucasa==0.50.0"], "dependencies": ["http", "webhook"], "after_dependencies": ["google_assistant", "alexa"], "codeowners": ["@home-assistant/cloud"], "iot_class": "cloud_push", "is_built_in": true}, {"domain": "onboarding", "name": "Home Assistant Onboarding", "documentation": "https://www.home-assistant.io/integrations/onboarding", "after_dependencies": ["hassio"], "dependencies": ["analytics", "auth", "http", "person"], "codeowners": ["@home-assistant/core"], "quality_scale": "internal", "is_built_in": true}, {"domain": "group", "name": "Group", "documentation": "https://www.home-assistant.io/integrations/group", "codeowners": ["@home-assistant/core"], "quality_scale": "internal", "iot_class": "calculated", "is_built_in": true}, {"domain": "automation", "name": "Automation", "documentation": "https://www.home-assistant.io/integrations/automation", "dependencies": ["blueprint", "trace"], "after_dependencies": ["device_automation", "webhook"], "codeowners": ["@home-assistant/core"], "quality_scale": "internal", "is_built_in": true}, {"domain": "updater", "name": "Updater", "documentation": "https://www.home-assistant.io/integrations/updater", "codeowners": ["@home-assistant/core"], "quality_scale": "internal", "iot_class": "cloud_polling", "is_built_in": true}, {"domain": "rest_command", "name": "RESTful Command", "documentation": "https://www.home-assistant.io/integrations/rest_command", "codeowners": [], "iot_class": "local_push", "is_built_in": true}, {"domain": "device_automation", "name": "Device Automation", "documentation": "https://www.home-assistant.io/integrations/device_automation", "codeowners": ["@home-assistant/core"], "quality_scale": "internal", "is_built_in": true}, {"domain": "usb", "name": "USB Discovery", "documentation": "https://www.home-assistant.io/integrations/usb", "requirements": ["pyudev==0.22.0", "pyserial==3.5"], "codeowners": ["@bdraco"], "dependencies": ["websocket_api"], "quality_scale": "internal", "iot_class": "local_push", "is_built_in": true}, {"domain": "config", "name": "Configuration", "documentation": "https://www.home-assistant.io/integrations/config", "dependencies": ["http"], "codeowners": ["@home-assistant/core"], "quality_scale": "internal", "is_built_in": true}, {"codeowners": ["@ludeeus"], "config_flow": true, "dependencies": ["http", "websocket_api", "frontend", "persistent_notification", "lovelace"], "documentation": "https://hacs.xyz/docs/configuration/start", "domain": "hacs", "iot_class": "cloud_polling", "issue_tracker": "https://github.com/hacs/integration/issues", "name": "HACS", "requirements": ["aiogithubapi>=21.11.0"], "version": "1.18.0", "is_built_in": false}, {"domain": "recorder", "name": "Recorder", "documentation": "https://www.home-assistant.io/integrations/recorder", "requirements": ["sqlalchemy==1.4.27"], "codeowners": ["@home-assistant/core"], "quality_scale": "internal", "iot_class": "local_push", "is_built_in": true}, {"domain": "analytics", "name": "Analytics", "documentation": "https://www.home-assistant.io/integrations/analytics", "codeowners": ["@home-assistant/core", "@ludeeus"], "dependencies": ["api", "websocket_api"], "after_dependencies": ["energy"], "quality_scale": "internal", "iot_class": "cloud_push", "is_built_in": true}, {"domain": "image", "name": "Image", "config_flow": false, "documentation": "https://www.home-assistant.io/integrations/image", "requirements": ["pillow==8.2.0"], "dependencies": ["http"], "codeowners": ["@home-assistant/core"], "quality_scale": "internal", "is_built_in": true}, {"domain": "my", "name": "My Home Assistant", "documentation": "https://www.home-assistant.io/integrations/my", "dependencies": ["frontend"], "codeowners": ["@home-assistant/core"], "quality_scale": "internal", "is_built_in": true}, {"domain": "api", "name": "Home Assistant API", "documentation": "https://www.home-assistant.io/integrations/api", "dependencies": ["http"], "codeowners": ["@home-assistant/core"], "quality_scale": "internal", "is_built_in": true}, {"domain": "sensor", "name": "Sensor", "documentation": "https://www.home-assistant.io/integrations/sensor", "codeowners": [], "quality_scale": "internal", "after_dependencies": ["recorder"], "is_built_in": true}, {"domain": "counter", "name": "Counter", "documentation": "https://www.home-assistant.io/integrations/counter", "codeowners": ["@fabaff"], "quality_scale": "internal", "is_built_in": true}, {"domain": "timer", "name": "Timer", "documentation": "https://www.home-assistant.io/integrations/timer", "codeowners": [], "quality_scale": "internal", "is_built_in": true}, {"domain": "zone", "name": "Zone", "config_flow": false, "documentation": "https://www.home-assistant.io/integrations/zone", "codeowners": ["@home-assistant/core"], "quality_scale": "internal", "is_built_in": true}, {"domain": "frontend", "name": "Home Assistant Frontend", "documentation": "https://www.home-assistant.io/integrations/frontend", "requirements": ["home-assistant-frontend==20211212.0"], "dependencies": ["api", "auth", "config", "device_automation", "http", "lovelace", "onboarding", "search", "system_log", "websocket_api"], "codeowners": ["@home-assistant/frontend"], "quality_scale": "internal", "is_built_in": true}, {"domain": "scene", "name": "Scenes", "documentation": "https://www.home-assistant.io/integrations/scene", "codeowners": ["@home-assistant/core"], "quality_scale": "internal", "is_built_in": true}, {"domain": "tag", "name": "Tags", "documentation": "https://www.home-assistant.io/integrations/tag", "codeowners": ["@balloob", "@dmulcahey"], "quality_scale": "internal", "is_built_in": true}, {"domain": "esphome", "name": "ESPHome", "config_flow": true, "documentation": "https://www.home-assistant.io/integrations/esphome", "requirements": ["aioesphomeapi==10.6.0"], "zeroconf": ["_esphomelib._tcp.local."], "codeowners": ["@OttoWinter", "@jesserockz"], "after_dependencies": ["zeroconf", "tag"], "iot_class": "local_push", "is_built_in": true}, {"domain": "trace", "name": "Trace", "documentation": "https://www.home-assistant.io/integrations/automation", "codeowners": ["@home-assistant/core"], "quality_scale": "internal", "is_built_in": true}, {"domain": "homeassistant", "name": "Home Assistant Core Integration", "documentation": "https://www.home-assistant.io/integrations/homeassistant", "codeowners": ["@home-assistant/core"], "quality_scale": "internal", "is_built_in": true}, {"domain": "notify", "name": "Notifications", "documentation": "https://www.home-assistant.io/integrations/notify", "codeowners": ["@home-assistant/core"], "quality_scale": "internal", "is_built_in": true}, {"domain": "http", "name": "HTTP", "documentation": "https://www.home-assistant.io/integrations/http", "requirements": ["aiohttp_cors==0.7.0"], "codeowners": ["@home-assistant/core"], "quality_scale": "internal", "iot_class": "local_push", "is_built_in": true}, {"domain": "energy", "name": "Energy", "documentation": "https://www.home-assistant.io/integrations/energy", "codeowners": ["@home-assistant/core"], "iot_class": "calculated", "dependencies": ["websocket_api", "history", "recorder"], "quality_scale": "internal", "is_built_in": true}, {"domain": "websocket_api", "name": "Home Assistant WebSocket API", "documentation": "https://www.home-assistant.io/integrations/websocket_api", "dependencies": ["http"], "codeowners": ["@home-assistant/core"], "quality_scale": "internal", "is_built_in": true}, {"domain": "input_boolean", "name": "Input Boolean", "documentation": "https://www.home-assistant.io/integrations/input_boolean", "codeowners": ["@home-assistant/core"], "quality_scale": "internal", "is_built_in": true}, {"domain": "switch", "name": "Switch", "documentation": "https://www.home-assistant.io/integrations/switch", "codeowners": [], "quality_scale": "internal", "is_built_in": true}, {"domain": "input_datetime", "name": "Input Datetime", "documentation": "https://www.home-assistant.io/integrations/input_datetime", "codeowners": ["@home-assistant/core"], "quality_scale": "internal", "is_built_in": true}, {"domain": "auth", "name": "Auth", "documentation": "https://www.home-assistant.io/integrations/auth", "dependencies": ["http"], "codeowners": ["@home-assistant/core"], "quality_scale": "internal", "is_built_in": true}, {"domain": "person", "name": "Person", "documentation": "https://www.home-assistant.io/integrations/person", "dependencies": ["image"], "after_dependencies": ["device_tracker"], "codeowners": [], "quality_scale": "internal", "iot_class": "calculated", "is_built_in": true}, {"domain": "input_number", "name": "Input Number", "documentation": "https://www.home-assistant.io/integrations/input_number", "codeowners": ["@home-assistant/core"], "quality_scale": "internal", "is_built_in": true}, {"domain": "media_source", "name": "Media Source", "documentation": "https://www.home-assistant.io/integrations/media_source", "dependencies": ["http"], "codeowners": ["@hunterjm"], "quality_scale": "internal", "is_built_in": true}, {"domain": "input_text", "name": "Input Text", "documentation": "https://www.home-assistant.io/integrations/input_text", "codeowners": ["@home-assistant/core"], "quality_scale": "internal", "is_built_in": true}, {"domain": "sun", "name": "Sun", "documentation": "https://www.home-assistant.io/integrations/sun", "codeowners": ["@Swamp-Ig"], "quality_scale": "internal", "iot_class": "calculated", "is_built_in": true}, {"domain": "device_tracker", "name": "Device Tracker", "documentation": "https://www.home-assistant.io/integrations/device_tracker", "dependencies": ["zone"], "after_dependencies": [], "codeowners": [], "quality_scale": "internal", "is_built_in": true}, {"domain": "system_log", "name": "System Log", "documentation": "https://www.home-assistant.io/integrations/system_log", "dependencies": ["http"], "codeowners": [], "quality_scale": "internal", "is_built_in": true}, {"domain": "ssdp", "name": "Simple Service Discovery Protocol (SSDP)", "documentation": "https://www.home-assistant.io/integrations/ssdp", "requirements": ["async-upnp-client==0.22.12"], "dependencies": ["network"], "after_dependencies": ["zeroconf"], "codeowners": [], "quality_scale": "internal", "iot_class": "local_push", "is_built_in": true}, {"domain": "default_config", "name": "Default Config", "documentation": "https://www.home-assistant.io/integrations/default_config", "dependencies": ["automation", "cloud", "counter", "dhcp", "energy", "frontend", "history", "input_boolean", "input_datetime", "input_number", "input_select", "input_text", "logbook", "map", "media_source", "mobile_app", "my", "network", "person", "scene", "script", "ssdp", "sun", "system_health", "tag", "timer", "usb", "updater", "webhook", "zeroconf", "zone"], "codeowners": [], "quality_scale": "internal", "is_built_in": true}, {"domain": "influxdb", "name": "InfluxDB", "documentation": "https://www.home-assistant.io/integrations/influxdb", "requirements": ["influxdb==5.2.3", "influxdb-client==1.14.0"], "codeowners": ["@fabaff", "@mdegat01"], "iot_class": "local_push", "is_built_in": true}, {"domain": "dhcp", "name": "DHCP Discovery", "documentation": "https://www.home-assistant.io/integrations/dhcp", "requirements": ["scapy==2.4.5", "aiodiscover==1.4.5"], "codeowners": ["@bdraco"], "quality_scale": "internal", "iot_class": "local_push", "is_built_in": true}, {"domain": "input_select", "name": "Input Select", "documentation": "https://www.home-assistant.io/integrations/input_select", "codeowners": ["@home-assistant/core"], "quality_scale": "internal", "is_built_in": true}, {"domain": "binary_sensor", "name": "Binary Sensor", "documentation": "https://www.home-assistant.io/integrations/binary_sensor", "codeowners": [], "quality_scale": "internal", "is_built_in": true}, {"domain": "logbook", "name": "Logbook", "documentation": "https://www.home-assistant.io/integrations/logbook", "dependencies": ["frontend", "http", "recorder"], "codeowners": [], "quality_scale": "internal", "is_built_in": true}, {"domain": "search", "name": "Search", "documentation": "https://www.home-assistant.io/integrations/search", "dependencies": ["websocket_api"], "after_dependencies": ["scene", "group", "automation", "script"], "codeowners": ["@home-assistant/core"], "quality_scale": "internal", "is_built_in": true}, {"domain": "system_health", "name": "System Health", "documentation": "https://www.home-assistant.io/integrations/system_health", "dependencies": ["http"], "codeowners": [], "quality_scale": "internal", "is_built_in": true}, {"domain": "statistics", "name": "Statistics", "documentation": "https://www.home-assistant.io/integrations/statistics", "after_dependencies": ["recorder"], "codeowners": ["@fabaff"], "quality_scale": "internal", "iot_class": "local_polling", "is_built_in": true}, {"domain": "webhook", "name": "Webhook", "documentation": "https://www.home-assistant.io/integrations/webhook", "dependencies": ["http"], "codeowners": [], "quality_scale": "internal", "is_built_in": true}, {"domain": "script", "name": "Scripts", "documentation": "https://www.home-assistant.io/integrations/script", "dependencies": ["blueprint", "trace"], "codeowners": ["@home-assistant/core"], "quality_scale": "internal", "is_built_in": true}, {"domain": "blueprint", "name": "Blueprint", "documentation": "https://www.home-assistant.io/integrations/blueprint", "codeowners": ["@home-assistant/core"], "quality_scale": "internal", "is_built_in": true}, {"domain": "history", "name": "History", "documentation": "https://www.home-assistant.io/integrations/history", "dependencies": ["http", "recorder"], "codeowners": ["@home-assistant/core"], "quality_scale": "internal", "is_built_in": true}, {"domain": "map", "name": "Map", "documentation": "https://www.home-assistant.io/integrations/map", "dependencies": ["frontend"], "codeowners": [], "quality_scale": "internal", "is_built_in": true}]}

I would appreciate any idea how I can further try to narrow down the reason why the plugin doesn't work for me.

Edit: I further tried if enabling HACS option "AppDaemon" or "NetDaemon" changes this behaviour, but it doesn't seem to make a diference

System Health

version core-2021.12.1
installation_type Home Assistant Core
dev false
hassio false
docker false
user homeassistant
virtualenv true
python_version 3.8.10
os_name Linux
os_version 5.4.0-91-generic
arch x86_64
timezone Europe/Berlin
Home Assistant Community Store GitHub API | ok -- | -- Github API Calls Remaining | 4969 Installed Version | 1.18.0 Stage | running Available Repositories | 970 Installed Repositories | 1
Home Assistant Cloud logged_in | false -- | -- can_reach_cert_server | ok can_reach_cloud_auth | ok can_reach_cloud | ok
Lovelace dashboards | 2 -- | -- resources | 0 views | 3 mode | storage
garbled1 commented 2 years ago

OK, so, what we need to know, is what protocol the device is talking. At a base level, we need to know if it's talking the same protocol as an ecowitt does, well, at all.

I am not familiar with tshark, or a good way to do this in wireshark, but tcpdump is my old friend here:

tcpdump -s1500 -X port 4199 (if it actually spits out ASCII, like it should you can switch that -X to -A to make it more readable)

If you can grab a bunch of data from it, we can see if it speaks the same protocol as the ecowitt. Once we have that, we can go from there to see what supporting it would look like. (ie, easy, or crazy hard)

ei-ke commented 2 years ago

Hi Tim,

I'll have access to something better than my mobile phone in 5 hours. I hope until than this probably not correctly formatted log helps:

13:08:06.081405 IP 192.168.4.10.57958 > gaia.localdomain.4199: Flags [S], seq 921038901, win 5840, options [mss 1460], length 0
E..,.-....db...
.....f.g6..5....`....1......
13:08:06.081431 IP gaia.localdomain.4199 > 192.168.4.10.57958: Flags [S.], seq 3446050979, ack 921038902, win 64240, options [mss 1460], length 0
E..,..@.@..........
.g.f.f..6..6`....Z......
13:08:06.082475 IP 192.168.4.10.57958 > gaia.localdomain.4199: Flags [.], ack 1, win 5840, length 0
E..(......de...
.....f.g6..6.f..P.......
13:08:06.259247 IP 192.168.4.10.57958 > gaia.localdomain.4199: Flags [P.], seq 1:349, ack 1, win 5840, length 348
E..../....c....
.....f.g6..6.f..P..."...GET /weatherstation/updateweatherstation.php?ID=&PASSWORD=&action=updateraww&realtime=1&rtfreq=5&dateutc=now&baromin=29.86&tempf=71.9&dewptf=51.8&humidity=49&windspeedmph=0.0&windgustmph=0.0&winddir=220&rainin=0.0&dailyrainin=0.03&solarradiation=0.0&UV=0.0&indoortempf=70.5&indoorhumidity=48 HTTP/1.1
Host:192.168.2.225
Connection: keep-alive

13:08:06.259275 IP gaia.localdomain.4199 > 192.168.4.10.57958: Flags [.], ack 349, win 63892, length 0
E..(..@.@..........
.g.f.f..6...P....V..
13:08:06.259743 IP gaia.localdomain.4199 > 192.168.4.10.57958: Flags [P.], seq 1:151, ack 349, win 63892, length 150
E.....@.@..c.......
.g.f.f..6...P.......HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Content-Length: 2
Date: Tue, 14 Dec 2021 12:08:06 GMT
Server: Python/3.8 aiohttp/3.8.1

13:08:06.259901 IP gaia.localdomain.4199 > 192.168.4.10.57958: Flags [P.], seq 151:153, ack 349, win 63892, length 2
E..*..@.@..........
.g.f.f.:6...P....X..OK
13:08:06.265068 IP 192.168.4.10.57958 > gaia.localdomain.4199: Flags [.], ack 153, win 5688, length 0
E..(.0....dc...
.....f.g6....f.<P..8.w..
13:08:06.494594 IP 192.168.4.10.57958 > gaia.localdomain.4199: Flags [F.], seq 349, ack 153, win 5688, length 0
E..(.1....db...
.....f.g6....f.<P..8.v..
13:08:06.494844 IP gaia.localdomain.4199 > 192.168.4.10.57958: Flags [F.], seq 153, ack 350, win 63892, length 0
E..(..@.@..........
.g.f.f.<6...P....V..
13:08:06.496025 IP 192.168.4.10.57958 > gaia.localdomain.4199: Flags [.], ack 154, win 5687, length 0
E..(.2....da...
.....f.g6....f.=P..7.v..
13:08:18.206174 IP 192.168.4.10.57960 > gaia.localdomain.4199: Flags [S], seq 921457329, win 5840, options [mss 1460], length 0
E..,.9....dV...
.....h.g6.R.....`...|.......
13:08:18.206199 IP gaia.localdomain.4199 > 192.168.4.10.57960: Flags [S.], seq 1452174607, ack 921457330, win 64240, options [mss 1460], length 0
E..,..@.@..........
.g.hV.m.6.R.`....Z......
13:08:18.207331 IP 192.168.4.10.57960 > gaia.localdomain.4199: Flags [.], ack 1, win 5840, length 0
E..(.:....dY...
.....h.g6.R.V.m.P.......
13:08:18.381438 IP 192.168.4.10.57960 > gaia.localdomain.4199: Flags [P.], seq 1:349, ack 1, win 5840, length 348
E....;....b....
.....h.g6.R.V.m.P...Z...GET /weatherstation/updateweatherstation.php?ID=&PASSWORD=&action=updateraww&realtime=1&rtfreq=5&dateutc=now&baromin=29.86&tempf=71.9&dewptf=51.8&humidity=49&windspeedmph=0.0&windgustmph=0.0&winddir=220&rainin=0.0&dailyrainin=0.03&solarradiation=0.0&UV=0.0&indoortempf=70.5&indoorhumidity=48 HTTP/1.1
Host:192.168.2.225
Connection: keep-alive

13:08:18.381464 IP gaia.localdomain.4199 > 192.168.4.10.57960: Flags [.], ack 349, win 63892, length 0
E..(.j@.@..*.......
.g.hV.m.6.T.P....V..
13:08:18.381909 IP gaia.localdomain.4199 > 192.168.4.10.57960: Flags [P.], seq 1:151, ack 349, win 63892, length 150
E....k@.@..........
.g.hV.m.6.T.P.......HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Content-Length: 2
Date: Tue, 14 Dec 2021 12:08:18 GMT
Server: Python/3.8 aiohttp/3.8.1

13:08:18.382024 IP gaia.localdomain.4199 > 192.168.4.10.57960: Flags [P.], seq 151:153, ack 349, win 63892, length 2
E..*.l@.@..&.......
.g.hV.m.6.T.P....X..OK
13:08:18.383360 IP 192.168.4.10.57960 > gaia.localdomain.4199: Flags [.], ack 153, win 5688, length 0
E..(.<....dW...
.....h.g6.T.V.m.P..8._..
13:08:18.612524 IP 192.168.4.10.57960 > gaia.localdomain.4199: Flags [F.], seq 349, ack 153, win 5688, length 0
E..(.=....dV...
.....h.g6.T.V.m.P..8.^..
13:08:18.612773 IP gaia.localdomain.4199 > 192.168.4.10.57960: Flags [F.], seq 153, ack 350, win 63892, length 0
E..(.m@.@..'.......
.g.hV.m.6.T.P....V..
13:08:18.613948 IP 192.168.4.10.57960 > gaia.localdomain.4199: Flags [.], ack 154, win 5687, length 0
E..(.>....dU...
.....h.g6.T.V.m.P..7.^..
garbled1 commented 2 years ago

The protocol looks the same, but I'm confused by the GET....

In the ecowitt setup, we setup a mini-web-server on 4199, and then the ecowitt sends data to that with a POST, where it dumps all the data onto the web server, and then we interpret that string, and update everything.

A GET is, backwards?

Oh, also, can you tell me what each of those IP's are? IE, I don't know what "gaia" is vs what 192.168.4.10 is.

ei-ke commented 2 years ago

Yeah, I also saw the "GET" and hust thought "whatever, if it works..." I also had a look at the different files of your plugin but programming, well, nothing that I'm really capable of until now.

Oh, sorry 192.168.2.225 (gaia) is the VM running HA on a Proxmox host. 192.168.4.10 is the Bresser base station in my "internet of shit" WLAN/VLAN :)

Edit: As soon as I'm at home I'll have a look at the FOSHKplugin. Seems that the Bresser base station spits out some weather undergroundish stuff:

FOSHKplugin acts as a web server and returns different values depending on the requested URL. In addition to "updateweatherstation" to accept a incoming data record in WU format (Weather Underground protocol) the integrated web server processes other http call parameters in GET: http://serverip:port/[URLpath]

garbled1 commented 2 years ago

I mean, it's not sending a ton of data, jsut a few stats, are those all of your various sensors? In theory, pyecowitt could be hacked up to take the GET params and use those instead of the POST when given.. I guess?

ei-ke commented 2 years ago

bresser_tcpdump.txt That's approx. 10 minutes of data sent by the station.

I'll play around with the pyecowitt library in another VM. For testing should I just directly modify the package in /opt/homeassistant/lib/python3.8/site-packages/pyecowitt or do the modification in the git clone and go with python setup.py install after source /opt/homeassistant/bin/activate? My HA installation is just on a plain ubuntu server without docker etc.

garbled1 commented 2 years ago

Either way works.. the first is probably the easiest way to just try it out and see.

ei-ke commented 2 years ago

Changing POST to GET in pyecowitt/pyecowitt/ecowitt.py:

2021-12-14 23:12:18 INFO (MainThread) [custom_components.ecowitt] First ecowitt data recd, setting up sensors.
2021-12-14 23:12:18 ERROR (MainThread) [custom_components.ecowitt] No passkey, cannot set unique id.

Commenting out the passkey check ends with:

2021-12-14 23:42:06 ERROR (MainThread) [custom_components.ecowitt] No sensors found to monitor, check device config.
garbled1 commented 2 years ago

Passkey will be an issue, but we could potentially fake it..

Try this instead. In the pyecowit library, the main.py will just fire up a webserver, that prints out everything it gets. This will let you test out your GET modifications to make sure the data is actually passing through..

You might have to change a little bit in there more than just POST->GET, like how the data is pulled in... iirc I'm just using aiohttp there, so it shouldn't be super hard.

ei-ke commented 2 years ago

Thats how far I made it until now. So far the missing sensors show up using the fake_client with the standard POST request

Currently I'm struggling here as I assume I need to set the relative URL /weatherstation/updateweatherstation.php? with the rel_url function of aiohttp. But I also don't know if web.TCPSite supports this at all.

For changing POST to GET I assume that request.post() must be replaced with with request.text() to be able to receive the data as GET with content type text/plain, But then I have no clue how to parse this into a MultiDict or what I would need to replace the MultiDict you use with.

Some hints what I would need to google for would be highly appreciated :)

garbled1 commented 2 years ago

https://docs.aiohttp.org/en/stable/web_reference.html

You want the request.query(), because that will return a multidict of the query params, which more or less should just map in. I think the server doesn't even look at the URL string at all, it just assumes whatever it got was a proper connection and parses it.

ei-ke commented 2 years ago

Thank you for your ongoing support. Seems to be working for the Bresser station now. Only have to add some more conversions in the ecowitt integration due to new sensor names. Would it make sense for you to also integrate that Bresser related stuff into your repo or should I keep it as a fork and rename/reduce it to Bresser only (for the integration as well as the pyecowitt library)? If you want to integrate it I'll try to clean my mess up as good as I can. For identifying Bresser stations maybe the ID property in the Bresser stations web interface should be used to identify it. Every Bresser user would need to set the Bresser article number as ID and a random password that will be used as Passkey. BTW: The Bresser stations are not from FOSHK but from CCLEL. So maybe there article number should be used.

Edit again: And is there an "easy" way to create localisations for the SENSOR_TYPES?

garbled1 commented 2 years ago

I think try to clean up what you can, so maybe I can pull it and work from there, rather than have to reinvent your wheel. I might not be able to get to it right away though...

My thoughts are going to be that somehow, you need to pick one of those other values, like ID, and internally in pyecowitt it will need to be remapped into PASSKEY so the integration works. I'm currently working on trying to get ecowitt into base HA, so I'm trying not to maintian both the HACS version and that one.. so there might be a time delay before I get everything working. You may want to maintain your local hack until then.

As for translations... not really, at least not yet..

ei-ke commented 2 years ago

https://github.com/ei-ke/pyecowitt/tree/pr-cclel No further work needed except than getting the fake client updated. Somehow I'm struggling with that.

https://github.com/ei-ke/homeassistant_ecowitt/tree/pr-cclel TODO: const.py LN55-56 should be automatically switched when the integration notices a GET instead of a POST, but I have no idea where to do this check so that the variable can be read from different parts of the programm.

Further I currently get two dewpoint sensors shown and I'm not sure how I should handle it. The station calculates the dewpoint based on the outdoor sensor.

            {
                "area_id": null,
                "capabilities": {
                    "state_class": "measurement"
                },
                "config_entry_id": "f2047c569cd74c3c5fa5ec60a0000566",
                "device_class": null,
                "device_id": "e25e05dc8860b8de47ca781e2f112d78",
                "disabled_by": null,
                "entity_category": null,
                "entity_id": "sensor.dewpoint",
                "icon": null,
                "id": "638f82b057eceefc54bae2ec1d3d90a3",
                "name": null,
                "original_device_class": "temperature",
                "original_icon": "mdi:thermometer",
                "original_name": "Dewpoint",
                "platform": "ecowitt",
                "supported_features": 0,
                "unique_id": "bresser1234-dewptf",
                "unit_of_measurement": "\u00b0C"
            },
[....]
            {
                "area_id": null,
                "capabilities": {
                    "state_class": "measurement"
                },
                "config_entry_id": "f2047c569cd74c3c5fa5ec60a0000566",
                "device_class": null,
                "device_id": "e25e05dc8860b8de47ca781e2f112d78",
                "disabled_by": null,
                "entity_category": null,
                "entity_id": "sensor.dewpoint_2",
                "icon": null,
                "id": "4bb81a653dede4094b869a595e0ac3c8",
                "name": null,
                "original_device_class": "temperature",
                "original_icon": "mdi:thermometer",
                "original_name": "Dewpoint",
                "platform": "ecowitt",
                "supported_features": 0,
                "unique_id": "bresser1234-dewpointc",
                "unit_of_measurement": "\u00b0C"
            }
garbled1 commented 2 years ago

Ahh.. right.. so one of the things the ecowitt code does, is translate any incoming F temps into C automatically. The newer code handles this differently than the older code, so if yuo are using newer pyecowitt with older HACS integration it won't all line up. Once I get a chance to sort through most of this, I can fix that up.

I think the big remaining thing is something for the unique id... hrmm

ei-ke commented 2 years ago

For the unique ID I would still suggest to tell the users to set the password entry on the base station to something random. Further a hint is needed that the user needs to redirect the traffic from P80 to e. g. P4199 as the base station supports no port directive. Should be no problem to do this with iptables when port 80 is not in use. Otherwise I guess if the webserver a user might run on P80 supports reverse proxying this could also be a solution.

garbled1 commented 2 years ago

Hrmm.. Would you mind typing up some instructions for setup... since it's a bit fiddly?

ei-ke commented 2 years ago

No problem. Will do so within the next days :)

ei-ke commented 2 years ago

Sorry for the epic delay, but at least I've added a short howto.