Pull Request / Add MQTT and persistent connection for real time

mrwiwi commented 4 years ago

Hi, still haven't processed how to use github, so if you could have a look at my code here :

It's a WIP of your client interfaced with a mqtt broker (here on homeassistant), i'm still learning python and programming in general, but it's working :)

Hard time on parsing data, can't use the type arg anymore (websocket is listened permanently, so we can have real time update of our devices), we need or more adaptable function.

Please someone review it, we're not far from a clean client i think.

Auto reconnect is implemented.

Output sample : `S C:\Users\lipit> & C:/bin/Python38-32/python.exe d:/Drive/Domotique/pydom/pydom.py ################################## Websocket is closed or inexistant, trying.... "Attempting websocket connection..." Tydom Websocket is Connected ! <websockets.client.WebSocketClientProtocol object at 0x0435C760> Attempting MQTT connection... ################################## MQTT is connected ! =) Subscribing to : homeassistant/+/tydom/# ################################## Requesting 1st data... MQTT SUBSCRIBED

1234568029 Baie 1234568089 Panoramique 1234568149 Cosina 1234568209 Salle De Bain 1234568269 Chambre Parents 1234568329 Chambre Quentin 1234568389 Chambre 3 1234591388 Alarme Cannot parse response string indices must be integers

Cannot parse response list indices must be integers or slices, not str

MQTT incoming: homeassistant/sensor/tydom/last_update b'2019-12-11 00:32:54.442518'

MQTT incoming: homeassistant/sensor/tydom/last_update b'2019-12-11 00:32:54.557796'

MQTT incoming: homeassistant/requests/tydom/update b'please'

1234568029 Baie 1234568089 Panoramique 1234568149 Cosina 1234568209 Salle De Bain 1234568269 Chambre Parents 1234568329 Chambre Quentin 1234568389 Chambre 3 1234591388 Alarme Cannot parse response`

Here it is !

https://pastebin.com/xy9R2RGk

cth35 commented 4 years ago

Hi great job ;) I will have a look to it and will merge it to the trunk. Thanks a lot for your contribution.

mrwiwi commented 4 years ago

Hi great job ;) I will have a look to it and will merge it to the trunk. Thanks a lot for your contribution.

Thanks a lot !!

I've updated everything, now with handling of error, single parsing every kind of message, still figuring out some put commands but hey, it's stable as hell :)

https://pastebin.com/xssNyj5s

#!/usr/bin/env python
import asyncio
import websockets
import http.client
from requests.auth import HTTPDigestAuth
import sys
import logging
from http.client import HTTPResponse
from io import BytesIO
import urllib3
import json
import os
import base64
import time
from http.server import BaseHTTPRequestHandler
import ssl
from datetime import datetime
from gmqtt import Client as MQTTClient

# Globals
####################################### MQTT
tydom_topic = "homeassistant/+/tydom/#"

cover_config_topic = "homeassistant/cover/tydom/{id}/config"
cover_config = "homeassistant/cover/tydom/{id}/config"
cover_position_topic = "homeassistant/cover/tydom/{id}/current_position"
cover_set_postion_topic = "homeassistant/cover/tydom/{id}/set_position"
cover_attributes_topic = "homeassistant/cover/tydom/{id}/attributes"

alarm_topic = "homeassistant/alarm_control_panel/tydom/#"
alarm_config = "homeassistant/alarm_control_panel/tydom/{id}/config"
alarm_state_topic = "homeassistant/alarm_control_panel/tydom/{id}/state"
alarm_command_topic = "homeassistant/alarm_control_panel/tydom/{id}/set"
alarm_sos_topic = "homeassistant/binary_sensor/tydom/{id}/sos"
alarm_attributes_topic = "homeassistant/alarm_control_panel/tydom/{id}/attributes"

refresh_topic = "homeassistant/tydom/please_update"

mac = "" #sys.argv[2]
login = mac
password = "" #sys.argv[3]
host = "mediation.tydom.com" #"192.168.0.20" # Local ip address or mediation.tydom.com for remote connexion

mqtt_host = ''
mqtt_user = ''
mqtt_pass = ''
mqtt_port = 8883
mqtt_ssl = True
#INIT Servers
hassio = None
tydom = None

# Set Host, ssl context and prefix for remote or local connection
if host == "mediation.tydom.com":
    remote_mode = True
    ssl_context = None
    cmd_prefix = "\x02"
else:
    remote_mode = False
    ssl_context = ssl._create_unverified_context()
    cmd_prefix = ""

deviceAlarmKeywords = ['alarmMode','alarmState','alarmSOS','zone1State','zone2State','zone3State','zone4State','zone5State','zone6State','zone7State','zone8State','gsmLevel','inactiveProduct','zone1State','liveCheckRunning','networkDefect','unitAutoProtect','unitBatteryDefect','unackedEvent','alarmTechnical','systAutoProtect','sysBatteryDefect','zsystSupervisionDefect','systOpenIssue','systTechnicalDefect','videoLinkDefect']
# Device dict for parsing
device_dict = dict()

#MQTT
STOP = asyncio.Event()
def on_connect(client, flags, rc, properties):
    print("##################################")

    print("Subscribing to : ", tydom_topic)
    # client.subscribe('homeassistant/#', qos=0)
    client.subscribe(tydom_topic, qos=0)

async def on_message(client, topic, payload, qos, properties):
    # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
    # print('MQTT incoming : ', topic, payload)
    if (topic == "homeassistant/requests/tydom/update") or (payload == "please"):
        await get_data(tydom)
    if ('set_position' in str(topic)):
        print('MQTT set_position incoming : ', topic, payload)
        get_id = (topic.split("/"))[3] #extract id from mqtt
        await put_devices_data(tydom, str(get_id), 'position', str(payload))
    else:
        return 0

async def on_disconnect(client, packet, exc=None):
    print('MQTT Disconnected')
    print("##################################")
    await main_task()

def on_subscribe(client, mid, qos):
    print("MQTT is connected and suscribed ! =)")

def ask_exit(*args):
    STOP.set()

async def mqttconnection(broker_host, user, password):

    global hassio
    if (hassio == None):
        print('Attempting MQTT connection...')
        client = MQTTClient("client-id")

        client.on_connect = on_connect
        client.on_message = on_message
        client.on_disconnect = on_disconnect
        client.on_subscribe = on_subscribe

        client.set_auth_credentials(user, password)
        await client.connect(broker_host, port=mqtt_port, ssl=mqtt_ssl)
        hassio = client

    # client.publish('TEST/TIME', str(time.time()), qos=1)

    # await STOP.wait()
    # await client.disconnect()

#######" END MQTT"

class Cover:
    def __init__(self, id, name, current_position=None, set_position=None, attributes=None):

        self.id = id
        self.name = name
        self.current_position = current_position
        self.set_position = set_position
        self.attributes = attributes

    def id(self):
        return self.id

    def name(self):
        return self.name

    def current_position(self):
        return self.current_position

    def set_position(self):
        return self.set_position

    def attributes(self):
        return self.attributes

    # cover_config_topic = "homeassistant/cover/tydom/{id}/config"
    # cover_position_topic = "homeassistant/cover/tydom/{id}/current_position"
    # cover_set_postion_topic = "homeassistant/cover/tydom/{id}/set_position"
    # cover_attributes_topic = "homeassistant/cover/tydom/{id}/attributes"

    def setup(self):
        self.device = {}
        self.device['manufacturer'] = 'Delta Dore'
        self.device['model'] = 'Volet'
        self.device['name'] = self.name
        self.device['identifiers'] = id=self.id
        self.config_topic = cover_config_topic.format(id=self.id)
        self.config = {}
        self.config['name'] = self.name
        self.config['unique_id'] = self.id
        # self.config['attributes'] = self.attributes
        self.config['command_topic'] = cover_set_postion_topic.format(id=self.id)
        self.config['set_position_topic'] = cover_set_postion_topic.format(id=self.id)
        self.config['position_topic'] = cover_position_topic.format(id=self.id)
        self.config['payload_open'] = 100
        self.config['payload_close'] = 0
        self.config['retain'] = 'true'
        self.config['device'] = self.device

        # print(self.config)
        hassio.publish(self.config_topic, json.dumps(self.config), qos=0)

    def update(self):
        self.setup()
        self.position_topic = cover_position_topic.format(id=self.id, current_position=self.current_position)
        hassio.publish(self.position_topic, self.current_position, qos=0, retain=True)

        # self.attributes_topic = cover_attributes_topic.format(id=self.id, attributes=self.attributes)
        # hassio.publish(self.attributes_topic, self.attributes, qos=0)

class Alarm:
    def __init__(self, id, name, current_state=None, attributes=None):
        self.id = id
        self.name = name
        self.current_state = current_state
        self.attributes = attributes

    # def id(self):
    #     return id

    # def name(self):
    #     return name

    # def current_state(self):
    #     return current_state

    # def attributes(self):
    #     return attributes

    def setup(self):
        self.device = {}
        self.device['manufacturer'] = 'Delta Dore'
        self.device['model'] = 'Tyxal'
        self.device['name'] = self.name
        self.device['identifiers'] = id=self.id
        self.config_alarm = alarm_config.format(id=self.id)
        self.config = {}
        self.config['name'] = self.name
        self.config['unique_id'] = self.id
        self.config['device'] = self.device
        # self.config['attributes'] = self.attributes
        self.config['command_topic'] = alarm_command_topic.format(id=self.id)
        self.config['state_topic'] = alarm_state_topic.format(id=self.id)

        # print(self.config)
        hassio.publish(self.config_alarm, json.dumps(self.config), qos=0)

    def update(self):
        self.setup()
        self.state_topic = alarm_state_topic.format(id=self.id, state=self.current_state)
        hassio.publish(self.state_topic, self.current_state, qos=0, retain=True)

        # self.attributes_topic = alarm_attributes_topic.format(id=self.id, attributes=self.attributes)
        # hassio.publish(self.attributes_topic, self.attributes, qos=0)

    # alarm_topic = "homeassistant/alarm_control_panel/tydom/#"
    # alarm_config = "homeassistant/alarm_control_panel/tydom/{id}/config"
    # alarm_state_topic = "homeassistant/alarm_control_panel/tydom/{id}/state"
    # alarm_sos_topic = "homeassistant/binary_sensor/tydom/{id}/sos"
    # alarm_attributes_topic = "homeassistant/alarm_control_panel/tydom/{id}/attributes"

class BytesIOSocket:
    def __init__(self, content):
        self.handle = BytesIO(content)

    def makefile(self, mode):
        return self.handle

class HTTPRequest(BaseHTTPRequestHandler):
    def __init__(self, request_text):
        #self.rfile = StringIO(request_text)
        self.raw_requestline = request_text
        self.error_code = self.error_message = None
        self.parse_request()

    def send_error(self, code, message):
        self.error_code = code
        self.error_message = message

def response_from_bytes(data):
    sock = BytesIOSocket(data)
    response = HTTPResponse(sock)
    response.begin()
    return urllib3.HTTPResponse.from_httplib(response)

def put_response_from_bytes(data):
    request = HTTPRequest(data)
    return request

# Get pretty name for a device id
def get_name_from_id(id):
    name = ""
    if len(device_dict) != 0:
        name = device_dict[id]
    return(name)

# Basic response parsing. Typically GET responses
async def parse_response(incoming):
    data = incoming
    msg_type = None
    first = str(data[:20])

    # Detect type of incoming data
    if (data != ''):
        if ("id" in first):
            print('Incoming message type : data detected')
            msg_type = 'msg_data'
        elif ("date" in first):
            print('Incoming message type : config detected')
            msg_type = 'msg_config'
        elif ("doctype" in first):
            print('Incoming message type : html detected (probable pong)')
            msg_type = 'msg_html'
            print(data)
        else:
            print('Incoming message type : no type detected')
            print(first)

        if not (msg_type == None):
            try:
                parsed = json.loads(data)
                # print(parsed)
                if (msg_type == 'msg_config'):
                    for i in parsed["endpoints"]:
                        # Get list of shutter
                        if i["last_usage"] == 'shutter':
                            # print('{} {}'.format(i["id_endpoint"],i["name"]))
                            device_dict[i["id_endpoint"]] = i["name"]

                            # TODO get other device type
                        if i["last_usage"] == 'alarm':
                            # print('{} {}'.format(i["id_endpoint"], i["name"]))
                            device_dict[i["id_endpoint"]] = "Tyxal Alarm"
                    print('Configuration updated')
                elif (msg_type == 'msg_data'):
                    for i in parsed:
                        attr = {}
                        if i["endpoints"][0]["error"] == 0:
                            for elem in i["endpoints"][0]["data"]:
                                # Get full name of this id
                                endpoint_id = i["endpoints"][0]["id"]
                                # Element name
                                elementName = elem["name"]
                                # Element value
                                elementValue = elem["value"]

                                # Get last known position (for shutter)
                                if elementName == 'position':
                                    name_of_id = get_name_from_id(endpoint_id)
                                    if len(name_of_id) != 0:
                                        print_id = name_of_id
                                    else:
                                        print_id = endpoint_id
                                    # print('{} : {}'.format(print_id, elementValue))
                                    new_cover = "cover_tydom_"+str(endpoint_id)
                                    print("Cover created / updated : "+new_cover)
                                    new_cover = Cover(id=endpoint_id,name=print_id, current_position=elementValue, attributes=i)
                                    new_cover.update()

                                # Get last known position (for alarm)
                                if elementName in deviceAlarmKeywords:
                                    alarm_data = '{} : {}'.format(elementName, elementValue)
                                    # print(alarm_data)
                                    # alarmMode  : ON or ZONE or OFF
                                    # alarmState : ON = Triggered
                                    # alarmSOS   : true = SOS triggered
                                    state = None
                                    sos = False

                                    if alarm_data == "alarmMode : ON":
                                        state = "armed_away"
                                    elif alarm_data == "alarmMode : ZONE":
                                        state = "armed_home"
                                    elif alarm_data == "alarmMode : OFF":
                                        state = "disarmed"
                                    elif alarm_data == "alarmState : ON":
                                        state = "triggered"
                                    elif alarm_data == "alarmSOS : true":
                                        sos = True
                                    else:
                                        attr[elementName] = [elementValue]
                                    #     attr[alarm_data]
                                        # print(attr)
                                    #device_dict[i["id_endpoint"]] = i["name"]
                                    if (sos == True):
                                        print("SOS !")
                                    if not (state == None):
                                        # print(state)
                                        alarm = "alarm_tydom_"+str(endpoint_id)
                                        print("Alarm created / updated : "+alarm)
                                        alarm = Alarm(id=endpoint_id,name="Tyxal Alarm", current_state=state, attributes=attr)
                                        alarm.update()
                elif (msg_type == 'msg_html'):
                    print("pong")
                else:
                    # Default json dump
                    print()
                    print(json.dumps(parsed, sort_keys=True, indent=4, separators=(',', ': ')))
            except Exception as e:
                print('Cannot parse response !')
                # print('Response :')
                # print(data)
                if (e != 'Expecting value: line 1 column 1 (char 0)'):
                    print(e)

# PUT response DIRTY parsing
def parse_put_response(bytes_str):
    # TODO : Find a cooler way to parse nicely the PUT HTTP response
    resp = bytes_str[len(cmd_prefix):].decode("utf-8")
    fields = resp.split("\r\n")
    fields = fields[6:]  # ignore the PUT / HTTP/1.1
    end_parsing = False
    i = 0
    output = str()
    while not end_parsing:
        field = fields[i]
        if len(field) == 0 or field == '0':
            end_parsing = True
        else:
            output += field
            i = i + 2
    parsed = json.loads(output)
    return json.dumps(parsed)
    # print(json.dumps(parsed, sort_keys=True, indent=4, separators=(',', ': ')))

# Generate 16 bytes random key for Sec-WebSocket-Keyand convert it to base64
def generate_random_key():
    return base64.b64encode(os.urandom(16))

# Build the headers of Digest Authentication
def build_digest_headers(nonce):
    digestAuth = HTTPDigestAuth(login, password)
    chal = dict()
    chal["nonce"] = nonce[2].split('=', 1)[1].split('"')[1]
    chal["realm"] = "ServiceMedia" if remote_mode is True else "protected area"
    chal["qop"] = "auth"
    digestAuth._thread_local.chal = chal
    digestAuth._thread_local.last_nonce = nonce
    digestAuth._thread_local.nonce_count = 1
    return digestAuth.build_digest_header('GET', "https://{}:443/mediation/client?mac={}&appli=1".format(host, mac))

# Send Generic GET message
async def send_message(websocket, msg):
    str = cmd_prefix + "GET " + msg +" HTTP/1.1\r\nContent-Length: 0\r\nContent-Type: application/json; charset=UTF-8\r\nTransac-Id: 0\r\n\r\n"
    a_bytes = bytes(str, "ascii")
    await websocket.send(a_bytes)
    return 0
    # return await websocket.recv() #disable if handler

# Send Generic POST message
async def send_post_message(websocket, msg):
    str = cmd_prefix + "POST " + msg +" HTTP/1.1\r\nContent-Length: 0\r\nContent-Type: application/json; charset=UTF-8\r\nTransac-Id: 0\r\n\r\n"
    a_bytes = bytes(str, "ascii")
    await websocket.send(a_bytes)
    return 0
    # return await websocket.recv()

###############################################################
# Commands                                                    #
###############################################################

# Get some information on Tydom
async def get_info(websocket):
    msg_type = '/info'
    parse_response(await send_message(websocket, msg_type), msg_type)

# Refresh (all)
async def post_refresh(websocket):
    print("Refresh....")
    msg_type = '/refresh/all'
    await send_post_message(websocket, msg_type)

# Get the moments (programs)
async def get_moments(websocket):
    msg_type = '/moments/file'
    await send_message(websocket, msg_type)

# Get the scenarios
async def get_scenarios(websocket):
    msg_type = '/scenarios/file'
    await send_message(websocket, msg_type)

# Get a ping (pong should be returned)
async def get_ping(websocket):
    msg_type = 'ping'
    await send_message(websocket, msg_type)

# Get all devices metadata
async def get_devices_meta(websocket):
    msg_type = '/devices/meta'
    parse_response(await send_message(websocket, msg_type), msg_type)

# Get all devices data
async def get_devices_data(websocket):
    msg_type = '/devices/data'
    await send_message(websocket, msg_type)

# List the device to get the endpoint id
async def get_configs_file(websocket):
    msg_type = '/configs/file'
    await send_message(websocket, msg_type)

async def get_data(websocket):
    await get_configs_file(websocket)
    await asyncio.sleep(2)
    await get_devices_data(websocket)

# Give order (name + value) to endpoint
async def put_devices_data(websocket, endpoint_id, name, value):
    # For shutter, value is the percentage of closing
    body="[{\"name\":\"" + name + "\",\"value\":\""+ value + "\"}]"
    # endpoint_id is the endpoint = the device (shutter in this case) to open.
    str_request = cmd_prefix + "PUT /devices/{}/endpoints/{}/data HTTP/1.1\r\nContent-Length: ".format(str(endpoint_id),str(endpoint_id))+str(len(body))+"\r\nContent-Type: application/json; charset=UTF-8\r\nTransac-Id: 0\r\n\r\n"+body+"\r\n\r\n"
    a_bytes = bytes(str_request, "ascii")
    await websocket.send(a_bytes)
    # name = await websocket.recv()
    # parse_response(name)
    # name = await websocket.recv()
    # try:
    #     parse_response(name)
    # except:
    #     parse_put_response(name)

# Run scenario
async def put_scenarios(websocket, scenario_id):
    body=""
    # scenario_id is the id of scenario got from the get_scenarios command
    str_request = cmd_prefix + "PUT /scenarios/{} HTTP/1.1\r\nContent-Length: ".format(str(scenario_id))+str(len(body))+"\r\nContent-Type: application/json; charset=UTF-8\r\nTransac-Id: 0\r\n\r\n"+body+"\r\n\r\n"
    a_bytes = bytes(str_request, "ascii")
    await websocket.send(a_bytes)
    name = await websocket.recv()
    parse_response(name)

# Give order to endpoint
async def get_device_data(websocket, id):
    # 10 here is the endpoint = the device (shutter in this case) to open.
    str_request = cmd_prefix + "GET /devices/{}/endpoints/{}/data HTTP/1.1\r\nContent-Length: 0\r\nContent-Type: application/json; charset=UTF-8\r\nTransac-Id: 0\r\n\r\n".format(str(id),str(id))
    a_bytes = bytes(str_request, "ascii")
    await websocket.send(a_bytes)
    # name = await websocket.recv()
    # parse_response(name)

######## Messages Logic

async def consumer(websocket):
    # Receiver
    while True:
        bytes_str = await websocket.recv()
        first = str(bytes_str[:20]) # Scanning 1st characters

        if ("PUT" in first):
            print('PUT message detected !')

            # print('RAW INCOMING :')
            # print(bytes_str)
            # print('END RAW')

            try:
                incoming = parse_put_response(bytes_str)
                # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                await parse_response(incoming)
                print('PUT message processed !')
            except:
                print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                print(incoming)
                print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")

        elif ("POST" in first):

            # print('RAW INCOMING :')
            # print(bytes_str)
            # print('END RAW')

            try:
                # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                incoming = parse_put_response(bytes_str)
                await parse_response(incoming)
                print('POST message processed !')
            except:
                print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                print('RAW INCOMING :')
                print(bytes_str)
                print('END RAW')

                print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
        elif ("HTTP/1.1" in first): #(bytes_str != 0) and 
            response = response_from_bytes(bytes_str[len(cmd_prefix):])
            incoming = response.data.decode("utf-8")
            # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
            # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
            # print(incoming)
            # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
            # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
            hassio.publish('homeassistant/sensor/tydom/last_update', str(datetime.fromtimestamp(time.time())), qos=1)
            try:
                # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                await parse_response(incoming)
                print('GET response message processed !')
            except:
                print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                print(incoming)
                print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                # await parse_put_response(incoming)
        else:
            print(bytes_str)

async def producer(websocket):
        # while True:
    await asyncio.sleep(48)
    # await get_ping(websocket)
    await get_data(tydom)
    print("Websocket refreshed at ", str(datetime.fromtimestamp(time.time())))

async def consumer_handler(websocket):
    while True:
        try:
            await consumer(websocket)
        except Exception as e:
            print("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!")
            print("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!")
            print("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!")
            print('Consumer task has crashed !')
            print(e)
            print('Restarting..............')
            await main_task()

async def producer_handler(websocket):
    while True:
        await producer(websocket)

######## HANDLER
async def handler(websocket):
    try:
        # print("Starting handlers...")
        consumer_task = asyncio.ensure_future(
            consumer_handler(websocket))
        producer_task = asyncio.ensure_future(
            producer_handler(websocket))
        done, pending = await asyncio.wait(
            [consumer_task, producer_task],
            return_when=asyncio.FIRST_COMPLETED,
        )
        for task in pending:
            task.cancel()

    except Exception as e:
        print("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!")
        print(e)
        print('Handler crashed.')

async def websocket_connection():
    global tydom
    # logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)
    httpHeaders =  {"Connection": "Upgrade",
                    "Upgrade": "websocket",
                    "Host": host + ":443",
                    "Accept": "*/*",
                    "Sec-WebSocket-Key": generate_random_key(),
                    "Sec-WebSocket-Version": "13"
                    }
    # http.client.HTTPSConnection.debuglevel = 1
    # http.client.HTTPConnection.debuglevel = 1
    # Create HTTPS connection on tydom server
    conn = http.client.HTTPSConnection(host, 443, context=ssl_context)
    # Get first handshake
    conn.request("GET", "/mediation/client?mac={}&appli=1".format(mac), None, httpHeaders)
    res = conn.getresponse()
    # Get authentication
    nonce = res.headers["WWW-Authenticate"].split(',', 3)
    # read response
    res.read()
    # Close HTTPS Connection
    conn.close()
    # Build websocket headers
    websocketHeaders = {'Authorization': build_digest_headers(nonce)}
    if ssl_context is not None:
        websocket_ssl_context = ssl_context
    else:
        websocket_ssl_context = True # Verify certificate

    print('"Attempting websocket connection..."')
    ########## CONNECTION
    # websocket = await websockets.client.connect('wss://{}:443/mediation/client?mac={}&appli=1'.format(host, mac),
    #                                      extra_headers=websocketHeaders, ssl=websocket_ssl_context)

    async with websockets.client.connect('wss://{}:443/mediation/client?mac={}&appli=1'.format(host, mac),
                                         extra_headers=websocketHeaders, ssl=websocket_ssl_context) as websocket:

        tydom = websocket
        print("Tydom Websocket is Connected !", tydom)

        await mqttconnection(mqtt_host, mqtt_user, mqtt_pass)
        print("##################################")
        print('Requesting 1st data...')
        await get_data(tydom)
        while True:
            # await consumer(tydom) #Only receiving from socket in real time
            await handler(tydom) #If you want to send periodically something, disable consumer

# Main async task
async def main_task():
    print(str(datetime.fromtimestamp(time.time())))
    try:
        if (tydom == None) or not tydom.open:
            print("##################################")
            start = time.time()
            await websocket_connection()
            print('Connection total time :')
            end = time.time()
            print(end - start)

        else:
            print('Websocket is still opened ! requesting data...')
            await get_data(tydom)
    except Exception as e:
        print("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!")
        print('Connection total time :')
        end = time.time()
        print(end - start)
        print(str(datetime.fromtimestamp(time.time())))
        print(e)
        print('Something bad happened, reconnecting...')
        # await asyncio.sleep(8)
        await main_task()

if __name__ == '__main__':
    loop = asyncio.get_event_loop()

    loop.run_until_complete(main_task())
    loop.run_forever()

        # Get informations (not very useful)
        # await get_info(websocket)

        # Get all moments stored on Tydom
        # await get_moments(websocket)

        # Get scenarios ids
        # print("Get scenarii")
        # await get_scenarios(websocket)

        # Run scenario with scn id returned in previous command
        # await put_scenarios(websocket, 15)
        # print("Get names of all devices")
        # await get_configs_file(websocket)
        # # await get_devices_meta(websocket)

        # print("Get data of all devices")
        # await get_devices_data(websocket)

        # Get data of a specific device
        #await get_device_data(websocket, 9)

        # Set a shutter position to 10%
        #await put_devices_data(websocket, 9, "position", "10.0")
        # TODO : Wait hardcoded for now to put response from websocket server
        #time.sleep(45)

mrwiwi commented 4 years ago

Exemple of logs from the last version 👍

2019-12-11 21:59:42.310451
##################################
"Attempting websocket connection..."
Tydom Websocket is Connected ! <websockets.client.WebSocketClientProtocol object at 0x039BD928>
Attempting MQTT connection...
##################################
Subscribing to :  homeassistant/+/tydom/#
##################################
Requesting 1st data...
MQTT is connected and suscribed ! =)
MQTT set_position incoming :  homeassistant/cover/tydom/1544688209/set_position b'0'
Incoming message type : config detected
Configuration updated
GET response message processed !
GET response message processed !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
PUT message detected !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
PUT message processed !
MQTT set_position incoming :  homeassistant/cover/tydom/1544688209/set_position b'0'
GET response message processed !
PUT message detected !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688209
PUT message processed !
Incoming message type : config detected
Configuration updated
GET response message processed !
Websocket refreshed at  2019-12-11 22:00:39.164854
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
Incoming message type : config detected
Configuration updated
GET response message processed !
Websocket refreshed at  2019-12-11 22:01:29.154541
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
MQTT set_position incoming :  homeassistant/cover/tydom/1544688209/set_position b'7'
GET response message processed !
MQTT set_position incoming :  homeassistant/cover/tydom/1544688209/set_position b'12'
GET response message processed !
PUT message detected !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
PUT message processed !
PUT message detected !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688209
PUT message processed !
Incoming message type : config detected
Configuration updated
GET response message processed !
Websocket refreshed at  2019-12-11 22:02:19.163388
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
PUT message detected !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688209
PUT message processed !
MQTT set_position incoming :  homeassistant/cover/tydom/1544688209/set_position b'0'
GET response message processed !
PUT message detected !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688209
PUT message processed !
PUT message detected !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688209
PUT message processed !
MQTT set_position incoming :  homeassistant/cover/tydom/1544688209/set_position b'4'
GET response message processed !
Incoming message type : config detected
Configuration updated
GET response message processed !
PUT message detected !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688209
PUT message processed !
Websocket refreshed at  2019-12-11 22:03:09.157773
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
MQTT set_position incoming :  homeassistant/cover/tydom/1544688209/set_position b'0'
GET response message processed !
PUT message detected !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688209
PUT message processed !
Websocket refreshed at  2019-12-11 22:03:59.158697
Incoming message type : config detected
Configuration updated
GET response message processed !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
b'\x02\x1e\x01\x00\x02\x00P\x02\x00\x14\xb1\xcc\x80\x00\x00\x82\x00\x00\x03\x00\x08\x00\x04\x0f\x00\x00\x01\xfdb\xfb|'  
Incoming message type : config detected
Configuration updated
GET response message processed !
Websocket refreshed at  2019-12-11 22:04:49.159226
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
Incoming message type : config detected
Configuration updated
GET response message processed !
Websocket refreshed at  2019-12-11 22:05:39.163886
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
Incoming message type : config detected
Configuration updated
GET response message processed !
Websocket refreshed at  2019-12-11 22:06:29.167847
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
Websocket refreshed at  2019-12-11 22:07:19.162555
Incoming message type : config detected
Configuration updated
GET response message processed !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
Incoming message type : config detected
Configuration updated
GET response message processed !
Websocket refreshed at  2019-12-11 22:08:09.185200
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
Incoming message type : config detected
Configuration updated
GET response message processed !
Websocket refreshed at  2019-12-11 22:08:59.192184
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
PUT message detected !
Incoming message type : data detected
Alarm created / updated : alarm_tydom_1544791388
PUT message processed !
Incoming message type : config detected
Configuration updated
GET response message processed !
Websocket refreshed at  2019-12-11 22:09:49.187374
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
Incoming message type : config detected
Configuration updated
GET response message processed !
Websocket refreshed at  2019-12-11 22:10:39.207062
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
Incoming message type : config detected
Configuration updated
GET response message processed !
Websocket refreshed at  2019-12-11 22:11:29.209469
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
PUT message detected !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
PUT message processed !
PUT message detected !
Incoming message type : data detected
Alarm created / updated : alarm_tydom_1544791388
PUT message processed !
Websocket refreshed at  2019-12-11 22:12:19.202075
Incoming message type : config detected
Configuration updated
GET response message processed !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
PUT message detected !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688209
PUT message processed !
MQTT set_position incoming :  homeassistant/cover/tydom/1544688209/set_position b'0'
GET response message processed !
PUT message detected !
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688209
PUT message processed !
Incoming message type : config detected
Configuration updated
GET response message processed !
Websocket refreshed at  2019-12-11 22:13:09.208218
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !
Incoming message type : config detected
Configuration updated
GET response message processed !
Websocket refreshed at  2019-12-11 22:13:59.208682
Incoming message type : data detected
Cover created / updated : cover_tydom_1544688029
Cover created / updated : cover_tydom_1544688089
Cover created / updated : cover_tydom_1544688149
Cover created / updated : cover_tydom_1544688209
Cover created / updated : cover_tydom_1544688269
Cover created / updated : cover_tydom_1544688329
Cover created / updated : cover_tydom_1544688389
Alarm created / updated : alarm_tydom_1544791388
GET response message processed !

mrwiwi commented 4 years ago

For the mqtt topics, "homeassistant" is the default prefix for hassio, but we can make it a variable.

It post config json for auto discovery, as it's documented here 👍 https://www.home-assistant.io/docs/mqtt/discovery/

mrwiwi commented 4 years ago

In the connection handler (producer and consumer), the producer peridcally send a get_data to the websocket to avoid disconnection when remote, i didn't succeded yet in the refresh/all POST as documented here : https://github.com/mgcrea/node-tydom-client

cth35 commented 4 years ago

Ideally, it should also run on domoticz. gmqtt is not installed by default. I will have a look. Le mercredi 11 décembre 2019 à 21:44:51 UTC+1, WiwiWillou notifications@github.com a écrit :

For the mqtt topics, "homeassistant" is the default prefix for hassio, but we can make it a variable.

It post config json for auto discovery, as it's documented here 👍 https://www.home-assistant.io/docs/mqtt/discovery/

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

mrwiwi commented 4 years ago

The script just need to have the mqtt broker address, i run it from my hotel :)

I run hassio on docker on debian, so i will get it to run from debian. Probably the most client independant solution, we just need a broker (SSL via port 8883 in the script BTW)

mrwiwi commented 4 years ago

Does domoticz have auto discovery MQTT ?

cth35 commented 4 years ago

yes, I think domoticz uses MQTT in its framework Le mercredi 11 décembre 2019 à 21:50:41 UTC+1, WiwiWillou notifications@github.com a écrit :

Does domoticz have auto discovery MQTT ?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

mrwiwi commented 4 years ago

Fixed MQTT set position for covers !

async def on_message(client, topic, payload, qos, properties):
    # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
    # print('MQTT incoming : ', topic, payload)
    if (topic == "homeassistant/requests/tydom/update") or (payload == "please"):
        await get_data(tydom)
    if ('set_position' in str(topic)):
        print('MQTT set_position incoming : ', topic, payload)
        get_id = (topic.split("/"))[3] #extract id from mqtt
        # print(tydom, str(get_id), 'position', json.loads(payload))
        await put_devices_data(tydom, str(get_id), 'position', str(json.loads(payload)))
    else:
        return 0

mrwiwi commented 4 years ago

TODO (IMHO) :

Refresh POST type message
ignore if values are flaged as expired, then refresh !
Review of the code (i just learn python and programming for a few days...)
Test with others systems
Add more devices types
MQTT topics for scenes/scenarii (set topics)
Add attributes to alarm and covers (defects, etc.)

Thanks again :)

cth35 commented 4 years ago

Domoticz suggests to use paho-mqtt for python. For someone that learned python for a few days, you rock ;)

mrwiwi commented 4 years ago

Domoticz suggests to use paho-mqtt for python. For someone that learned python for a few days, you rock ;)

Thanks that's very kind !

paho isn't async if i understand well, gmqtt works really well.

You launch the script from domoticz or by its side ?

Topics are topics, any library should work with any broker no ?

I've created a HA topic to recruit some more people, here : https://community.home-assistant.io/t/tydom2mqtt-make-delta-dore-devices-home-assistant-compatible/154439

It points to your github obviously !

mrwiwi commented 4 years ago

I have 1006 connection closed codes when on local, not an issue on remote !

cth35 commented 4 years ago

Yes I've noticed that also.

cth35 commented 4 years ago

DEBUG:websockets.protocol:client - received unsolicited pong: [empty] DEBUG:websockets.protocol:client ! timed out waiting for pong DEBUG:websockets.protocol:client ! failing WebSocket connection in the OPEN state: 1011 [no reason] DEBUG:websockets.protocol:client - state = CLOSING

mrwiwi commented 4 years ago

DEBUG:websockets.protocol:client - received unsolicited pong: [empty] DEBUG:websockets.protocol:client ! timed out waiting for pong DEBUG:websockets.protocol:client ! failing WebSocket connection in the OPEN state: 1011 [no reason] DEBUG:websockets.protocol:client - state = CLOSING

I didn't change the connection logic did i ?

cth35 commented 4 years ago

No it seems to be inherent to websockets logic. Adding ping_interval=None to the websocket client function arguments seems to solve the problem. But it still disconnect after a while with error 1000

mrwiwi commented 4 years ago

Consumer task has crashed !
'NoneType' object has no attribute 'is_closing'
Restarting..............
2019-12-11 23:30:50.868462
Websocket is still opened ! requesting data...
DEBUG:websockets.protocol:client > Frame(fin=True, opcode=2, data=b'\x02GET /configs/file HTTP/1.1\r\nContent-Length: 0\r\nContent-Type: application/json; charset=UTF-8\r\nTransac-Id: 0\r\n\r\n', rsv1=False, rsv2=False, rsv3=False)
DEBUG:websockets.protocol:client > Frame(fin=True, opcode=9, data=b'\xb8-\xd0\x05', rsv1=False, rsv2=False, rsv3=False)
DEBUG:websockets.protocol:client - event = data_received(<2 bytes>)
DEBUG:websockets.protocol:client - event = data_received(<4 bytes>)
DEBUG:websockets.protocol:client < Frame(fin=True, opcode=10, data=b'\xb8-\xd0\x05', rsv1=False, rsv2=False, rsv3=False)
DEBUG:websockets.protocol:client - received solicited pong: b82dd005
DEBUG:websockets.protocol:client - event = data_received(<71 bytes>)

DEBUG:websockets.protocol:client > Frame(fin=True, opcode=2, data=b'\x02GET /devices/data HTTP/1.1\r\nContent-Length: 0\r\nContent-Type: application/json; charset=UTF-8\r\nTransac-Id: 0\r\n\r\n', rsv1=False, rsv2=False, rsv3=False)

is the culpirt for me, still work to do on consumer side :)

cth35 commented 4 years ago

async with websockets.client.connect('wss://{}:443/mediation/client?mac={}&appli=1'.format(host, mac), extra_headers=websocketHeaders, ssl=websocket_ssl_context, ping_interval=None) as websocket:

mrwiwi commented 4 years ago

Fixed refresh/all, now pinging with it, added some MQTT topics (refresh,secenarii,etc.) Reworked consumer again Added pusblish of error reason to MQTT MQTT server connection before trying websocket, to publish errors Added fatal error Added mechanism for only one instance of the script (max clients is before10 by experience).

#!/usr/bin/env python
import asyncio
import websockets
import http.client
from requests.auth import HTTPDigestAuth
import sys
import logging
from http.client import HTTPResponse
from io import BytesIO
import urllib3
import json
import os
import base64
import time
from http.server import BaseHTTPRequestHandler
import ssl
from datetime import datetime
from gmqtt import Client as MQTTClient

from tendo import singleton
me = singleton.SingleInstance() # will sys.exit(-1) if other instance is running

import socket    
hostname = socket.gethostname()    
IPAddr = socket.gethostbyname(hostname) 

local = False
if (IPAddr == '192.168.0.y'): #Local IP adress of your host machine
    print('Script exec locally')
    local = True
else:
    print('Script exec remotly')
    local = False

# os.open("lock", os.O_CREAT|os.O_EXCL)
####### CREDENTIALS
mac = "" #MAC Address of Tydom Box
login = mac
password = "" #Tydom password
mqtt_user = ''
mqtt_pass = ''

if (local == False):
    host = "mediation.tydom.com" #"192.168.0.6"  #"192.168.0.20" # Local ip address or mediation.tydom.com for remote connexion
    mqtt_host = ''
else:
    host = "192.168.0.x" #  #"192.168.0.20" # Local ip address or mediation.tydom.com for remote connexion
    mqtt_host = 'localhost'

mqtt_port = 8883
mqtt_ssl = True
#INIT Servers
hassio = None
tydom = None

# Globals
####################################### MQTT
tydom_topic = "homeassistant/+/tydom/#"

cover_config_topic = "homeassistant/cover/tydom/{id}/config"
cover_config = "homeassistant/cover/tydom/{id}/config"
cover_position_topic = "homeassistant/cover/tydom/{id}/current_position"
cover_set_postion_topic = "homeassistant/cover/tydom/{id}/set_position"
cover_attributes_topic = "homeassistant/cover/tydom/{id}/attributes"

alarm_topic = "homeassistant/alarm_control_panel/tydom/#"
alarm_config = "homeassistant/alarm_control_panel/tydom/{id}/config"
alarm_state_topic = "homeassistant/alarm_control_panel/tydom/{id}/state"
alarm_command_topic = "homeassistant/alarm_control_panel/tydom/{id}/set"
alarm_sos_topic = "homeassistant/binary_sensor/tydom/{id}/sos"
alarm_attributes_topic = "homeassistant/alarm_control_panel/tydom/{id}/attributes"

refresh_topic = "homeassistant/requests/tydom/refresh"

# Set Host, ssl context and prefix for remote or local connection
if host == "mediation.tydom.com":
    remote_mode = True
    ssl_context = None
    cmd_prefix = "\x02"
else:
    remote_mode = False
    ssl_context = ssl._create_unverified_context()
    cmd_prefix = ""

deviceAlarmKeywords = ['alarmMode','alarmState','alarmSOS','zone1State','zone2State','zone3State','zone4State','zone5State','zone6State','zone7State','zone8State','gsmLevel','inactiveProduct','zone1State','liveCheckRunning','networkDefect','unitAutoProtect','unitBatteryDefect','unackedEvent','alarmTechnical','systAutoProtect','sysBatteryDefect','zsystSupervisionDefect','systOpenIssue','systTechnicalDefect','videoLinkDefect']
# Device dict for parsing
device_dict = dict()

#MQTT
STOP = asyncio.Event()
def on_connect(client, flags, rc, properties):
    print("##################################")

    print("Subscribing to : ", tydom_topic)
    # client.subscribe('homeassistant/#', qos=0)
    client.subscribe(tydom_topic, qos=0)

async def on_message(client, topic, payload, qos, properties):
    # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
    # print('MQTT incoming : ', topic, payload)
    if tydom:
        if (topic == "homeassistant/requests/tydom/update"):
            await get_data(tydom)
        if (topic == "homeassistant/requests/tydom/refresh"):
            await post_refresh(tydom)
        if (topic == "homeassistant/requests/tydom/scenarii"):
            await get_scenarii(tydom)
        if ('set_position' in str(topic)):
            print('Incoming MQTT set_position request : ', topic, payload)
            get_id = (topic.split("/"))[3] #extract id from mqtt
            # print(tydom, str(get_id), 'position', json.loads(payload))
            await put_devices_data(tydom, str(get_id), 'position', str(json.loads(payload)))
        else:
            return 0
    else:
        print("No websocket connection yet !")

def on_disconnect(client, packet, exc=None):
    print('MQTT Disconnected')
    print("##################################")

def on_subscribe(client, mid, qos):
    print("MQTT is connected and suscribed ! =)")

def ask_exit(*args):
    STOP.set()

async def mqttconnection(broker_host, user, password):
    try:
        global hassio
        if (hassio == None):
            print('Attempting MQTT connection...')
            client = MQTTClient("client-id")

            client.on_connect = on_connect
            client.on_message = on_message
            client.on_disconnect = on_disconnect
            client.on_subscribe = on_subscribe

            client.set_auth_credentials(user, password)
            await client.connect(broker_host, port=mqtt_port, ssl=mqtt_ssl)
            hassio = client
    except:
        print('MQTT error, restarting...')
        await main_task()
    # client.publish('TEST/TIME', str(time.time()), qos=1)

    # await STOP.wait()
    # await client.disconnect()

#######" END MQTT"

class Cover:
    def __init__(self, id, name, current_position=None, set_position=None, attributes=None):

        self.id = id
        self.name = name
        self.current_position = current_position
        self.set_position = set_position
        self.attributes = attributes

    def id(self):
        return self.id

    def name(self):
        return self.name

    def current_position(self):
        return self.current_position

    def set_position(self):
        return self.set_position

    def attributes(self):
        return self.attributes

    # cover_config_topic = "homeassistant/cover/tydom/{id}/config"
    # cover_position_topic = "homeassistant/cover/tydom/{id}/current_position"
    # cover_set_postion_topic = "homeassistant/cover/tydom/{id}/set_position"
    # cover_attributes_topic = "homeassistant/cover/tydom/{id}/attributes"

    def setup(self):
        self.device = {}
        self.device['manufacturer'] = 'Delta Dore'
        self.device['model'] = 'Volet'
        self.device['name'] = self.name
        self.device['identifiers'] = id=self.id
        self.config_topic = cover_config_topic.format(id=self.id)
        self.config = {}
        self.config['name'] = self.name
        self.config['unique_id'] = self.id
        # self.config['attributes'] = self.attributes
        self.config['command_topic'] = cover_set_postion_topic.format(id=self.id)
        self.config['set_position_topic'] = cover_set_postion_topic.format(id=self.id)
        self.config['position_topic'] = cover_position_topic.format(id=self.id)
        self.config['payload_open'] = 100
        self.config['payload_close'] = 0
        self.config['retain'] = 'false'
        self.config['device'] = self.device

        # print(self.config)
        hassio.publish(self.config_topic, json.dumps(self.config), qos=0)

    def update(self):
        self.setup()
        self.position_topic = cover_position_topic.format(id=self.id, current_position=self.current_position)
        hassio.publish(self.position_topic, self.current_position, qos=0, retain=True)

        # self.attributes_topic = cover_attributes_topic.format(id=self.id, attributes=self.attributes)
        # hassio.publish(self.attributes_topic, self.attributes, qos=0)

class Alarm:
    def __init__(self, id, name, current_state=None, attributes=None):
        self.id = id
        self.name = name
        self.current_state = current_state
        self.attributes = attributes

    # def id(self):
    #     return id

    # def name(self):
    #     return name

    # def current_state(self):
    #     return current_state

    # def attributes(self):
    #     return attributes

    def setup(self):
        self.device = {}
        self.device['manufacturer'] = 'Delta Dore'
        self.device['model'] = 'Tyxal'
        self.device['name'] = self.name
        self.device['identifiers'] = id=self.id
        self.config_alarm = alarm_config.format(id=self.id)
        self.config = {}
        self.config['name'] = self.name
        self.config['unique_id'] = self.id
        self.config['device'] = self.device
        # self.config['attributes'] = self.attributes
        self.config['command_topic'] = alarm_command_topic.format(id=self.id)
        self.config['state_topic'] = alarm_state_topic.format(id=self.id)

        # print(self.config)
        hassio.publish(self.config_alarm, json.dumps(self.config), qos=0)

    def update(self):
        self.setup()
        self.state_topic = alarm_state_topic.format(id=self.id, state=self.current_state)
        hassio.publish(self.state_topic, self.current_state, qos=0, retain=True)

        # self.attributes_topic = alarm_attributes_topic.format(id=self.id, attributes=self.attributes)
        # hassio.publish(self.attributes_topic, self.attributes, qos=0)

    # alarm_topic = "homeassistant/alarm_control_panel/tydom/#"
    # alarm_config = "homeassistant/alarm_control_panel/tydom/{id}/config"
    # alarm_state_topic = "homeassistant/alarm_control_panel/tydom/{id}/state"
    # alarm_sos_topic = "homeassistant/binary_sensor/tydom/{id}/sos"
    # alarm_attributes_topic = "homeassistant/alarm_control_panel/tydom/{id}/attributes"

class BytesIOSocket:
    def __init__(self, content):
        self.handle = BytesIO(content)

    def makefile(self, mode):
        return self.handle

class HTTPRequest(BaseHTTPRequestHandler):
    def __init__(self, request_text):
        #self.rfile = StringIO(request_text)
        self.raw_requestline = request_text
        self.error_code = self.error_message = None
        self.parse_request()

    def send_error(self, code, message):
        self.error_code = code
        self.error_message = message

def response_from_bytes(data):
    sock = BytesIOSocket(data)
    response = HTTPResponse(sock)
    response.begin()
    return urllib3.HTTPResponse.from_httplib(response)

def put_response_from_bytes(data):
    request = HTTPRequest(data)
    return request

# Get pretty name for a device id
def get_name_from_id(id):
    name = ""
    if len(device_dict) != 0:
        name = device_dict[id]
    return(name)

# Basic response parsing. Typically GET responses
async def parse_response(incoming):
    data = incoming
    msg_type = None
    first = str(data[:20])

    # Detect type of incoming data
    if (data != ''):
        if ("id" in first):
            print('Incoming message type : data detected')
            msg_type = 'msg_data'
        elif ("date" in first):
            print('Incoming message type : config detected')
            msg_type = 'msg_config'
        elif ("doctype" in first):
            print('Incoming message type : html detected (probable pong)')
            msg_type = 'msg_html'
            print(data)
        else:
            print('Incoming message type : no type detected')
            print(first)

        if not (msg_type == None):
            try:
                parsed = json.loads(data)
                # print(parsed)
                if (msg_type == 'msg_config'):
                    for i in parsed["endpoints"]:
                        # Get list of shutter
                        if i["last_usage"] == 'shutter':
                            # print('{} {}'.format(i["id_endpoint"],i["name"]))
                            device_dict[i["id_endpoint"]] = i["name"]

                            # TODO get other device type
                        if i["last_usage"] == 'alarm':
                            # print('{} {}'.format(i["id_endpoint"], i["name"]))
                            device_dict[i["id_endpoint"]] = "Tyxal Alarm"
                    print('Configuration updated')
                elif (msg_type == 'msg_data'):
                    for i in parsed:
                        attr = {}
                        if i["endpoints"][0]["error"] == 0:
                            for elem in i["endpoints"][0]["data"]:
                                # Get full name of this id
                                endpoint_id = i["endpoints"][0]["id"]
                                # Element name
                                elementName = elem["name"]
                                # Element value
                                elementValue = elem["value"]

                                # Get last known position (for shutter)
                                if elementName == 'position':
                                    name_of_id = get_name_from_id(endpoint_id)
                                    if len(name_of_id) != 0:
                                        print_id = name_of_id
                                    else:
                                        print_id = endpoint_id
                                    # print('{} : {}'.format(print_id, elementValue))
                                    new_cover = "cover_tydom_"+str(endpoint_id)
                                    print("Cover created / updated : "+new_cover)
                                    new_cover = Cover(id=endpoint_id,name=print_id, current_position=elementValue, attributes=i)
                                    new_cover.update()

                                # Get last known position (for alarm)
                                if elementName in deviceAlarmKeywords:
                                    alarm_data = '{} : {}'.format(elementName, elementValue)
                                    # print(alarm_data)
                                    # alarmMode  : ON or ZONE or OFF
                                    # alarmState : ON = Triggered
                                    # alarmSOS   : true = SOS triggered
                                    state = None
                                    sos = False

                                    if alarm_data == "alarmMode : ON":
                                        state = "armed_away"
                                    elif alarm_data == "alarmMode : ZONE":
                                        state = "armed_home"
                                    elif alarm_data == "alarmMode : OFF":
                                        state = "disarmed"
                                    elif alarm_data == "alarmState : ON":
                                        state = "triggered"
                                    elif alarm_data == "alarmSOS : true":
                                        sos = True
                                    else:
                                        attr[elementName] = [elementValue]
                                    #     attr[alarm_data]
                                        # print(attr)
                                    #device_dict[i["id_endpoint"]] = i["name"]
                                    if (sos == True):
                                        print("SOS !")
                                    if not (state == None):
                                        # print(state)
                                        alarm = "alarm_tydom_"+str(endpoint_id)
                                        print("Alarm created / updated : "+alarm)
                                        alarm = Alarm(id=endpoint_id,name="Tyxal Alarm", current_state=state, attributes=attr)
                                        alarm.update()
                elif (msg_type == 'msg_html'):
                    print("pong")
                else:
                    # Default json dump
                    print()
                    print(json.dumps(parsed, sort_keys=True, indent=4, separators=(',', ': ')))
            except Exception as e:
                print('Cannot parse response !')
                # print('Response :')
                # print(data)
                if (e != 'Expecting value: line 1 column 1 (char 0)'):
                    print(e)

# PUT response DIRTY parsing
def parse_put_response(bytes_str):
    # TODO : Find a cooler way to parse nicely the PUT HTTP response
    resp = bytes_str[len(cmd_prefix):].decode("utf-8")
    fields = resp.split("\r\n")
    fields = fields[6:]  # ignore the PUT / HTTP/1.1
    end_parsing = False
    i = 0
    output = str()
    while not end_parsing:
        field = fields[i]
        if len(field) == 0 or field == '0':
            end_parsing = True
        else:
            output += field
            i = i + 2
    parsed = json.loads(output)
    return json.dumps(parsed)
    # print(json.dumps(parsed, sort_keys=True, indent=4, separators=(',', ': ')))

# Generate 16 bytes random key for Sec-WebSocket-Keyand convert it to base64
def generate_random_key():
    return base64.b64encode(os.urandom(16))

# Build the headers of Digest Authentication
def build_digest_headers(nonce):
    digestAuth = HTTPDigestAuth(login, password)
    chal = dict()
    chal["nonce"] = nonce[2].split('=', 1)[1].split('"')[1]
    chal["realm"] = "ServiceMedia" if remote_mode is True else "protected area"
    chal["qop"] = "auth"
    digestAuth._thread_local.chal = chal
    digestAuth._thread_local.last_nonce = nonce
    digestAuth._thread_local.nonce_count = 1
    return digestAuth.build_digest_header('GET', "https://{}:443/mediation/client?mac={}&appli=1".format(host, mac))

# Send Generic GET message
async def send_message(websocket, msg):
    str = cmd_prefix + "GET " + msg +" HTTP/1.1\r\nContent-Length: 0\r\nContent-Type: application/json; charset=UTF-8\r\nTransac-Id: 0\r\n\r\n"
    a_bytes = bytes(str, "ascii")
    await websocket.send(a_bytes)
    return 0
    # return await websocket.recv() #disable if handler

# Send Generic POST message
async def send_post_message(websocket, msg):
    str = cmd_prefix + "POST " + msg +" HTTP/1.1\r\nContent-Length: 0\r\nContent-Type: application/json; charset=UTF-8\r\nTransac-Id: 0\r\n\r\n"
    a_bytes = bytes(str, "ascii")
    await websocket.send(a_bytes)
    return 0
    # return await websocket.recv()

###############################################################
# Commands                                                    #
###############################################################

# Get some information on Tydom
async def get_info(websocket):
    msg_type = '/info'
    parse_response(await send_message(websocket, msg_type), msg_type)

# Refresh (all)
async def post_refresh(websocket):
    print("Refresh....")
    msg_type = '/refresh/all'
    await send_post_message(websocket, msg_type)

# Get the moments (programs)
async def get_moments(websocket):
    msg_type = '/moments/file'
    await send_message(websocket, msg_type)

# Get the scenarios
async def get_scenarii(websocket):
    msg_type = '/scenarios/file'
    await send_message(websocket, msg_type)

# Get a ping (pong should be returned)
async def get_ping(websocket):
    msg_type = 'ping'
    await send_message(websocket, msg_type)

# Get all devices metadata
async def get_devices_meta(websocket):
    msg_type = '/devices/meta'
    parse_response(await send_message(websocket, msg_type), msg_type)

# Get all devices data
async def get_devices_data(websocket):
    msg_type = '/devices/data'
    await send_message(websocket, msg_type)

# List the device to get the endpoint id
async def get_configs_file(websocket):
    msg_type = '/configs/file'
    await send_message(websocket, msg_type)

async def get_data(websocket):
    await get_configs_file(websocket)
    await asyncio.sleep(2)
    await get_devices_data(websocket)

# Give order (name + value) to endpoint
async def put_devices_data(websocket, endpoint_id, name, value):
    # For shutter, value is the percentage of closing
    body="[{\"name\":\"" + name + "\",\"value\":\""+ value + "\"}]"
    # endpoint_id is the endpoint = the device (shutter in this case) to open.
    str_request = cmd_prefix + "PUT /devices/{}/endpoints/{}/data HTTP/1.1\r\nContent-Length: ".format(str(endpoint_id),str(endpoint_id))+str(len(body))+"\r\nContent-Type: application/json; charset=UTF-8\r\nTransac-Id: 0\r\n\r\n"+body+"\r\n\r\n"
    a_bytes = bytes(str_request, "ascii")
    await websocket.send(a_bytes)
    # name = await websocket.recv()
    # parse_response(name)
    # name = await websocket.recv()
    # try:
    #     parse_response(name)
    # except:
    #     parse_put_response(name)

# Run scenario
async def put_scenarios(websocket, scenario_id):
    body=""
    # scenario_id is the id of scenario got from the get_scenarios command
    str_request = cmd_prefix + "PUT /scenarios/{} HTTP/1.1\r\nContent-Length: ".format(str(scenario_id))+str(len(body))+"\r\nContent-Type: application/json; charset=UTF-8\r\nTransac-Id: 0\r\n\r\n"+body+"\r\n\r\n"
    a_bytes = bytes(str_request, "ascii")
    await websocket.send(a_bytes)
    # name = await websocket.recv()
    # parse_response(name)

# Give order to endpoint
async def get_device_data(websocket, id):
    # 10 here is the endpoint = the device (shutter in this case) to open.
    str_request = cmd_prefix + "GET /devices/{}/endpoints/{}/data HTTP/1.1\r\nContent-Length: 0\r\nContent-Type: application/json; charset=UTF-8\r\nTransac-Id: 0\r\n\r\n".format(str(id),str(id))
    a_bytes = bytes(str_request, "ascii")
    await websocket.send(a_bytes)
    # name = await websocket.recv()
    # parse_response(name)

######## Messages Logic

async def consumer(websocket):
    # Receiver
    while True:
        bytes_str = await websocket.recv()
        print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
        # print(bytes_str)
        first = str(bytes_str[:40]) # Scanning 1st characters
        try:
            if ("refresh" in first):
                print('OK refresh message detected !')
                try:

                    hassio.publish('homeassistant/sensor/tydom/last_update', str(datetime.fromtimestamp(time.time())), qos=1, retain=True)
                except:
                    print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                    print('RAW INCOMING :')
                    print(bytes_str)
                    print('END RAW')
                    print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
            if ("PUT /devices/data" in first):
                print('PUT /devices/data message detected !')
                try:
                    incoming = parse_put_response(bytes_str)
                    # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                    await parse_response(incoming)
                    print('PUT message processed !')
                    print("##################################")
                    hassio.publish('homeassistant/sensor/tydom/last_update', str(datetime.fromtimestamp(time.time())), qos=1, retain=True)
                except:
                    print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                    print('RAW INCOMING :')
                    print(bytes_str)
                    print('END RAW')
                    print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")

            elif ("POST" in first):

                try:
                    # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                    incoming = parse_put_response(bytes_str)
                    await parse_response(incoming)
                    print('POST message processed !')
                    print("##################################")
                except:
                    print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                    print('RAW INCOMING :')
                    print(bytes_str)
                    print('END RAW')
                    print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")

            elif ("HTTP/1.1" in first): #(bytes_str != 0) and 
                response = response_from_bytes(bytes_str[len(cmd_prefix):])
                incoming = response.data.decode("utf-8")
                # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                # print(incoming)
                # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                try:
                    # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                    await parse_response(incoming)
                    print('Common / GET response message processed !')
                    print("##################################")
                    hassio.publish('homeassistant/sensor/tydom/last_update', str(datetime.fromtimestamp(time.time())), qos=1, retain=True)
                except:
                    print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                    print('RAW INCOMING :')
                    print(bytes_str)
                    print('END RAW')
                    print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                    # await parse_put_response(incoming)
            else:
                print("Didn't detect incoming type, here it is :")
                print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                print('RAW INCOMING :')
                print(bytes_str)
                print('END RAW')
                print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
        except Exception as e:
            print('Consumer has crashed !')    
            print(e)
            hassio.publish('homeassistant/sensor/tydom/last_crash', e+' '+str(datetime.fromtimestamp(time.time())), qos=1, retain=True)
            print('Restarting..............')
            await main_task()

async def producer(websocket):
        # while True:
    await asyncio.sleep(48)
    # await get_ping(websocket)
    await post_refresh(tydom)
    # await get_data(tydom)
    print("Websocket refreshed at ", str(datetime.fromtimestamp(time.time())))

async def consumer_handler(websocket):
    while True:
        try:
            await consumer(websocket)
        except Exception as e:
            print("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!")
            print("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!")
            print("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!")
            print('Consumer task has crashed !')
            print(e)
            hassio.publish('homeassistant/sensor/tydom/last_crash', e+' '+str(datetime.fromtimestamp(time.time())), qos=1, retain=True)
            print('Restarting..............')
            await main_task()

async def producer_handler(websocket):
    while True:
        await producer(websocket)

######## HANDLER
async def handler(websocket):
    try:
        # print("Starting handlers...")
        consumer_task = asyncio.ensure_future(
            consumer_handler(websocket))
        producer_task = asyncio.ensure_future(
            producer_handler(websocket))
        done, pending = await asyncio.wait(
            [consumer_task, producer_task],
            return_when=asyncio.FIRST_COMPLETED,
        )
        for task in pending:
            task.cancel()

    except Exception as e:
        print("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!")
        print(e)
        print('Handler crashed.')

async def websocket_connection():
    global tydom
    # logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)
    httpHeaders =  {"Connection": "Upgrade",
                    "Upgrade": "websocket",
                    "Host": host + ":443",
                    "Accept": "*/*",
                    "Sec-WebSocket-Key": generate_random_key(),
                    "Sec-WebSocket-Version": "13"
                    }
    # http.client.HTTPSConnection.debuglevel = 1
    # http.client.HTTPConnection.debuglevel = 1
    # Create HTTPS connection on tydom server
    conn = http.client.HTTPSConnection(host, 443, context=ssl_context)
    # Get first handshake
    conn.request("GET", "/mediation/client?mac={}&appli=1".format(mac), None, httpHeaders)
    res = conn.getresponse()
    # Get authentication
    nonce = res.headers["WWW-Authenticate"].split(',', 3)
    # read response
    res.read()
    # Close HTTPS Connection
    conn.close()
    # Build websocket headers
    websocketHeaders = {'Authorization': build_digest_headers(nonce)}
    if ssl_context is not None:
        websocket_ssl_context = ssl_context
    else:
        websocket_ssl_context = True # Verify certificate

    print('"Attempting websocket connection..."')
    ########## CONNECTION
    # websocket = await websockets.client.connect('wss://{}:443/mediation/client?mac={}&appli=1'.format(host, mac),
    #                                      extra_headers=websocketHeaders, ssl=websocket_ssl_context)

    async with websockets.client.connect('wss://{}:443/mediation/client?mac={}&appli=1'.format(host, mac),
                                         extra_headers=websocketHeaders, ssl=websocket_ssl_context) as websocket:

        tydom = websocket
        print("Tydom Websocket is Connected !", tydom)
        print("##################################")
        print('Requesting 1st data...')
        await post_refresh(tydom)
        await get_data(tydom)
        while True:
            # await consumer(tydom) #Only receiving from socket in real time
            await handler(tydom) #If you want to send periodically something, disable consumer

# Main async task
async def main_task():
    print(str(datetime.fromtimestamp(time.time())))
    try:
        if (tydom == None) or not tydom.open:
            print("##################################")
            start = time.time()
            await mqttconnection(mqtt_host, mqtt_user, mqtt_pass)
            await websocket_connection()
            print('Connection total time :')
            end = time.time()
            print(end - start)

        else:
            print('Websocket is still opened ! requesting data...')
            await get_data(tydom)
    except Exception as e:
        print("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!")
        print('Connection total time :')
        end = time.time()
        print(end - start)
        print(str(datetime.fromtimestamp(time.time())))
        print(e)
        hassio.publish('homeassistant/sensor/tydom/last_crash', e+' '+str(datetime.fromtimestamp(time.time())), qos=1, retain=True)
        print('Something bad happened, reconnecting...')
        # await asyncio.sleep(8)
        await main_task()

if __name__ == '__main__':
    try:
        loop = asyncio.get_event_loop()

        loop.run_until_complete(main_task())
        loop.run_forever()
    except Exception as e:
        print('FATAL ERROR !')
        print(e)
        hassio.publish('homeassistant/sensor/tydom/last_crash', e+' '+str(datetime.fromtimestamp(time.time())), qos=1, retain=True)
        sys.exit(-1)

        # Get informations (not very useful)
        # await get_info(websocket)

        # Get all moments stored on Tydom
        # await get_moments(websocket)

        # Get scenarios ids
        # print("Get scenarii")
        # await get_scenarios(websocket)

        # Run scenario with scn id returned in previous command
        # await put_scenarios(websocket, 15)
        # print("Get names of all devices")
        # await get_configs_file(websocket)
        # # await get_devices_meta(websocket)

        # print("Get data of all devices")
        # await get_devices_data(websocket)

        # Get data of a specific device
        #await get_device_data(websocket, 9)

        # Set a shutter position to 10%
        #await put_devices_data(websocket, 9, "position", "10.0")
        # TODO : Wait hardcoded for now to put response from websocket server
        #time.sleep(45)

mrwiwi commented 4 years ago

Lastest update, no more error but the one when local mode, with 1006 error code.

Rock solid for hours now :)

#!/usr/bin/env python
import asyncio
import websockets
import http.client
from requests.auth import HTTPDigestAuth
import sys
import logging
from http.client import HTTPResponse
from io import BytesIO
import urllib3
import json
import os
import base64
import time
from http.server import BaseHTTPRequestHandler
import ssl
from datetime import datetime
from gmqtt import Client as MQTTClient

from tendo import singleton
me = singleton.SingleInstance() # will sys.exit(-1) if other instance is running

# logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)

# http.client.HTTPSConnection.debuglevel = 1
# http.client.HTTPConnection.debuglevel = 1

import socket    
hostname = socket.gethostname()    
IPAddr = socket.gethostbyname(hostname)
print (hostname)
print(IPAddr)

local = False
if (hostname == ''): #Local IP adress of your host machine

    local = True
else:

    local = False

#Force local
local = True
print('Execution type forced')
print('Local Execution Detected') if (local == True) else print('Remote Execution Detected')
####### CREDENTIALS
mac = "" #MAC Address of Tydom Box
login = mac
password = "" #Tydom password
mqtt_user = ''
mqtt_pass = ''

if (local == True):
    host = "mediation.tydom.com"  #"192.168.0.6" # Local ip address or mediation.tydom.com for remote connexion
    mqtt_host = "localhost"
    mqtt_port = 1883
    mqtt_ssl = False

else:
    host = "mediation.tydom.com" #mediation.tydom.com for remote connexion
    mqtt_host = 'wwwwwwww'
    mqtt_port = 8883
    mqtt_ssl = True

#INIT Servers
hassio = None
tydom = None

# Set Host, ssl context and prefix for remote or local connection
if host == "mediation.tydom.com":

    remote_mode = True
    ssl_context = None
    cmd_prefix = "\x02"
else:

    remote_mode = False
    ssl_context = ssl._create_unverified_context()
    cmd_prefix = ""

deviceAlarmKeywords = ['alarmMode','alarmState','alarmSOS','zone1State','zone2State','zone3State','zone4State','zone5State','zone6State','zone7State','zone8State','gsmLevel','inactiveProduct','zone1State','liveCheckRunning','networkDefect','unitAutoProtect','unitBatteryDefect','unackedEvent','alarmTechnical','systAutoProtect','sysBatteryDefect','zsystSupervisionDefect','systOpenIssue','systTechnicalDefect','videoLinkDefect']
# Device dict for parsing
device_dict = dict()

# Globals
####################################### MQTT
tydom_topic = "homeassistant/+/tydom/#"

cover_config_topic = "homeassistant/cover/tydom/{id}/config"
cover_config = "homeassistant/cover/tydom/{id}/config"
cover_position_topic = "homeassistant/cover/tydom/{id}/current_position"
cover_set_postion_topic = "homeassistant/cover/tydom/{id}/set_position"
cover_attributes_topic = "homeassistant/cover/tydom/{id}/attributes"

alarm_topic = "homeassistant/alarm_control_panel/tydom/#"
alarm_config = "homeassistant/alarm_control_panel/tydom/{id}/config"
alarm_state_topic = "homeassistant/alarm_control_panel/tydom/{id}/state"
alarm_command_topic = "homeassistant/alarm_control_panel/tydom/{id}/set"
alarm_sos_topic = "homeassistant/binary_sensor/tydom/{id}/sos"
alarm_attributes_topic = "homeassistant/alarm_control_panel/tydom/{id}/attributes"

refresh_topic = "homeassistant/requests/tydom/refresh"
#MQTT
STOP = asyncio.Event()
def on_connect(client, flags, rc, properties):
    print("##################################")
    try:
        print("Subscribing to : ", tydom_topic)
        # client.subscribe('homeassistant/#', qos=0)
        client.subscribe(tydom_topic, qos=0)
    except Exception as e:
        print(e)
async def on_message(client, topic, payload, qos, properties):
    # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
    # print('MQTT incoming : ', topic, payload)
    if tydom:
        if (topic == "homeassistant/requests/tydom/update"):
            print('Incoming MQTT update request : ', topic, payload)
            await get_data(tydom)
        if (topic == "homeassistant/requests/tydom/refresh"):
            print('Incoming MQTT refresh request : ', topic, payload)
            await post_refresh(tydom)
        if (topic == "homeassistant/requests/tydom/scenarii"):
            print('Incoming MQTT scenarii request : ', topic, payload)
            await get_scenarii(tydom)
        if ('set_position' in str(topic)):
            print('Incoming MQTT set_position request : ', topic, payload)
            get_id = (topic.split("/"))[3] #extract id from mqtt
            # print(tydom, str(get_id), 'position', json.loads(payload))
            await put_devices_data(tydom, str(get_id), 'position', str(json.loads(payload)))

        else:
            return 0
    else:
        print("No websocket connection yet !")

def on_disconnect(client, packet, exc=None):
    print('MQTT Disconnected')
    print("##################################")
    mqttconnection(mqtt_host, mqtt_user, mqtt_pass)    

def on_subscribe(client, mid, qos):
    print("MQTT is connected and suscribed ! =)", client)
    pyld = 'Started !',str(datetime.fromtimestamp(time.time()))
    hassio.publish('homeassistant/sensor/tydom/last_clean_startup', pyld, qos=1, retain=True)

def ask_exit(*args):
    STOP.set()

async def mqttconnection(broker_host, user, password):

    try:
        global hassio
        if (hassio == None):
            print('Attempting MQTT connection...')
            client = MQTTClient("client-id")

            client.on_connect = on_connect
            client.on_message = on_message
            client.on_disconnect = on_disconnect
            client.on_subscribe = on_subscribe

            client.set_auth_credentials(user, password)
            await client.connect(broker_host, port=mqtt_port, ssl=mqtt_ssl)
            hassio = client
            hassio.publish('homeassistant/sensor/tydom/last_crash', '', qos=1, retain=True)
    except Exception as e:
        print(e)
        print('MQTT error, restarting...')
        await asyncio.sleep(8)
        await mqttconnection(mqtt_host, mqtt_user, mqtt_pass)

# client.publish('TEST/TIME', str(time.time()), qos=1)

# await STOP.wait()
    # await client.disconnect()

#######" END MQTT"

class Cover:
    def __init__(self, id, name, current_position=None, set_position=None, attributes=None):

        self.id = id
        self.name = name
        self.current_position = current_position
        self.set_position = set_position
        self.attributes = attributes

    def id(self):
        return self.id

    def name(self):
        return self.name

    def current_position(self):
        return self.current_position

    def set_position(self):
        return self.set_position

    def attributes(self):
        return self.attributes

    # cover_config_topic = "homeassistant/cover/tydom/{id}/config"
    # cover_position_topic = "homeassistant/cover/tydom/{id}/current_position"
    # cover_set_postion_topic = "homeassistant/cover/tydom/{id}/set_position"
    # cover_attributes_topic = "homeassistant/cover/tydom/{id}/attributes"

    def setup(self):
        self.device = {}
        self.device['manufacturer'] = 'Delta Dore'
        self.device['model'] = 'Volet'
        self.device['name'] = self.name
        self.device['identifiers'] = id=self.id
        self.config_topic = cover_config_topic.format(id=self.id)
        self.config = {}
        self.config['name'] = self.name
        self.config['unique_id'] = self.id
        # self.config['attributes'] = self.attributes
        self.config['command_topic'] = cover_set_postion_topic.format(id=self.id)
        self.config['set_position_topic'] = cover_set_postion_topic.format(id=self.id)
        self.config['position_topic'] = cover_position_topic.format(id=self.id)
        self.config['payload_open'] = 100
        self.config['payload_close'] = 0
        self.config['retain'] = 'false'
        self.config['device'] = self.device

        # print(self.config)
        hassio.publish(self.config_topic, json.dumps(self.config), qos=0)

    def update(self):
        self.setup()
        self.position_topic = cover_position_topic.format(id=self.id, current_position=self.current_position)
        hassio.publish(self.position_topic, self.current_position, qos=0, retain=True)

        # self.attributes_topic = cover_attributes_topic.format(id=self.id, attributes=self.attributes)
        # hassio.publish(self.attributes_topic, self.attributes, qos=0)

class Alarm:
    def __init__(self, id, name, current_state=None, attributes=None):
        self.id = id
        self.name = name
        self.current_state = current_state
        self.attributes = attributes

    # def id(self):
    #     return id

    # def name(self):
    #     return name

    # def current_state(self):
    #     return current_state

    # def attributes(self):
    #     return attributes

    def setup(self):
        self.device = {}
        self.device['manufacturer'] = 'Delta Dore'
        self.device['model'] = 'Tyxal'
        self.device['name'] = self.name
        self.device['identifiers'] = id=self.id
        self.config_alarm = alarm_config.format(id=self.id)
        self.config = {}
        self.config['name'] = self.name
        self.config['unique_id'] = self.id
        self.config['device'] = self.device
        # self.config['attributes'] = self.attributes
        self.config['command_topic'] = alarm_command_topic.format(id=self.id)
        self.config['state_topic'] = alarm_state_topic.format(id=self.id)

        # print(self.config)
        hassio.publish(self.config_alarm, json.dumps(self.config), qos=0)

    def update(self):
        self.setup()
        self.state_topic = alarm_state_topic.format(id=self.id, state=self.current_state)
        hassio.publish(self.state_topic, self.current_state, qos=0, retain=True)

        # self.attributes_topic = alarm_attributes_topic.format(id=self.id, attributes=self.attributes)
        # hassio.publish(self.attributes_topic, self.attributes, qos=0)

    # alarm_topic = "homeassistant/alarm_control_panel/tydom/#"
    # alarm_config = "homeassistant/alarm_control_panel/tydom/{id}/config"
    # alarm_state_topic = "homeassistant/alarm_control_panel/tydom/{id}/state"
    # alarm_sos_topic = "homeassistant/binary_sensor/tydom/{id}/sos"
    # alarm_attributes_topic = "homeassistant/alarm_control_panel/tydom/{id}/attributes"

class BytesIOSocket:
    def __init__(self, content):
        self.handle = BytesIO(content)

    def makefile(self, mode):
        return self.handle

class HTTPRequest(BaseHTTPRequestHandler):
    def __init__(self, request_text):
        #self.rfile = StringIO(request_text)
        self.raw_requestline = request_text
        self.error_code = self.error_message = None
        self.parse_request()

    def send_error(self, code, message):
        self.error_code = code
        self.error_message = message

def response_from_bytes(data):
    sock = BytesIOSocket(data)
    response = HTTPResponse(sock)
    response.begin()
    return urllib3.HTTPResponse.from_httplib(response)

def put_response_from_bytes(data):
    request = HTTPRequest(data)
    return request

# Get pretty name for a device id
def get_name_from_id(id):
    name = ""
    if len(device_dict) != 0:
        name = device_dict[id]
    return(name)

# Basic response parsing. Typically GET responses
async def parse_response(incoming):
    data = incoming
    msg_type = None
    first = str(data[:20])

    # Detect type of incoming data
    if (data != ''):
        if ("id" in first):
            print('Incoming message type : data detected')
            msg_type = 'msg_data'
        elif ("date" in first):
            print('Incoming message type : config detected')
            msg_type = 'msg_config'
        elif ("doctype" in first):
            print('Incoming message type : html detected (probable pong)')
            msg_type = 'msg_html'
            print(data)
        else:
            print('Incoming message type : no type detected')
            print(first)

        if not (msg_type == None):
            try:
                parsed = json.loads(data)
                # print(parsed)
                if (msg_type == 'msg_config'):
                    for i in parsed["endpoints"]:
                        # Get list of shutter
                        if i["last_usage"] == 'shutter':
                            # print('{} {}'.format(i["id_endpoint"],i["name"]))
                            device_dict[i["id_endpoint"]] = i["name"]

                            # TODO get other device type
                        if i["last_usage"] == 'alarm':
                            # print('{} {}'.format(i["id_endpoint"], i["name"]))
                            device_dict[i["id_endpoint"]] = "Tyxal Alarm"
                    print('Configuration updated')
                elif (msg_type == 'msg_data'):
                    for i in parsed:
                        attr = {}
                        if i["endpoints"][0]["error"] == 0:
                            for elem in i["endpoints"][0]["data"]:
                                # Get full name of this id
                                endpoint_id = i["endpoints"][0]["id"]
                                # Element name
                                elementName = elem["name"]
                                # Element value
                                elementValue = elem["value"]

                                # Get last known position (for shutter)
                                if elementName == 'position':
                                    name_of_id = get_name_from_id(endpoint_id)
                                    if len(name_of_id) != 0:
                                        print_id = name_of_id
                                    else:
                                        print_id = endpoint_id
                                    # print('{} : {}'.format(print_id, elementValue))
                                    new_cover = "cover_tydom_"+str(endpoint_id)
                                    print("Cover created / updated : "+new_cover)
                                    new_cover = Cover(id=endpoint_id,name=print_id, current_position=elementValue, attributes=i)
                                    new_cover.update()

                                # Get last known position (for alarm)
                                if elementName in deviceAlarmKeywords:
                                    alarm_data = '{} : {}'.format(elementName, elementValue)
                                    # print(alarm_data)
                                    # alarmMode  : ON or ZONE or OFF
                                    # alarmState : ON = Triggered
                                    # alarmSOS   : true = SOS triggered
                                    state = None
                                    sos = False

                                    if alarm_data == "alarmMode : ON":
                                        state = "armed_away"
                                    elif alarm_data == "alarmMode : ZONE":
                                        state = "armed_home"
                                    elif alarm_data == "alarmMode : OFF":
                                        state = "disarmed"
                                    elif alarm_data == "alarmState : ON":
                                        state = "triggered"
                                    elif alarm_data == "alarmSOS : true":
                                        sos = True
                                    else:
                                        attr[elementName] = [elementValue]
                                    #     attr[alarm_data]
                                        # print(attr)
                                    #device_dict[i["id_endpoint"]] = i["name"]
                                    if (sos == True):
                                        print("SOS !")
                                    if not (state == None):
                                        # print(state)
                                        alarm = "alarm_tydom_"+str(endpoint_id)
                                        print("Alarm created / updated : "+alarm)
                                        alarm = Alarm(id=endpoint_id,name="Tyxal Alarm", current_state=state, attributes=attr)
                                        alarm.update()
                elif (msg_type == 'msg_html'):
                    print("pong")
                else:
                    # Default json dump
                    print()
                    print(json.dumps(parsed, sort_keys=True, indent=4, separators=(',', ': ')))
            except Exception as e:
                print('Cannot parse response !')
                # print('Response :')
                # print(data)
                if (e != 'Expecting value: line 1 column 1 (char 0)'):
                    print(e)

# PUT response DIRTY parsing
def parse_put_response(bytes_str):
    # TODO : Find a cooler way to parse nicely the PUT HTTP response
    resp = bytes_str[len(cmd_prefix):].decode("utf-8")
    fields = resp.split("\r\n")
    fields = fields[6:]  # ignore the PUT / HTTP/1.1
    end_parsing = False
    i = 0
    output = str()
    while not end_parsing:
        field = fields[i]
        if len(field) == 0 or field == '0':
            end_parsing = True
        else:
            output += field
            i = i + 2
    parsed = json.loads(output)
    return json.dumps(parsed)
    # print(json.dumps(parsed, sort_keys=True, indent=4, separators=(',', ': ')))

# Generate 16 bytes random key for Sec-WebSocket-Keyand convert it to base64
def generate_random_key():
    return base64.b64encode(os.urandom(16))

# Build the headers of Digest Authentication
def build_digest_headers(nonce):
    digestAuth = HTTPDigestAuth(login, password)
    chal = dict()
    chal["nonce"] = nonce[2].split('=', 1)[1].split('"')[1]
    chal["realm"] = "ServiceMedia" if remote_mode is True else "protected area"
    chal["qop"] = "auth"
    digestAuth._thread_local.chal = chal
    digestAuth._thread_local.last_nonce = nonce
    digestAuth._thread_local.nonce_count = 1
    return digestAuth.build_digest_header('GET', "https://{}:443/mediation/client?mac={}&appli=1".format(host, mac))

# Send Generic GET message
async def send_message(websocket, msg):
    str = cmd_prefix + "GET " + msg +" HTTP/1.1\r\nContent-Length: 0\r\nContent-Type: application/json; charset=UTF-8\r\nTransac-Id: 0\r\n\r\n"
    a_bytes = bytes(str, "ascii")
    await websocket.send(a_bytes)
    return 0
    # return await websocket.recv() #disable if handler

# Send Generic POST message
async def send_post_message(websocket, msg):
    str = cmd_prefix + "POST " + msg +" HTTP/1.1\r\nContent-Length: 0\r\nContent-Type: application/json; charset=UTF-8\r\nTransac-Id: 0\r\n\r\n"
    a_bytes = bytes(str, "ascii")
    await websocket.send(a_bytes)
    return 0
    # return await websocket.recv()

###############################################################
# Commands                                                    #
###############################################################

# Get some information on Tydom
async def get_info(websocket):
    msg_type = '/info'
    await send_message(websocket, msg_type)

# Refresh (all)
async def post_refresh(websocket):
    print("Refresh....")
    msg_type = '/refresh/all'
    await send_post_message(websocket, msg_type)

# Get the moments (programs)
async def get_moments(websocket):
    msg_type = '/moments/file'
    await send_message(websocket, msg_type)

# Get the scenarios
async def get_scenarii(websocket):
    msg_type = '/scenarios/file'
    await send_message(websocket, msg_type)

# Get a ping (pong should be returned)
async def get_ping(websocket):
    msg_type = 'ping'
    await send_message(websocket, msg_type)

# Get all devices metadata
async def get_devices_meta(websocket):
    msg_type = '/devices/meta'
    await send_message(websocket, msg_type)

# Get all devices data
async def get_devices_data(websocket):
    msg_type = '/devices/data'
    await send_message(websocket, msg_type)

# List the device to get the endpoint id
async def get_configs_file(websocket):
    msg_type = '/configs/file'
    await send_message(websocket, msg_type)

async def get_data(websocket):
    await get_configs_file(websocket)
    await asyncio.sleep(2)
    await get_devices_data(websocket)

# Give order (name + value) to endpoint
async def put_devices_data(websocket, endpoint_id, name, value):
    # For shutter, value is the percentage of closing
    body="[{\"name\":\"" + name + "\",\"value\":\""+ value + "\"}]"
    # endpoint_id is the endpoint = the device (shutter in this case) to open.
    str_request = cmd_prefix + "PUT /devices/{}/endpoints/{}/data HTTP/1.1\r\nContent-Length: ".format(str(endpoint_id),str(endpoint_id))+str(len(body))+"\r\nContent-Type: application/json; charset=UTF-8\r\nTransac-Id: 0\r\n\r\n"+body+"\r\n\r\n"
    a_bytes = bytes(str_request, "ascii")
    await websocket.send(a_bytes)
    # name = await websocket.recv()
    # parse_response(name)
    # name = await websocket.recv()
    # try:
    #     parse_response(name)
    # except:
    #     parse_put_response(name)

# Run scenario
async def put_scenarios(websocket, scenario_id):
    body=""
    # scenario_id is the id of scenario got from the get_scenarios command
    str_request = cmd_prefix + "PUT /scenarios/{} HTTP/1.1\r\nContent-Length: ".format(str(scenario_id))+str(len(body))+"\r\nContent-Type: application/json; charset=UTF-8\r\nTransac-Id: 0\r\n\r\n"+body+"\r\n\r\n"
    a_bytes = bytes(str_request, "ascii")
    await websocket.send(a_bytes)
    # name = await websocket.recv()
    # parse_response(name)

# Give order to endpoint
async def get_device_data(websocket, id):
    # 10 here is the endpoint = the device (shutter in this case) to open.
    str_request = cmd_prefix + "GET /devices/{}/endpoints/{}/data HTTP/1.1\r\nContent-Length: 0\r\nContent-Type: application/json; charset=UTF-8\r\nTransac-Id: 0\r\n\r\n".format(str(id),str(id))
    a_bytes = bytes(str_request, "ascii")
    await websocket.send(a_bytes)
    # name = await websocket.recv()
    # parse_response(name)

######## Messages Logic

async def consumer(websocket):
    # Receiver
    while True:
        bytes_str = await websocket.recv()
        print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
        # print(bytes_str)
        first = str(bytes_str[:40]) # Scanning 1st characters
        try:
            if ("refresh" in first):
                print('OK refresh message detected !')
                try:

                    hassio.publish('homeassistant/sensor/tydom/last_update', str(datetime.fromtimestamp(time.time())), qos=1, retain=True)
                except:
                    print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                    print('RAW INCOMING :')
                    print(bytes_str)
                    print('END RAW')
                    print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
            if ("PUT /devices/data" in first):
                print('PUT /devices/data message detected !')
                try:
                    incoming = parse_put_response(bytes_str)
                    # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                    await parse_response(incoming)
                    print('PUT message processed !')
                    print("##################################")
                    hassio.publish('homeassistant/sensor/tydom/last_update', str(datetime.fromtimestamp(time.time())), qos=1, retain=True)
                except:
                    print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                    print('RAW INCOMING :')
                    print(bytes_str)
                    print('END RAW')
                    print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
            elif ("scn" in first):

                try:
                    # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                    incoming = get(bytes_str)
                    await parse_response(incoming)
                    print('Scenarii message processed !')
                    print("##################################")
                except:
                    print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                    print('RAW INCOMING :')
                    print(bytes_str)
                    print('END RAW')
                    print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")            
            elif ("POST" in first):

                try:
                    # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                    incoming = parse_put_response(bytes_str)
                    await parse_response(incoming)
                    print('POST message processed !')
                    print("##################################")
                except:
                    print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                    print('RAW INCOMING :')
                    print(bytes_str)
                    print('END RAW')
                    print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")

            elif ("HTTP/1.1" in first): #(bytes_str != 0) and 
                response = response_from_bytes(bytes_str[len(cmd_prefix):])
                incoming = response.data.decode("utf-8")
                # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                # print(incoming)
                # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                try:
                    # print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                    await parse_response(incoming)
                    print('Common / GET response message processed !')
                    print("##################################")
                    hassio.publish('homeassistant/sensor/tydom/last_update', str(datetime.fromtimestamp(time.time())), qos=1, retain=True)
                except:
                    print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                    print('RAW INCOMING :')
                    print(bytes_str)
                    print('END RAW')
                    print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                    # await parse_put_response(incoming)
            else:
                print("Didn't detect incoming type, here it is :")
                print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
                print('RAW INCOMING :')
                print(bytes_str)
                print('END RAW')
                print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
        except Exception as e:
            print('Consumer has crashed when analysing incoming !')    
            print(e)
            if (hassio != None):
                hassio.publish('homeassistant/sensor/tydom/last_crash', 'Consumer crashed !', qos=1, retain=True)
            print('Webconnection error, retrying in 8 seconds...')
            tydom = None
            await asyncio.sleep(8)
            await websocket_connection()

async def producer(websocket):

#HEARTBEAT, ping seems ineffective, so "pinging" with refresh command every 48s for remote
        # while True:
    await asyncio.sleep(48)
    # await get_ping(websocket)
    await post_refresh(tydom)
    # await get_data(tydom)
    print("Websocket refreshed at ", str(datetime.fromtimestamp(time.time())))

async def consumer_handler(websocket):
    while True :
        try:
            await consumer(websocket)
        except Exception as e:
            print("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!")
            print("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!")
            print("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!")
            print('Consumer task has crashed !')

            print(e)
            if (hassio != None):
                hassio.publish('homeassistant/sensor/tydom/last_crash', "Cosumer handler crashed !", qos=1, retain=True)
            print('Webconnection error, retrying in 8 seconds...')
            tydom = None
            await asyncio.sleep(8)
            await websocket_connection()

async def producer_handler(websocket):
    while True :
        await producer(websocket)

######## HANDLER
async def handler(websocket):
    try:
        # print("Starting handlers...")
        consumer_task = asyncio.ensure_future(
            consumer_handler(websocket))
        producer_task = asyncio.ensure_future(
            producer_handler(websocket))
        done, pending = await asyncio.wait(
            [consumer_task, producer_task],
            return_when=asyncio.FIRST_COMPLETED,
        )
        for task in pending:
            task.cancel()

    except Exception as e:
        print("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!")
        print(e)
        if (hassio != None):
            hassio.publish('homeassistant/sensor/tydom/last_crash', "Handler handler crashed !", qos=1, retain=True)
        print('Webconnection error, retrying in 8 seconds...')
        tydom = None
        await asyncio.sleep(8)
        await websocket_connection()

async def websocket_connection():
    global tydom
    httpHeaders =  {"Connection": "Upgrade",
                    "Upgrade": "websocket",
                    "Host": host + ":443",
                    "Accept": "*/*",
                    "Sec-WebSocket-Key": generate_random_key(),
                    "Sec-WebSocket-Version": "13"
                    }

    conn = http.client.HTTPSConnection(host, 443, context=ssl_context)
    # Get first handshake
    conn.request("GET", "/mediation/client?mac={}&appli=1".format(mac), None, httpHeaders)
    res = conn.getresponse()
    # Get authentication
    nonce = res.headers["WWW-Authenticate"].split(',', 3)
    # read response
    res.read()
    # Close HTTPS Connection
    conn.close()
    # Build websocket headers
    websocketHeaders = {'Authorization': build_digest_headers(nonce)}
    if ssl_context is not None:
        websocket_ssl_context = ssl_context
    else:
        websocket_ssl_context = True # Verify certificate
    try:
        print('"Attempting websocket connection..."')
        ########## CONNECTION
        # websocket = await websockets.client.connect('wss://{}:443/mediation/client?mac={}&appli=1'.format(host, mac),
        #                                      extra_headers=websocketHeaders, ssl=websocket_ssl_context)

        async with websockets.client.connect('wss://{}:443/mediation/client?mac={}&appli=1'.format(host, mac),
                                            extra_headers=websocketHeaders, ssl=websocket_ssl_context) as websocket:

            tydom = websocket
            print("Tydom Websocket is Connected !", tydom)
            print("##################################")
            print('Requesting 1st data...')
            await post_refresh(tydom)
            await get_data(tydom)
            while True:
                # await consumer(tydom) #Only receiving from socket in real time
                await handler(tydom) #If you want to send periodically something, disable consumer
    except Exception as e:
        print('Webconnection error, retrying in 8 seconds...')
        if (hassio != None):
            hassio.publish('homeassistant/sensor/tydom/last_crash', "Websocket connexion crashed !", qos=1, retain=True)

        await asyncio.sleep(8)
        await websocket_connection()
# Main async task
async def main_task():
    print(str(datetime.fromtimestamp(time.time())))
    try:
        if (tydom == None) or not tydom.open or (hassio == None):
            print("##################################")
            start = time.time()
            await mqttconnection(mqtt_host, mqtt_user, mqtt_pass)
            await websocket_connection()
            print('Connection total time :')
            end = time.time()
            print(end - start)

        else:
            print('Websocket is still opened ! requesting data...')
            await post_refresh(tydom)
    except Exception as e:
        print("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!")
        print('Connection total time :')
        end = time.time()
        print(end - start)
        print(str(datetime.fromtimestamp(time.time())))
        print(e)
        if (hassio != None):
            hassio.publish('homeassistant/sensor/tydom/last_crash', 'Main task crashed !', qos=1, retain=True)
        print('Something bad happened, reconnecting...')
        # await asyncio.sleep(8)
        await main_task()

def start_loop():
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main_task())
    loop.run_forever()

if __name__ == '__main__':
    while True:
        try:
            start_loop()
        except Exception as e:
            print('FATAL ERROR !')
            print(e)
            try:
                hassio.publish('homeassistant/sensor/tydom/last_crash', 'FATAL ERROR', qos=1, retain=True)
            except:
                os.excel("tydom2mqtt_restarter.sh","")
                sys.exit(-1)
        # start_loop()

        # Get informations (not very useful)
        # await get_info(websocket)

        # Get all moments stored on Tydom
        # await get_moments(websocket)

        # Get scenarios ids
        # print("Get scenarii")
        # await get_scenarios(websocket)

        # Run scenario with scn id returned in previous command
        # await put_scenarios(websocket, 15)
        # print("Get names of all devices")
        # await get_configs_file(websocket)
        # # await get_devices_meta(websocket)

        # print("Get data of all devices")
        # await get_devices_data(websocket)

        # Get data of a specific device
        #await get_device_data(websocket, 9)

        # Set a shutter position to 10%
        #await put_devices_data(websocket, 9, "position", "10.0")
        # TODO : Wait hardcoded for now to put response from websocket server
        #time.sleep(45)

cth35 commented 4 years ago

Not get enough time to look to your work. The idea would be to isolate the root code (i.e the code that deals with the tydom communication) and keep it independant of the MQTT stuff you developped. In order the root code still works with other any future async mecanism.

mrwiwi commented 4 years ago

https://github.com/WiwiWillou/tydom2mqtt

Made my homework !

Some new here : https://community.home-assistant.io/t/delta-dore-tydom-custom-component-need-help/151333/8

cth35 commented 4 years ago

Hey well done ! I unfortunately don't have enough time to go on working on this project :-( Did you get rid of deconnection (Error 1006) when connected localy ? Le lundi 13 janvier 2020 à 22:16:06 UTC+1, WiwiWillou notifications@github.com a écrit :

https://github.com/WiwiWillou/tydom2mqtt

Made my homework !

Some new here : https://community.home-assistant.io/t/delta-dore-tydom-custom-component-need-help/151333/8

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

cth35 / tydom_python

Pull Request / Add MQTT and persistent connection for real time #7