Local only access for the gen2 miniserver

[davosian]

According to the documentation, I need to enter the loxonecloud address for my gen2 miniserver. However, I am not allowing my miniserver to be externally accessible. Is this a hard requirement and therefore an all-local integration not possible?If so, can you tell me where this restriction comes from so that I can try to find an alternative approach myself?

I can access the server using things like http://local-ip/data/LoxAPP3.json 1. Also, simply going to http://local-ip gives me the web ui for the server. I am also ok with self signed certificates or even using http only as long as I do not need to open up the miniserver externally.

Thanks a lot for providing this binding in the first place!

If I do follow the documentation, I am getting the following error message in home assistant:

2021-08-06 15:12:31 ERROR (MainThread) [custom_components.loxone.api] Could not connect to Loxone! Status code 409.
2021-08-06 15:12:31 ERROR (MainThread) [custom_components.loxone.miniserver] Error connecting to loxone miniserver #2 Code (False)

This makes sense, since my loxone server is not exposed and known to the loxone dns server.

[davosian]

According to the official API documentation, accessing the Gen2 miniserver should also be possible without cloud access. See https://www.loxone.com/dede/wp-content/uploads/sites/2/2021/06/1201_Communicating-with-the-Miniserver.pdf on page 12:

How can I connect locally with TLS? As long as your Miniserver is capable of TLS and is using our CloudDNS-Certificate, you can. You need to create a hostname just like if you would connect from the internet, but use your Miniservers local IP and port. E.g. “192-168-1-47.{snr}.dyndns.loxonecloud.com:443”

If the Miniserver is using a custom certificate, connecting with it via it’s local IP will result in certificate verification errors.

[davosian]

When I open the URL in the form of https://local-ip.serialnumber.dyndns.loxonecloud.com (with dashes instead of dots for the ip), I can reach the server just fine. Source: https://github.com/Loxone/lxcommunicator

Using the same IP and port 443 for PyLoxone, I am getting the following error message in the home assistant log:

2021-08-06 15:04:33 ERROR (MainThread) [homeassistant.config_entries] Error setting up entry PyLoxone for loxone
Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/h11/_state.py", line 249, in _fire_event_triggered_transitions
new_state = EVENT_TRIGGERED_TRANSITIONS[role][state][event_type]
KeyError: <class 'h11._events.ConnectionClosed'>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/homeassistant/config_entries.py", line 293, in async_setup
result = await component.async_setup_entry(hass, self) # type: ignore
File "/config/custom_components/loxone/__init__.py", line 132, in async_setup_entry
if not await miniserver.async_setup():
File "/config/custom_components/loxone/miniserver.py", line 101, in async_setup
request_code = await self.lox_config.getJson()
File "/config/custom_components/loxone/api.py", line 60, in getJson
api_resp = await requests.get(url_api,
File "/config/.local/lib/python3.8/site-packages/requests_async/api.py", line 11, in get
return await request("get", url, params=params, **kwargs)
File "/config/.local/lib/python3.8/site-packages/requests_async/api.py", line 6, in request
return await session.request(method=method, url=url, **kwargs)
File "/config/.local/lib/python3.8/site-packages/requests_async/sessions.py", line 79, in request
resp = await self.send(prep, **send_kwargs)
File "/config/.local/lib/python3.8/site-packages/requests_async/sessions.py", line 136, in send
r = await adapter.send(request, **kwargs)
File "/config/.local/lib/python3.8/site-packages/requests_async/adapters.py", line 48, in send
response = await self.pool.request(
File "/config/.local/lib/python3.8/site-packages/http3/interfaces.py", line 49, in request
return await self.send(request, verify=verify, cert=cert, timeout=timeout)
File "/config/.local/lib/python3.8/site-packages/http3/dispatch/connection_pool.py", line 130, in send
raise exc
File "/config/.local/lib/python3.8/site-packages/http3/dispatch/connection_pool.py", line 120, in send
response = await connection.send(
File "/config/.local/lib/python3.8/site-packages/http3/dispatch/connection.py", line 59, in send
response = await self.h11_connection.send(request, timeout=timeout)
File "/config/.local/lib/python3.8/site-packages/http3/dispatch/http11.py", line 58, in send
http_version, status_code, headers = await self._receive_response(timeout)
File "/config/.local/lib/python3.8/site-packages/http3/dispatch/http11.py", line 130, in _receive_response
event = await self._receive_event(timeout)
File "/config/.local/lib/python3.8/site-packages/http3/dispatch/http11.py", line 161, in _receive_event
event = self.h11_state.next_event()
File "/usr/lib/python3.8/site-packages/h11/_connection.py", line 443, in next_event
exc._reraise_as_remote_protocol_error()
File "/usr/lib/python3.8/site-packages/h11/_util.py", line 76, in _reraise_as_remote_protocol_error
raise self
File "/usr/lib/python3.8/site-packages/h11/_connection.py", line 427, in next_event
self._process_event(self.their_role, event)
File "/usr/lib/python3.8/site-packages/h11/_connection.py", line 242, in _process_event
self._cstate.process_event(role, type(event), server_switch_event)
File "/usr/lib/python3.8/site-packages/h11/_state.py", line 238, in process_event
self._fire_event_triggered_transitions(role, event_type)
File "/usr/lib/python3.8/site-packages/h11/_state.py", line 251, in _fire_event_triggered_transitions
raise LocalProtocolError(
h11._util.RemoteProtocolError: can't handle event type ConnectionClosed when role=SERVER and state=SEND_RESPONSE

[JoDehli]

@davosian Now I am back. By the way are you german? If so we can speak german.

For me it is hard to test it because I have a gen1 which is not capable to access via the external cloud loxone dns service. What I know is that it is working with external access enabled (remote connect)

I already had some users which are using the gen2 but all with remote connect enabled. Can you try it only for test purpose with the remote connect?

You have to enter the Miniserver ip without https://

[davosian]

Hey @JoDehli, welcome back :)

I have tried a few things in the meantime. Unfortunately, I did not get it to work yet, but I have learned a few things along the way. Here is what I tried:

For testing, I enabled Remote Connect and made sure to allow for Loxone Config to configure the server remotely. In order to make this work, you have to register your Miniserver with Loxone which I did.

Testing with Home Assistant

First attempt

I then tried to connect through homeassistant with the following configuration (I double-checked the password):

After a home assistant restart, I find the following error in the log:

Second attempt

When I switch the port to 443, I get a different 500 error:

Testing with Pyloxone-api

I then discovered your second project Pyloxone-api so I used this to debug the issue further:

# running in `poetry shell`
pytest ./tests/test_online.py --host=MYLOCALIP --username=MYUSER --password=MYPASSWORD

This was giving me a successful (local) connection, so I went on to use the remote connection instead:

# running in `poetry shell`
pytest ./tests/test_online.py --host=dns.loxonecloud.com/MYSERIALNR --username=MYUSER --password=MYPASSWORD --use-tls

This results in the test failing with a 500 error (like the home assistant integration):

        # Handle errors. An http error getting any of the required data is
        # probably fatal, so log it and raise it for handling elsewhere. Other errors
        # are (hopefully) unlikely, but are not handled here, so will be raised
        # normally.
        except httpx.RequestError as exc:
            _LOGGER.error(
                f'An error "{exc}" occurred while requesting {exc.request.url!r}.'
            )
            raise LoxoneRequestError(exc) from None
        except LoxoneHTTPStatusError as exc:
            _LOGGER.error(exc)
>           raise LoxoneHTTPStatusError(exc) from None
E           pyloxone_api.exceptions.LoxoneHTTPStatusError: Code 500. Miniserver response was <Answer cmd="getip" IP="" Code="500" />

pyloxone_api/api.py:183: LoxoneHTTPStatusError
----------------------------------------------------- Captured log call -----------------------------------------------------
ERROR    pyloxone_api.api:api.py:182 Code 500. Miniserver response was <Answer cmd="getip" IP="" Code="500" />
================================================== short test summary info ==================================================
FAILED tests/test_online.py::test_online - pyloxone_api.exceptions.LoxoneHTTPStatusError: Code 500. Miniserver response wa...
===================================================== 1 failed in 0.31s =====================================================

I then tried variants of the test command without --use-tls and with different ports using --port=80 and --port=443 but I always ended up with 500 error messages.

Testing with LxCommunicator

At this point, I was wondering whether my remote access was the problem, so I decided to test the "official" node.js code sample from the LxCommunicator v1.0.1 project: I cloned the repository, changed the connection parameters inside the ./test/index.js file to use my cloud DNS credentials and then ran the test:

$ npm test

> lxcommunicator@1.1.1 test /home/myuser/lxcommunicator
> node ./test/index.js

{ token:
   'SOMETOKEN=',
  key:
   'SOMEKEY',
  validUntil: 402914356,
  tokenRights: 1668,
  unsecurePass: false,
  username: 'MYUSER',
  msPermission: 1668,
  timeouts:
   { '1668':
      Timeout {
        _called: false,
        _idleTimeout: 86400000,
        _idlePrev: [TimersList],
        _idleNext: [TimersList],
        _idleStart: 3399,
        _onTimeout: [Function: bound _keepAliveFired],
        _timerArgs: undefined,
        _repeat: null,
        _destroyed: false,
        [Symbol(unrefed)]: false,
        [Symbol(asyncId)]: 109,
        [Symbol(triggerId)]: 108 } } }
Successfully executed 'dev/sps/enablebinstatusupdate' with code 200 and value 1
17ab174e-0162-545f-ffff9f8aab4cf11a -> 7
17ab174e-0172-5471-ffff9f8aab4cf11a -> 381
17ab174e-0172-5474-ffff9f8aab4cf11a -> 1242
17ab174e-020e-5739-ffff9f8aab4cf11a -> 1
17ab174e-020e-5735-ffff9f8aab4cf11a -> 1
17ab174e-0173-2587-ffffacc9ae31a9ea -> 0
17ab174e-0191-54af-ffff9f8aab4cf11a -> 0
17ab174e-0191-54b8-ffff9f8aab4cf11a -> 398079782000

As you can see, this test was successful in that I was able to connect and also retrieved actual sensor data.

My conclusion out of this is, that there probably is something not right using remote access in your current implementation - at least related to my particular setup. Let me know how I can help to debug this further. And yes, we can have a conversation "auf Deutsch", no problem ;) I simply stuck to English in case someone else stumbles across this issue (or in case other contributors would like to jump in).

[davosian]

Btw, I also tried adding the Loxone test server from https://github.com/Loxone/lxcommunicator/blob/dc70d793407bcab2edeecd0c3550acda2f536d24/test/index.js#L64 to home assistant, but I am getting the same Error connecting to loxone miniserver #1 in Home Assistant. I am not sure though whether it is a Gen1 or a Gen2 and adding it remotely is enabled in the first place.

Also, my own Miniserver is still in a testing mode and I could provide you with access to help you with debugging.

[JoDehli]

The Testserver is a gen1 and remote access will not function.

The loxone-api project is also from a pyloxone user and me. Well he did most of the work based of my pyloxone implementation. The goal is to use the loxone-api in future releases of pyloxone. Maybe you can wait if we merged them together. I do not know when we do that because I want to make sure that it is working correctly.

And as I already mentioned I can not test it because I have no gen2 for testing. If you want you can give me access to your test system. I could try to implement it for both systems (pyloxone and pyloxone-api) for remote access enabled if it is no so much work.

[davosian]

@JoDehli I will gladly give you access if this helps debugging the issue. I have now set up a user for this that I will pass on to you. Kindly let me know how I can contact you to exchange the details.

Regarding the two projects: I am mainly interested in the home assistant integration but I thought that it might be easier to debug with the test case you have for the loxone-api project. Whatever makes the most sense to you.

Maybe it also helps to understand how Loxone is handling the authentication by checking their project at https://github.com/Loxone/lxcommunicator? Just an idea...

[JoDehli]

@davosian I have made a discord server. https://discord.gg/ZTbDjxw8 there you can write me a private message.

[GledholtHall]

I am watching this topic with interest so staying in English really helps me :-)

[edzonca]

Hi @JoDehli , the same error here... any update on that? Thanks for your work

[JoDehli]

I have now gen2 so I can not test it. Sorry. You must use the local access for now. I can look for it after I finished my work on the beta branch.

[JunoGregoire]

I have the same question, but for the gen1 miniserver. There are no gen1 miniserver instructions. Though the cloud instructions do work for gen1 too.

[JoDehli]

@JunoGregoire why do you want a external access from you homeassistant? For security it makes no sense I think? Is your homeassistant not in the same network as you loxone?

[JunoGregoire]

@JoDehli I don't really want external access, that's the point. But it appears like I have to allow external access to my Miniserver to be able to use HA and Loxone gen 1 Miniserver together. Is there no option to make them talk to each other over LAN?

[JoDehli]

I have a gen1 with external access disabled. I have chosen a different port than 80. Then I entered the IP in the configuration from the pyloxone with that port. No need to external access.

[JoDehli]

Of course I disabled external access in the loxone. Maybe you have there a manual external access defined. Did you checked that..

[JunoGregoire]

I followed the directions above which specify to use Loxone Cloud.

[jburget]

Hi, I am a little bit confused. So currently, I don't need to know how, but whether is there a way to connect HA to Loxone miniserver gen2 totally locally. Because I can not risk not working integration with internet black out. At the moment, it seems to me that communication flows over Loxone cloud? Or Loxone cloud just provides local ip of Loxone miniserver? Than I would just set static ips for local network and should be good. I would really like to have HA and also Loxone server disconnected from the internet. Just considering these two systems to use, so thanks for quick answer.

[JoDehli]

Sorry I can not implement it because I have no gen2. I can not help you with this problem.

[brechtvhb]

I have a gen 2 mini server but I am also having issues.

If I enter the dyndns.loxonecloud.com IP I can not start the integration.

2022-10-19 21:51:17.538 ERROR (MainThread) [homeassistant.config_entries] Error setting up entry PyLoxone for loxone
File "/config/custom_components/loxone/__init__.py", line 156, in async_setup_entry
File "/config/custom_components/loxone/miniserver.py", line 118, in async_setup
File "/config/custom_components/loxone/api.py", line 121, in getJson

If I enter my local IP (port 80) I can start the integration but I am not receiving any sensor values and can not send any command to the miniserver

Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/websockets/legacy/protocol.py", line 956, in transfer_data
    await asyncio.shield(self._put_message_waiter)
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/config/custom_components/loxone/miniserver.py", line 167, in listen_loxone_send
    await self.api.send_websocket_command(device_uuid, value)
  File "/config/custom_components/loxone/api.py", line 387, in send_websocket_command
    await self._ws.send(command)
  File "/usr/local/lib/python3.10/site-packages/websockets/legacy/protocol.py", line 620, in send
    await self.ensure_open()
  File "/usr/local/lib/python3.10/site-packages/websockets/legacy/protocol.py", line 921, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedError: sent 1011 (unexpected error) keepalive ping timeout; no close frame received

Anyone who got a gen 2 server working locally?

[brechtvhb]

Used remote connection as test but I am still receiving this error

Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/websockets/legacy/protocol.py", line 956, in transfer_data await asyncio.shield(self._put_message_waiter) asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/config/custom_components/loxone/miniserver.py", line 167, in listen_loxone_send await self.api.send_websocket_command(device_uuid, value) File "/config/custom_components/loxone/api.py", line 387, in send_websocket_command await self._ws.send(command) File "/usr/local/lib/python3.10/site-packages/websockets/legacy/protocol.py", line 620, in send await self.ensure_open() File "/usr/local/lib/python3.10/site-packages/websockets/legacy/protocol.py", line 921, in ensure_open raise self.connection_closed_exc() websockets.exceptions.ConnectionClosedError: sent 1011 (unexpected error) keepalive ping timeout; no close frame received

[brechtvhb]

Disabling remote access in loxone miniserver seems to have make it work. /happydance !!

[machinekoder]

Currently working on another project directly using the API from this repo and I get the following code at https://github.com/JoDehli/PyLoxone/blob/master/custom_components/loxone/api.py#L477 websockets.exceptions.ConnectionClosedOK: received 1000 (OK) normally closed; then sent 1000 (OK) normally closed.

Any ideas what could cause this?

[JoDehli]

No. When did it happen on start or after a certain time?

[machinekoder]

Right away.

[wema1043]

Are there any updates on this? I get the same error message as brechtvhb also don‘t get any updated sensor values. I don‘t want to disable remote access on the miniserver.

[davosian]

I am using the pyloxone beta from https://github.com/JoDehli/PyLoxone_beta/releases with a gen2 miniserver and remote access disabled. This setup is working fine also delivering sensor values. However, when I enable remote access, I am also not getting any updates (at least last time I checked on release 0.0.8 a few months ago).

[leonmeijer]

Currently working on another project directly using the API from this repo and I get the following code at https://github.com/JoDehli/PyLoxone/blob/master/custom_components/loxone/api.py#L477 websockets.exceptions.ConnectionClosedOK: received 1000 (OK) normally closed; then sent 1000 (OK) normally closed.

Any ideas what could cause this?

Getting the same error. Worked all fine an hour ago. Then made some changes (added new Virtual Inputs), now getting this received 1000 (OK) normally closed; then sent 1000 (OK) normally closed error.

[gertst]

What is the conclusion on this? Can a gen 2 miniserver directly communicate over LAN without having internet access? I have remote access on, but nevertheless, Loxone <> HA communication should always be over LAN only. Did anybody have a working setup for this?

Thanks!

[danzika]

@davosian

I am using the pyloxone beta from https://github.com/JoDehli/PyLoxone_beta/releases with a gen2 miniserver and remote access disabled.

But with remote access disabled you can't use the Loxone dns, can you? I definitely was able to run MS (gen2) with LoxoneDNS and PyLoxone connecting to it. Unfortunately, I have broken the setup (due to some infrastructure upgrade) and now, with newer HA (:stable) and latest PyLoxone I'm not able to connect to MS remote access allowed through their cloud; ATM I don't want any open ports into my how network.