search

home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
70.01k stars 29.08k forks source link

Errors during setup of integration "Webmin" #113100

Closed Chris-126 closed 1 day ago

Chris-126 commented 4 months ago

The problem

Hi there,

I found the Webmin integration and would like to utilize it. Unfortunately it was first throwing "Unknown error occurred" and since I've assigned more permissions and enabled "Can accept RPC calls?" = yes (instead of default "Only for root and admin") in Webmin I get the error "Fault 1: Invalid user for RPC" the users is a privileged user with access to everything (for now, I really would like to reduce later).

During the "Unknown error occured" phase the logs has shown the following errors. Now, with the new "RPC" error, there is no new log entry to be found.

I can perfectly login to Webmin with the given user and password.

I've seen the recent issue where you did a limitation to WiFi and Ethernet interfaces. If the mentioned mac addresses are from the target then this might have an effect here. The server is based on a kernel-based virtual machine. So I do have a loopback device as well as virtual interfaces named "venet0" and "venet0:0". Not sure, if this is related...

Do you have any ideas on this?

Thanks in advance and best regards Christian

What version of Home Assistant Core has the issue?

core-2024.3.0

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Webmin

Link to integration documentation on our website

https://www.home-assistant.io/integrations/webmin/

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

Logger: aiohttp.server
Source: /usr/local/lib/python3.12/site-packages/aiohttp/web_protocol.py:421
First occurred: 22:30:31 (4 occurrences)
Last logged: 22:35:56

Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/aiohttp/web_protocol.py", line 452, in _handle_request
    resp = await request_handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/web_app.py", line 543, in _handle
    resp = await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/web_middlewares.py", line 114, in impl
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 91, in security_filter_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 227, in forwarded_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 25, in request_context_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 79, in ban_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 235, in auth_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/headers.py", line 31, in headers_middleware
    response = await handler(request)
               ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/http.py", line 68, in handle
    result = await handler(request, **request.match_info)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/decorators.py", line 72, in with_admin
    return await func(self, request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 197, in post
    return await super().post(request, flow_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/data_validator.py", line 72, in wrapper
    result = await method(view, request, data, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/data_entry_flow.py", line 111, in post
    result = await self._flow_mgr.async_configure(flow_id, data)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 359, in async_configure
    result = await self._async_configure(flow_id, user_input)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 405, in _async_configure
    result = await self._async_handle_step(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 502, in _async_handle_step
    result: FlowResult = await getattr(flow, method)(user_input)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/schema_config_entry_flow.py", line 347, in _async_step
    result = await self._common_handler.async_step(step_id, user_input)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/schema_config_entry_flow.py", line 132, in async_step
    return await self._async_form_step(step_id, user_input)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/schema_config_entry_flow.py", line 168, in _async_form_step
    user_input = await form_step.validate_user_input(self, user_input)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/webmin/config_flow.py", line 57, in validate_user_input
    get_sorted_mac_addresses(data)[0]
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range

Additional information

The target is a Ubuntu 22.04 based webserver which is managed with Virtualmin which is based on Webmin.

home-assistant[bot] commented 4 months ago

Hey there @autinerd, mind taking a look at this issue as it has been labeled with an integration (webmin) you are listed as a code owner for? Thanks!

Code owner commands Code owners of `webmin` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign webmin` Removes the current integration label and assignees on the issue, add the integration domain after the command. - `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue. - `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

(message by CodeOwnersMention)

webmin documentation webmin source (message by IssueLinks)

CinciRyan commented 4 months ago

I didn't know this was possible (the webmin integration), so I added the integration (running 2024.3.0) and added one of my machines. It came right up. I left the checkbox for SSL, but otherwise just put in the IP, user, and password. Maybe try deleting your machine from the integration and adding again. Or, delete the integration and add again.

Chris-126 commented 4 months ago

Hey CiniRyan, I cannot remove ... it's not yet implemented.

just tested the following setup attempts:

  1. servername, ssl, invalid password for dedicated user--> Invalid authentication
  2. servername, ssl, correct credentials for dedicated user --> Fault 1: Invalid user for RPC
  3. servername, no-ssl, correct credentials for dedicated user--> Unexpected error
  4. IP(v4), no-ssl, correct credentials --> Unexpected error
  5. IP(v6), no-ssl, correct credentials --> Unexpected error
  6. servername, ssl, correct credentials for root user --> Unknown error occurred

Each time closed the integration popup and restarted the setup.

So, in regards to my second attempt. Did you use a dedicated user? If so: any special settings? Permissions? I just used "Webmin --> Webmin Users --> Create a new privileged user" then choose "Select all" under "Available Webmin modules". After creation a flagged "yes" for "Can accept RPC calls" and saved.

Unfortunately the last action seems to be not saved at all. Every time I come back to the user details it says "Only for root or admin" instead. But that may be ok, as the user is privileged?

So, which user did you use?

After activating webmin debug log I can see the following entries for the second attempt:

531586 [12/Mar/2024 20:04:06.806030] userabcdefg xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxx - READ "/usr/share/webmin/authentic-theme/config"
531586 [12/Mar/2024 20:04:06.807615] userabcdefg xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxx - READ "/etc/webmin/custom-lang"
531586 [12/Mar/2024 20:04:06.808551] userabcdefg xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxx - READ "/usr/share/webmin//defaultacl"
531586 [12/Mar/2024 20:04:06.808744] userabcdefg xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxx - READ "/etc/webmin/userabcdefg.acl"
531586 [12/Mar/2024 20:04:06.808960] userabcdefg xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxx - READ "/etc/webmin//userabcdefg.acl"
531586 [12/Mar/2024 20:04:06.809342] userabcdefg xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxx - STOP "runtime=0"

Is there a way to debug the setup itself?

autinerd commented 4 months ago

Hi, thanks for your report! I don't have any experiences with Virtualmin, so maybe they do something different.

Can you please run this Python script (after filling out your connection details in <USERNAME>, <PASSWORD> and <HOST>) and enter the result?

import ssl
import urllib
from xmlrpc.client import SafeTransport, ServerProxy

USERNAME = urllib.parse.quote("<USERNAME>")
PASSWORD = urllib.parse.quote("<PASSWORD>")
HOST = urllib.parse.quote("<HOST>")
WEBMIN_ADDRESS = f"https://{USERNAME}:{PASSWORD}@{HOST}:10000/xmlrpc.cgi"

with ServerProxy(WEBMIN_ADDRESS, transport=SafeTransport(context=ssl._create_unverified_context())) as client:
    print(client.net.active_interfaces())

(You can redact the last two hexadecimal groups of all MAC addresses with :xx:xx and fully redact all IP addresses in the result)

Chris-126 commented 4 months ago

So, I ran the script on my Home Assistant command line using my new (privileged) user:

python3 webmin_script.py Traceback (most recent call last): File "/homeassistant/python_scripts/webmin_script.py", line 11, in print(client.net.active_interfaces()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/xmlrpc/client.py", line 1122, in call return self.send(self.name, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/xmlrpc/client.py", line 1464, in request response = self.transport.request( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/xmlrpc/client.py", line 1166, in request return self.single_request(host, handler, request_body, verbose) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/xmlrpc/client.py", line 1182, in single_request return self.parse_response(resp) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/xmlrpc/client.py", line 1354, in parse_response return u.close() ^^^^^^^^^ File "/usr/lib/python3.11/xmlrpc/client.py", line 668, in close raise Fault(**self._stack[0]) xmlrpc.client.Fault: <Fault 1: 'Invalid user for RPC'>

Actually, that is the very same response ...

When I try the very same with "root" (although I try to avoid it as far as possible):

[ { 'address': '127.0.0.1', 'index': 0, 'fullname': 'lo', 'netmask6': [128], 'mtu': 65536, 'name': 'lo', 'netmask': '255.0.0.0', 'edit': 1, 'address6': ['::1'], 'up': 1, 'broadcast': 0, 'scope6': ['host'] }, { 'address': '8x.2xx.xxx.xxx', 'netmask6': [128], 'fullname': 'venet0', 'mtu': 1500, 'index': 1, 'edit': 1, 'netmask': '255.255.255.255', 'name': 'venet0', 'scope6': ['global'], 'up': 1, 'broadcast': '8x.2xx.xxx.xxx', 'address6': ['2axx:23x:42xx:3xxx:exxx:xxx:xxxx:xxxx'] }, { 'mtu': 1500, 'fullname': 'venet0:0', 'index': 2, 'address': '8x.2xx.xxx.xxx', 'virtual': 0, 'broadcast': '8x.2xx.xxx.xxx', 'up': 1, 'edit': 1, 'netmask': '255.255.255.255', 'name': 'venet0' } ]

So, the "root" login during setting up the integration is failing with "Unknown error occurred". This seems to be caused by the fact that there actually is NO mac address ?

I just verified the result with ifconfig / ip link on the target server... matches the result: no mac address.

`sudo ifconfig lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10 loop txqueuelen 1000 (Local Loopback) RX packets 811524 bytes 978197398 (978.1 MB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 811524 bytes 978197398 (978.1 MB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

venet0: flags=211<UP,BROADCAST,POINTOPOINT,RUNNING,NOARP> mtu 1500 inet 127.0.0.1 netmask 255.255.255.255 broadcast 0.0.0.0 destination 127.0.0.1 inet6 2axx:23x:42xx:3xxx:exxx:xxx:xxxx:xxxx prefixlen 128 scopeid 0x0 unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 txqueuelen 0 (UNSPEC) RX packets 817393 bytes 363557915 (363.5 MB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 1287899 bytes 1014092096 (1.0 GB) TX errors 0 dropped 5 overruns 0 carrier 0 collisions 0

venet0:0: flags=211<UP,BROADCAST,POINTOPOINT,RUNNING,NOARP> mtu 1500 inet 8x.2xx.xxx.xxx netmask 255.255.255.255 broadcast 8x.2xx.xxx.xxx destination 8x.2xx.xxx.xxx unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 txqueuelen 0 (UNSPEC)

sudo ip link 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: venet0: <BROADCAST,POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default link/void`

This is a VPS, a Virtual Privat Server hosted by a provider, a kernel-based virtual machine (KVM). It shares the kernel with the underlying host.

So, ... end of my attempt? I guess your solution is based on the mac address....

autinerd commented 4 months ago

Thank you very much! Yes, currently the unique ID is based on the MAC address, but so I will need to implement a fallback.

CinciRyan commented 4 months ago

I used my linux username - not a webmin user. I'm not at home at the moment, but I do not think my account has any special permissions because I sudo for everything admin-wise.

issue-triage-workflows[bot] commented 1 month ago

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

Chris-126 commented 1 month ago

This is still not working in current version of Home Assistant. So, keeping this open.