new feature requests - FRITZ!Smart Gateway - support for Zigbee Devices

[Peter1980FFM]

Hallo I use the - FRITZ!Smart Gateway - in Home Assistant i see, that some Devices not Fully Supportet. support for zigbee units (see chapter 1.2 in avm api docs)

If you need some Examples write me Back

[Peter1980FFM]

AVM_Zigbee_Devices.xlsx Add the Device Information from the not correct working Devices

[Peter1980FFM]

@mib1185 i just create the Json file from my Fritzbox overview.json Groups are Devices 82 - 84 Device 81 is a Window Sensor in Homeassistant Device not get the Batterie Value Device 79 is a Fritz Dect 440 Device in Home assistant are the 4 Buttons missing Device 78 is aFritz Dect 400 Button in Home Assistan it only show the Batterie not the Buttons Device 75 is a On/Off Button in Home Assistan it is not shown Device 73 is a Zigbee Repeater and not show in Homeassistant Device 69 i a Zigbee One Button and not show in Homeassitant Device 66/67 is a Open/Close Button in Home Assistan it is not shown Device 64 is a shutter in Home Assistan it is not shown in Home Assistant Batterie is Missing Devices 50/51/52 aber one power strip AVM Make 3 Devices but all work in Home Assistant Device 34 / 36 / 38 / 40 / 42 / 44 are only CCT Ligts but in Home Assistant show as Full collor Devices in Fritzbox CCT Devices 9-12 and 4-7 are 4 ch Zigbee Devices and Each Channel is a Singel Entry

I have seen in the Json File the AVM makes for Every Button on Zigbee Device a One Device in the Json file I hope i could help a Littel bit.

i have Order a Fritz DECT 350 Sensor but Hermes is not abel to Deliver so i can´t test it and a new oder is not posible its sold out yeah If you need further support contact me

[mib1185]

i've started to do some pre-work with #91, #94 and #95. there is a further PR #96 in my queue which will improve the interoperability with zigbee devices (and maybe other dect devices as well)

[Peter1980FFM]

If you need the json for the DECT 350 let me know

[mib1185]

@Peter1980FFM Are you willing and able to help with some development tests in your HA installation? If yes, I would like to provide you a custom version of the fritzbox integration which relies on the current development (open PRs above) so you could test if this will improve the compatibility with your zigbee devices. Feel free to contact me via discord (my name is mib1185)

[Peter1980FFM]

@mib1185 i dont have a discord account but yes i can help

[mib1185]

ok, than let's go with the very first step

1. create a backup !!!

2. now install the custom fritzbox component:

via HACS:

• add https://github.com/mib1185/hass_fritzbox.git as custom repository
• you should find the "Custom AVM FRITZ!SmartHome - only for development purpose" integration
• install it and restart HA

or manually:

• copy the whole custom fritzbox into your /config/custom_components folder (_should be /config/custom_components/fritzbox afterwards_)
• restart HA

---

as soon as you got the custom fritzbox component installed, please restart HA, download and provide the diagnostics data and check if there are any noticeable improvements.

expected improvements:

• missing battery values should be shown
• buttons will still not be added (because the Fritz!Box api is pull only), but the possible sensors (eq. temperature and humidity) should be shown
• same goes for repeaters - their possible sensors should be shown, but not the repeater itself

---

btw. creating a discord account lasts only about 5 minutes, has no need for using real life names and is absolutely for free and communication and sharing ob debug files is much easier/faster.

[Peter1980FFM]

I cant check it dont can bei Installed

[mib1185]

hi @Peter1980FFM is there anything in the log? usually you do not need to start any new config flow, because this custom component will just replace the HA core integrated one, so use the already existing configuration

[Peter1980FFM]

After reboot and want to be reboot and than comes the Message above. i junst switch on debug in the logs i found Home Assistant Core Error setting up integration fritzbox - received exception 09:34:40 – (FEHLER) util/package.py

2024-01-15 09:34:40.632 ERROR (MainThread) [homeassistant.bootstrap] Error setting up integration fritzbox - received exception File "/usr/local/lib/python3.11/importlib/metadata/init.py", line 563, in from_name return Distribution.from_name(distribution_name) File "/usr/local/lib/python3.11/importlib/metadata/init.py", line 565, in from_name importlib.metadata.PackageNotFoundError: No package metadata was found for git+https://github.com/mib1185/python-fritzhome.git@0.6.10.dev0 git+https://github.com/mib1185/python-fritzhome.git@0.6.10.dev0 raise InvalidRequirement(str(e)) from e git+https://github.com/mib1185/python-fritzhome.git@0.6.10.dev0 req = Requirement(urlparse(package).fragment) raise InvalidRequirement(str(e)) from e File "/usr/local/lib/python3.11/importlib/metadata/init.py", line 563, in from_name return Distribution.from_name(distribution_name) File "/usr/local/lib/python3.11/importlib/metadata/init.py", line 565, in from_name importlib.metadata.PackageNotFoundError: No package metadata was found for git+https://github.com/mib1185/python-fritzhome.git@0.6.10.dev0 git+https://github.com/mib1185/python-fritzhome.git@0.6.10.dev0 raise InvalidRequirement(str(e)) from e git+https://github.com/mib1185/python-fritzhome.git@0.6.10.dev0 req = Requirement(urlparse(package).fragment) raise InvalidRequirement(str(e)) from e

[mib1185]

@Peter1980FFM i've uploaded a new version of the custom component. would be great if you could test this new one

[Peter1980FFM]

@mib1185 Logger: homeassistant.bootstrap Source: util/package.py:48 First occurred: 19:37:50 (1 occurrences) Last logged: 19:37:50

Error setting up integration fritzbox - received exception Traceback (most recent call last): File "/usr/local/lib/python3.11/importlib/metadata/init.py", line 563, in from_name return next(cls.discover(name=name)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/util/package.py", line 40, in is_installed distribution(package) File "/usr/local/lib/python3.11/importlib/metadata/init.py", line 981, in distribution return Distribution.from_name(distribution_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/importlib/metadata/init.py", line 565, in from_name raise PackageNotFoundError(name) importlib.metadata.PackageNotFoundError: No package metadata was found for git+https://github.com/mib1185/python-fritzhome.git@0.6.10.dev0

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/packaging/requirements.py", line 35, in init parsed = _parse_requirement(requirement_string) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/packaging/_parser.py", line 64, in parse_requirement return _parse_requirement(Tokenizer(source, rules=DEFAULT_RULES)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/packaging/_parser.py", line 82, in _parse_requirement url, specifier, marker = _parse_requirement_details(tokenizer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/packaging/_parser.py", line 126, in _parse_requirement_details marker = _parse_requirement_marker( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/packaging/_parser.py", line 147, in _parse_requirement_marker tokenizer.raise_syntax_error( File "/usr/local/lib/python3.11/site-packages/packaging/_tokenizer.py", line 165, in raise_syntax_error raise ParserSyntaxError( packaging._tokenizer.ParserSyntaxError: Expected end or semicolon (after name and no valid version specifier) git+https://github.com/mib1185/python-fritzhome.git@0.6.10.dev0 ^

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/util/package.py", line 44, in is_installed req = Requirement(package) ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/packaging/requirements.py", line 37, in init raise InvalidRequirement(str(e)) from e packaging.requirements.InvalidRequirement: Expected end or semicolon (after name and no valid version specifier) git+https://github.com/mib1185/python-fritzhome.git@0.6.10.dev0 ^

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/packaging/requirements.py", line 35, in init parsed = _parse_requirement(requirement_string) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/packaging/_parser.py", line 64, in parse_requirement return _parse_requirement(Tokenizer(source, rules=DEFAULT_RULES)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/packaging/_parser.py", line 73, in _parse_requirement name_token = tokenizer.expect( ^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/packaging/_tokenizer.py", line 140, in expect raise self.raise_syntax_error(f"Expected {expected}") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/packaging/_tokenizer.py", line 165, in raise_syntax_error raise ParserSyntaxError( packaging._tokenizer.ParserSyntaxError: Expected package name at the start of dependency specifier

^

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/setup.py", line 138, in async_setup_component return await task ^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/setup.py", line 243, in _async_setup_component await async_process_deps_reqs(hass, config, integration) File "/usr/src/homeassistant/homeassistant/setup.py", line 466, in async_process_deps_reqs await requirements.async_get_integration_with_requirements( File "/usr/src/homeassistant/homeassistant/requirements.py", line 52, in async_get_integration_with_requirements return await manager.async_get_integration_with_requirements(domain) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/requirements.py", line 168, in async_get_integration_with_requirements await self._async_process_integration(integration, done) File "/usr/src/homeassistant/homeassistant/requirements.py", line 183, in _async_process_integration await self.async_process_requirements( File "/usr/src/homeassistant/homeassistant/requirements.py", line 248, in async_process_requirements await self._async_process_requirements(name, missing) File "/usr/src/homeassistant/homeassistant/requirements.py", line 280, in _async_process_requirements installed, failures = await self.hass.async_add_executor_job( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/requirements.py", line 110, in _install_requirements_if_missing if pkg_util.is_installed(req) or _install_with_retry(req, kwargs): ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/util/package.py", line 48, in is_installed req = Requirement(urlparse(package).fragment) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/packaging/requirements.py", line 37, in init raise InvalidRequirement(str(e)) from e packaging.requirements.InvalidRequirement: Expected package name at the start of dependency specifier

^

[mib1185]

this is still the old version "0.0.1.dev0" 🤔 how did you install it? if hacs, please try to search for updates, so that "0.0.1.dev1" will be shown

[Peter1980FFM]

Look not so good home-assistant_fritzbox_2024-01-15T19-14-36.194Z.log

[mib1185]

ok, next try ... a new version -> "0.0.1.dev2" ... it contains now the just fresh released 0.6.10

[Peter1980FFM]

looks a littel bit better some Device are 2 time in the List now Groups are show unknown home-assistant_fritzbox_2024-01-15T19-51-49.953Z.log

[mib1185]

Please download and provide the diagnostics data. The unknown model is expected for groups, since they do not have a model.

[mib1185]

Further please also provide screenshot from the devices detailed page (eq. both "Esszimmer Fenster unten")

[Peter1980FFM]

config_entry-fritzbox-76a34de47a331963a19f9a3241f29eef.json (7).txt

[mib1185]

mehh ... we need some kind of identifiers migration within the already existing devicees ... will work on it and come back with an updated version ^^

[mib1185]

are there any of these duplicated devices, where both has entities? were there any duplicated device before?

If you have console access (eq. via the terminal add-on or ssh) than it would be great if you could capture the actual device registry data like follows

change into /config/.storage directory

$ cd /config/.storage

get the id of the current fritzbox configuration entry

$ cat core.config_entries | jq '.data.entries[] | select(.domain == "fritzbox" and .source != "ignore") | .entry_id'
"4c042584ff6711eaa7a09baa9efee9fd"

now get all device registry entries connected to this configuration entry (replace 4c042584ff6711eaa7a09baa9efee9fd with the id from above) and provide all the output

$ cat core.device_registry | jq '.data.devices[] | select(.config_entries[0] == "4c042584ff6711eaa7a09baa9efee9fd")'

[mib1185]

ok, here we are -> new version "0.0.1.dev3" with identifiers migration logic.

the duplicated devices may remain, but all entities should now be attached to the "correct" device (usually this with a correct firmware version information). if this is the case, please delete these "empty" devices and restart HA ... they should not re-occur

[Peter1980FFM]

i can not delet the unknown devices

[mib1185]

please open the detail pages of these devices and provide some screenshots. on the top right corner of the detail page, you should find the delete option via the edit (pencil) dialog.

what ist the current overall state? are the devices and entities now as expected?

[Peter1980FFM]

Deletion not work I dont know why, just go back to my backup an Install from git again, now it looks better, some Device miss the Firmware Information but the rest looks good

[mib1185]

Could you please download and provide the diagnostics data again? and provide some screenshots of the devices with missing/wrong firmware infomration?

[Peter1980FFM]

Good Morning 3 Examples

config_entry-fritzbox-76a34de47a331963a19f9a3241f29eef.json (6).txt

[mib1185]

hi @Peter1980FFM sorry for late response, was busy these days. i've checked the last diagnostics data and the screenshots - ich think we are near to get this finished and ready 👍 Would be great if you could provide a fresh set of debug logs - because the fritzbox integration is now used as custom component the settings needs to be changed a bit like follows in your configuration.yaml:

logger:
  default: info
  logs:
    custom_components.fritzbox: debug
    pyfritzhome: debug

[Peter1980FFM]

home-assistant_fritzbox_2024-01-19T08-26-40.548Z.log Hi now the new Debug, i think i see somethingin the Debug Note, the main Device have the Firmware but the Datatpoints not

[mib1185]

the main Device have the Firmware but the Datatpoints not

correct, that's why i tried to create the DeviceInfo in the fritzbox component based on if it is the main device or a sub unit. But as long as the main device do not provide any data which are consumed for any sensor entity, the DeviceInfo (with correct firmware) is not created ... anyhow this will lead to a breaking change in HA for the fritzbox component, so i'm, working on it to make this breaking change as less impact and "user friendly" as possible.

[mib1185]

Hi @Peter1980FFM unfortunately there is no acceptable way to automatically migrate all device ids and their possible usage in automations and scripts, so it will now create so called repair issues, so that the user is informed when automations and scripts needs to be changed by the user this is now included in the new "0.0.1.dev4" version, would be great if you could test it again - if possible with one your previous backups

[Peter1980FFM]

@mib1185 got the error Logger: homeassistant.config_entries Source: config_entries.py:406 First occurred: 15:46:58 (1 occurrences) Last logged: 15:46:58

Error setting up entry 192.168.122.24 for fritzbox Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/config_entries.py", line 406, in async_setup result = await component.async_setup_entry(hass, self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/fritzbox/init.py", line 105, in async_setup_entry hass.data[DATA_INSTANCES][automation.DOMAIN] TypeError: unsupported operand type(s) for +: 'EntityComponent' and 'EntityComponent'

[mib1185]

mehhh ... forgot to upload the latest code ... new version "0.0.1.dev5" is now ready

[Peter1980FFM]

better :-) home-assistant_fritzbox_2024-01-21T15-42-12.935Z.log

[Peter1980FFM]

The gray of the switch are ok the Device is not Connect now to the Gateway christmal is Over ;-)

[Peter1980FFM]

@mib1185 i found a Error when i want to update my Automations

[mib1185]

there should be repair issues for every replaced device, which was used in any automation or script

[Peter1980FFM]

@mib1185 not Realy staple i reboot home Assist

Logger: homeassistant.config_entries Source: config_entries.py:406 First occurred: 16:54:51 (1 occurrences) Last logged: 16:54:51 home-assistant_fritzbox_2024-01-21T16-06-24.628Z.log

Error setting up entry 192.168.122.24 for fritzbox Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/config_entries.py", line 406, in async_setup result = await component.async_setup_entry(hass, self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/fritzbox/init.py", line 102, in async_setup_entry automation_component: EntityComponent[automation.AutomationEntity] = hass.data[ ^^^^^^^^^^ KeyError: 'automation'

After Manual Rebbot the Integration

[mib1185]

mehhh 😖 the automations seems not to be loaded in the first place, so the detection of issues couldn't work 🙄

[Peter1980FFM]

May the Automatic Repair is not the Solution :-( if you have more than one Entry in the Automation it only takes the Last one and Correct them but it not work i have to del it an make it new

[mib1185]

there is no automatic repair of scripts and automations - this is not possible. It will just inform the user, that there are automations and scripts ( the names of these automations and scripts are also shown) to be fixed by the user manually.

[Peter1980FFM]

thank you good to know i will whait for the final releas or is anything to test in future?

[mib1185]

unfortunately even the final release will not have any automatic fixes. nevertheless I still need to solve the last issue you observed (btw many thanks for all your testing yet 🤝). Would be great if you could do some further tests as soon as I've I new development version ready

[robertkleinschuster]

What is the current state of Zigbee device support?

I would like to use Philips Hue Switches (4 Buttons) and Motion Sensors (that include a Motion, Brightness and Temperature Sensor)

In Home Assistant Philips Lightbulbs Seem to work fine but the Switches only Show up as Battery Level Sonsors and the Motion Sensors Show up as Battery and Alarm Sensors

BTW I am a software developer myself and lerning Python at the moment, i think with a bit of an introduction to the code i could help improve support for those things but I don't quite understand how the different device classes are instaciated yet from just looking at the code on GitHub.

Maybe i can checkout the repo and step debug the code to see whats going on with those devices.

[mib1185]

Hi @robertkleinschuster the Zigbee support is basically there, but the API of the Fritzbox does not provide push updates, so every device, which needs instant updates or reactions to be proper usable (eq. buttons and motion sensors) are still not supported. This is a limitation of the http api of the Fritzbox itself, those only AVM could change it, by adding some push update mechanism to their api.