Setup failed for custom integration 'miele': Unable to import component: No module named 'flatdict'

[WyriHaximus]

The problem

Integration doesn't start

Version of miele integration having the issue?

v2024.8.1

Version of Home Assistant Core having the issue?

2024.10.3

Anything in the logs that might be useful for us?

Logger: homeassistant.setup
Source: setup.py:334
First occurred: 19:34:07 (1 occurrences)
Last logged: 19:34:07

Setup failed for custom integration 'miele': Unable to import component: No module named 'flatdict'
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/setup.py", line 334, in _async_setup_component
    component = await integration.async_get_component()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/loader.py", line 1027, in async_get_component
    self._component_future.result()
  File "/usr/src/homeassistant/homeassistant/loader.py", line 1007, in async_get_component
    comp = await self.hass.async_add_import_executor_job(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/loader.py", line 1067, in _get_component
    ComponentProtocol, importlib.import_module(self.pkg_path)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/util/loop.py", line 200, in protected_loop_func
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/config/custom_components/miele/__init__.py", line 13, in <module>
    import flatdict
ModuleNotFoundError: No module named 'flatdict'

Additional information

Restarting HA doesn't fix this - refs #279

[astrandb]

Same problem has been reported recently, though it works for many thousands of users. Is this a new install of the integration? How are you running HA? HAOS, docker, core venv?

[WyriHaximus]

Same problem has been reported recently, though it works for many thousands of users.

Whoops did I miss an existing issue?

Is this a new install of the integration?

Existing install but upgraded HA

How are you running HA? HAOS, docker, core venv?

Kubernetes so Docker

[astrandb]

Please see #568 It seems to be a problem with loading flatdict module in certain docker environments. Are there any errors or logs regarding flatdict when setting up the container? It is sometimes done in the container before you start HA.

[WyriHaximus]

It seems to be a problem with loading flatdict module in certain docker environments. Are there any errors or logs regarding flatdict when setting up the container? It is sometimes done in the container before you start HA.

Haven't found anything yet in the logs besides what I shared. Did find flatdict in the deps directory so it is there. Will dig deeper into the logs

[astrandb]

You can add these lines to your config/configuration.yaml, restart HA and see if anything more is revealed.

logger:
    default: info
    logs:
        custom_components.miele: debug
        pymiele: debug

[WyriHaximus]

Just rolled that out, the only things now I can find now are:

2024-10-23 22:12:00.250 INFO (SyncWorker_6) [homeassistant.util.package] Attempting install of flatdict==4.0.1
2024-10-23 22:12:08.819 INFO (SyncWorker_6) [homeassistant.util.package] Attempting install of pymiele==0.1.7

[astrandb]

Do you still get the stack trace as above?

[WyriHaximus]

Yes that is still there

[jeanmoulart]

Hey @WyriHaximus , got the same issue and could fix it by installing the missing libraries directly in the container

1. docker exec -it homeassistant /bin/bash (replace homeassistant by the name of your container)
2. pip install the requirements
3. Restart HA
4. Now configure Miele's integration.

Hope that works for you too.

[WyriHaximus]

Thanks for the suggestion @jeanmoulart. Tried that, also removed the integration and both flatdict and pymiele from deps but essentially the same issue persists even tho HA will install both packages. The error is now just:

2024-10-30 21:08:36.912 ERROR (MainThread) [homeassistant.config_entries] Error occurred loading flow for integration miele: No module named 'flatdict'

[astrandb]

I have seen in the logs when building and starting HA in my development container that flatdict, and a few other libraries not related to this integration, are built as wheels. To me this indicates that the wheels binary package is not prebuilt in the distribution. It works for the absolute majority of users and I have no real clue what is missing in your setup.

[WyriHaximus]

FWIW these are the install outputs: (The flatdict version also seems off tho.)

I have seen in the logs when building and starting HA in my development container that flatdict, and a few other libraries not related to this integration, are built as wheels. To me this indicates that the wheels binary package is not prebuilt in the distribution. It works for the absolute majority of users and I have no real clue what is missing in your setup.

Is there a way to add the wheels package? IIRC the only thing I've been doing is update the docker image to whichever is the newest, and update packages when HACS tells me a new version is out.

[astrandb]

I have tried to install miele in a fresh HA container without any problems - everything works as expected. Please try these commands and return the output.

# Access the container
docker exec -it homeassistant /bin/bash
# Show python version
python3 --version
#Show your installed flatdict version, if any...
pip show flatdict
#Uninstall flatdict
pip uninstall flatdict
# Install the required version manually
pip install flatdict==4.0.1

Restart HA and install Miele integration.

[WyriHaximus]

First time going into the container: Second time going into the container: Third time going in, but first time after the restart:

Whats the default install location for this? It's not /config/deps I assume at this point.

Managed to get it work by doing the pip install for both flatdict and pymiele and then installing the integration. But that only works as long as the container is up. Running the latest 2024.11.x versions for both this integration and HA. Main question is why did it get switched from using /config/deps to the global install 🤔 .

[astrandb]

This problem is not caused by this integration. It is obviously something wrong with the way requirements are loaded for custom_components. I know that there has been recent changes from pip to uv for loading and managing such requirements. There are also spurious reports from other integrations about similar problems. I suggest that you ask in the Discord chat. Here is a link to a similar issue: https://discord.com/channels/330944238910963714/554842238073700352/1305373492245565460

[WyriHaximus]

This problem is not caused by this integration. It is obviously something wrong with the way requirements are loaded for custom_components.

Agreed, and it's a nasty breaking change. Especially since HA is supposed to manage this for us.

I know that there has been recent changes from pip to uv for loading and managing such requirements. There are also spurious reports from other integrations about similar problems.

Having a similar issue with flightradar24 ye.

I suggest that you ask in the Discord chat. Here is a link to a similar issue: https://discord.com/channels/330944238910963714/554842238073700352/1305373492245565460

Thanks, I will have a look 👍 .

Do you prefer to keep this open until it's resolved at the root cause (assuming pip vs uv) or just close it?