Invalid configuration with HA 2024.2

[ronschaeffer]

Firstly, thanks for the integration.

Describe the bug On the first reboot after installing 2024.2, a persistent notification appears stating "Invalid config The following integrations and platforms could not be set up: worldtidesinfocustom Please check your config and logs.

I've set the logger to for worldtidesinfocustom to "debug", but nothing appears in the log.

To Reproduce Steps to reproduce the behavior:

1. Install HA 2024.2

Expected behavior No change to configuration from previous recent HA versions

Desktop (please complete the following information): N/A - Persistent notification of configuration error

Smartphone (please complete the following information): N/A - Persistent notification of configuration error

Additional context

EDIT - I now see the error in the log, which is the same as reported by others:

Setup failed for custom integration 'worldtidesinfocustom': Requirements for worldtidesinfocustom not found: ['matplotlib>=3.4.2'].

[wheindl]

A further update: I was mistaken in thinking that the integration was working. It appears it may have been displaying cached images. (That would explain why the matplotlib transparency flag had no effect.) I noticed that the graphs were showing tides from two days ago (probably rendered after I'd downgraded to 2024.1.6). Check and make sure that the graphs you're seeing are updated. I am still at a loss as to how to get this to work.

[gwegner]

You are right. Same here, didn't notice it right away. Shit.

[wheindl]

On the bright side, the integration is working again, save for the Matplotlib graphing. I have not figured out a way to have the matplotlib survive the upgrades to HA 2024.2.3 (and later) and version 13 of the integration. It's been suggested that the graphing SHOULD work if matplotlib is already there; but having it installed for an earlier version of the integration isn't enough. Any ideas on how matplotlib can be (re)installed, and work with the updated integration?

And are others having the same problem?

[FortranFour]

Thanks, Jugla, for creating this useful integration and for your updates. The V12.3.3 integration began working again for me with HA Core Update 2024.2.3 (running on X86-64 Intel hardware, if it makes a difference). The only thing I use the World Tides integration for is the graph; the other data I need is available from NOAA (github.com/jshufro/home_assistant_noaa_tides).

[wheindl]

I tried to repeat FortranFour's success (RPi running HA 2024.2.4) by uninstalling V13.0 and installing V12.3.3. While this DOES bring up the config flow and let me install the information, it's not generating plots using matplotlib. I'd guessed that V12.3.3 might have installed matplotlib (unlike V13.0) but no luck with that.

[jugla]

I have tested, several version of matplotlib with HA2024.2x : all failed. My configuration is docker + raspberry 4 . There is this issue that is opened on HA side : https://github.com/home-assistant/core/issues/111732 .

In V13.0.0, if you have a matplotlib seen by HA (i.e. core version for instance), it shall work. Thanks for all your feebacks :-)

[wheindl]

Thank you for building this great integration, and for your work in trying to get this straightened out. I've noticed that there's wider discussion of matplotlib breaking other integrations.

I do have a question. What do you mean by the following?

   In V13.0.0, if you have a matplotlib seen by HA (i.e. core version for instance), it shall work.

What do you mean by a "matplotlib seen by HA"? And to what are you referring as a "core version"?

[FortranFour]

I updated to V13.0.0 and can confirm that it provides graphs and works as it should without errors on my bare metal X86-64 Intel system running HA Core Update 2024.2.5 and HA OS 12.0.

[wheindl]

Is there anything new to report here? It looks like the issue is that matplotlib doesn't work under HA 2024.2 or later on RPi hardware (which is, unfortunately, a lot of us). I have been keeping an eye on other more general posts related to matplotlib and HA. Is there any hope for a solution?

For now, I'm keeping one of my three HA instances on 2024.1, and have all of my monitored tide stations on a page on that server. That page can be loaded as a dashboard in other instances of HA; and (using remote home assistant) the data (but not the graphs) can be displayed directly in the other HA instances. This is OK for now, but not a good long term solution, as it will require keeping an HA instance on 2024.1 indefinitely.

[valerix85]

Actually on my Proxmox VM and test environment on RPi3 everything works (HA 2024.3.0)

[wheindl]

Is the Proxmox VM on separate hardware from the RPi?

[valerix85]

I said something stupid, I was seeing the image but didn't notice that the generation date is stationary. 🤦‍♂️ both servers are physical and separate, if need be I can debug it

[FortranFour]

It seems as if the beach_plot.png and beach_plot_long.png images stopped updating when I updated to HA 2024.3.0 last Wednesday (when it was released, being the first Wednesday of the month). However, the beach.png image continues to update.

[CaptInsano]

@jugla

We all really appreciate the work you have put into this integration.

Is there any update on getting matplotlib working again with this integration?

No graphs being generated with most recent integration version (v13.0.0) on Homeassistant (v2024.3.1) running on x86 hardware with HomeassistantOS (v12.1).

The graphs were really helpful for my setup, would be great to see them back!

Thanks again

[jugla]

Hello, Thanks for all your feed back. On my raspberry PI 4, matplotlib does not work with HA 2024.2.x . I will make a test on release HA 2024.3.x

[jugla]

still does not work :-(

[wheindl]

HA 2024.4 is out and still not working on my RPi system. Any progress? Are there alternatives to matplotlib? I may have to set up another HA instance (on the old software) just to act as a tide server for my other instances (what I am sort of doing now). It's unfortunate, as HA has introduced (and will continue to introduce) a lot of good features in new releases

[m-s-e]

i was also just checking in to see if there was an up date too!

[CaptInsano]

I have found a hacky fix that works for me:

My system is HomeAssistant OS running in a VM on unRAID (x86 hardware)

Like the rest of you, graphs were not working since 2024.2.

After reading a lot of similar issues from other integrations:

https://github.com/home-assistant/core/issues/109961 https://github.com/home-assistant/core/issues/111732 https://github.com/tmjo/ha-norwegianweather/issues/4 https://github.com/tmjo/ha-norwegiantide/issues/8

I realised that the problem was that matplotlib was simply not being installed on my system so I tried to install it myself.

I first logged into the HomeAssistant OS, then open a bash prompt within the homeassistant core docker container and finally manually install matplotlib myself.

After rebooting the system my graphs are back!!

Installing matplotlib in the homeassistant core container:

Confirming it installed:

Reboot & graphs back working!

For reference, my commands were:

login
docker exec -it homeassistant bash
pip install matplotlib

Unfortunately this will need to repeated on every updated of HomeAssistant core until the actual cause of the bug is fixed

[jamesakidd]

That's really cool @CaptInsano I'm going to try that tonight, will report back. I'm running OS directly installed to a Rpi.

[fjfricke]

Unfortunately, not working for me on rpi 4 with Home Assistant OS installed. Here are my logs from the terminal:

➜  ~ docker exec -it homeassistant bash
homeassistant:/config# pip install matplotlib Looking in indexes: https://pypi.org/simple, https://wheels.home-assistant.io/musllinux-index/
Collecting matplotlib
  Downloading matplotlib-3.8.4.tar.gz (35.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 35.9/35.9 MB 29.6 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [123 lines of output]
      /tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/setuptools_scm/git.py:308: UserWarning: git archive did not support describe output
        warnings.warn("git archive did not support describe output")
      /tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/setuptools_scm/git.py:327: UserWarning: unprocessed git archival found (no export subst applied)
        warnings.warn("unprocessed git archival found (no export subst applied)")
      /tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/setuptools_scm/_integration/setuptools.py:90: UserWarning: version of matplotlib already set
        warnings.warn(f"version of {dist_name} already set")

      Edit mplsetup.cfg to change the build options; suppress output with --quiet.

      BUILDING MATPLOTLIB
            python: yes [3.12.2 (main, Mar 18 2024, 23:18:19) [GCC 13.2.1 20231014]]
          platform: yes [linux]
             tests: no  [skipping due to configuration]
            macosx: no  [Mac OS-X only]

      running egg_info
      writing lib/matplotlib.egg-info/PKG-INFO
      writing dependency_links to lib/matplotlib.egg-info/dependency_links.txt
      writing requirements to lib/matplotlib.egg-info/requires.txt
      writing top-level names to lib/matplotlib.egg-info/top_level.txt
      Traceback (most recent call last):
        File "/tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/numpy/_core/__init__.py", line 23, in <module>
          from . import multiarray
        File "/tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/numpy/_core/multiarray.py", line 10, in <module>
          from . import overrides
        File "/tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/numpy/_core/overrides.py", line 8, in <module>
          from numpy._core._multiarray_umath import (
      ImportError: Error loading shared library /tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/numpy/_core/_multiarray_umath.cpython-312-aarch64-linux-musl.so: Operation not permitted

      During handling of the above exception, another exception occurred:

      Traceback (most recent call last):
        File "/tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/numpy/__init__.py", line 114, in <module>
          from numpy.__config__ import show as show_config
        File "/tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/numpy/__config__.py", line 4, in <module>
          from numpy._core._multiarray_umath import (
        File "/tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/numpy/_core/__init__.py", line 49, in <module>
          raise ImportError(msg)
      ImportError:

      IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

      Importing the numpy C-extensions failed. This error can happen for
      many reasons, often due to issues with your setup or how NumPy was
      installed.

      We have compiled some common reasons and troubleshooting tips at:

          https://numpy.org/devdocs/user/troubleshooting-importerror.html

      Please note and check the following:

        * The Python version is: Python3.12 from "/usr/local/bin/python"
        * The NumPy version is: "2.0.0rc1"

      and make sure that they are the versions you expect.
      Please carefully study the documentation linked above for further help.

      Original error was: Error loading shared library /tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/numpy/_core/_multiarray_umath.cpython-312-aarch64-linux-musl.so: Operation not permitted

      The above exception was the direct cause of the following exception:

      Traceback (most recent call last):
        File "/usr/local/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/usr/local/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/local/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 279, in <module>
        File "/tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 104, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 967, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 321, in run
          self.find_sources()
        File "/tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 329, in find_sources
          mm.run()
        File "/tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 550, in run
          self.add_defaults()
        File "/tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 588, in add_defaults
          sdist.add_defaults(self)
        File "/tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/setuptools/command/sdist.py", line 102, in add_defaults
          super().add_defaults()
        File "/tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults
          self._add_defaults_ext()
        File "/tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/sdist.py", line 335, in _add_defaults_ext
          build_ext = self.get_finalized_command('build_ext')
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 305, in get_finalized_command
          cmd_obj.ensure_finalized()
        File "/tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 111, in ensure_finalized
          self.finalize_options()
        File "<string>", line 80, in finalize_options
        File "/tmp/pip-install-ftqe4h86/matplotlib_17806e3412e54ca7b56df832bb03d3c9/setupext.py", line 403, in get_extensions
          add_numpy_flags(ext)
        File "/tmp/pip-install-ftqe4h86/matplotlib_17806e3412e54ca7b56df832bb03d3c9/setupext.py", line 511, in add_numpy_flags
          import numpy as np
        File "/tmp/pip-build-env-8q2ay9rp/overlay/lib/python3.12/site-packages/numpy/__init__.py", line 119, in <module>
          raise ImportError(msg) from e
      ImportError: Error importing numpy: you should not try to import numpy from
              its source directory; please exit the numpy source tree, and relaunch
              your python interpreter from there.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

I would appreciate your help 😄. I also find it weird that it wants to access numpy 2.0.0rc1 even though numpy 1.26.0 is installed as seen when running pip list.

[CaptInsano]

Defo looks like an issue on RPi hardware as on my system (x86 VM) the numpy version is 1.26.0 without issues.

My install (x86) has been working perfectly since reinstalling matplotlib.

I even edit the custom_components/worldtidesinfocustom/manifest.json file to re-add the matplotlib requirement and now it has survived updates to Home Assistant Core.

My edited manifest.json is as follows:

{
  "domain": "worldtidesinfocustom",
  "name": "WorldTidesInfoCustom",
  "codeowners": ["@jugla"],
  "config_flow": true,
  "documentation": "https://github.com/jugla/worldtidesinfocustom",
  "iot_class": "cloud_polling",
  "issue_tracker": "https://github.com/jugla/worldtidesinfocustom/issues",
  "requirements": ["matplotlib==3.8.4","pyworldtidesinfo==3.0.1"],
  "version" : "13.0.0"
}

[wheindl]

It looks like some have found solutions or workarounds for x86 hardware. Has anyone had any success with a solution or workaround for RPi hardware?

[wheindl]

I tried the steps for direct install of matplotlib on my RPi, and got the same error message as fjfricke. I also tried a separate install of numpy to no avail. Has anyone else found a solution, or is matplotlib (and worldtidesinfocustom) gone forever from HA installs on RPi hardware?

[jugla]

Thanks for all messages! The @CaptInsano's solution works with a valid version of matplotlib on your HW. With RPI4, i'm stuck