Setup fails

[screenagerbe]

Version of the custom_component

HA 2024.10.1 FoxESS EM 1.9.1

Describe the bug

Integration will not load. After updating HA to v2024.10.1 the integration stopped working with the error below. I'm not able to get it up and running again

Error log

Logger: homeassistant.config_entries
Source: config_entries.py:594
First occurred: 08:40:50 (1 occurrences)
Last logged: 08:40:50

Error setting up entry FoxESS - Energy Management for foxess_em
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 594, in async_setup
    result = await component.async_setup_entry(hass, self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/foxess_em/__init__.py", line 77, in async_setup_entry
    entry.data = entry.options
    ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 443, in __setattr__
    raise AttributeError(
AttributeError: data cannot be changed directly, use async_update_entry instead

[stevie445]

Same issue

Logger: homeassistant.config_entries Source: config_entries.py:594 First occurred: 9:09:17 PM (3 occurrences) Last logged: 9:10:56 PM

Error setting up entry FoxESS - Energy Management for foxess_em Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/config_entries.py", line 594, in async_setup result = await component.async_setup_entry(hass, self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/foxess_em/init.py", line 77, in async_setup_entry entry.data = entry.options ^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/config_entries.py", line 443, in setattr raise AttributeError( AttributeError: data cannot be changed directly, use async_update_entry instead

[andy-hawkes-deltatre]

I am also seeing the same issue.

`Logger: homeassistant.config_entries Source: config_entries.py:594 First occurred: 16:13:05 (1 occurrences) Last logged: 16:13:05

Error setting up entry FoxESS - Energy Management for foxess_em Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/config_entries.py", line 594, in async_setup result = await component.async_setup_entry(hass, self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/foxess_em/init.py", line 77, in async_setup_entry entry.data = entry.options ^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/config_entries.py", line 443, in setattr raise AttributeError( AttributeError: data cannot be changed directly, use async_update_entry instead`

[FozzieUK]

I'm afraid I don't use this anymore and difficult for me to test - but I believe these changes would work, if one of you would be able to make these changes and try on HA 2024.10.x

edit the file /homeassistant/custom_components/foxess_em/__init__.py

add this at line 11

import copy

so that it looks like this

import asyncio
import logging
from datetime import time
import copy

from custom_components.foxess_em.battery.schedule import Schedule

then find the line that starts with this def async def async_setup_entry(hass: HomeAssistant, entry: ConfigEntry): (around line 67)

and replace the whole function with this code.

async def async_setup_entry(hass: HomeAssistant, entry: ConfigEntry):
    """Set up this integration using UI."""
    if hass.data.get(DOMAIN) is None:
        hass.data.setdefault(DOMAIN, {})
        _LOGGER.info(STARTUP_MESSAGE)

    # Check FoxESS API Key before settings up the other platforms to prevent those being setup if it
    # fails, which will prevent setup working when the FoxESS API key is fixed.
    entry_options = copy.deepcopy(dict(entry.options))
    entry_data = copy.deepcopy(dict(entry.data))
    _LOGGER.debug(f"Options {entry.options}")
    _LOGGER.debug(f"Data {entry.data}")

    if entry_options != entry_data:
        # overwrite data with options
        entry_data = copy.deepcopy(dict(entry.options))
        _LOGGER.info("Config has been updated")
    _LOGGER.debug(f"_Data {entry_data}")

    connection_type = entry_data.get(CONNECTION_TYPE, FOX_MODBUS_TCP)
    fox_api_key = entry_data.get(FOX_API_KEY)
    if connection_type == FOX_CLOUD:
        if not fox_api_key:
            raise ConfigEntryAuthFailed(
                "FoxESSCloud must now be accessed vi API Keys. Please reconfigure."
            )

    for platform in PLATFORMS:
        if entry_options.get(platform, True):
            hass.async_add_job(
                hass.config_entries.async_forward_entry_setup(entry, platform)
            )

    solcast_api_key = entry_data.get(SOLCAST_API_KEY)

    eco_start_time = time.fromisoformat(entry_data.get(ECO_START_TIME))
    eco_end_time = time.fromisoformat(entry_data.get(ECO_END_TIME))
    house_power = entry_data.get(HOUSE_POWER)
    battery_soc = entry_data.get(BATTERY_SOC)
    aux_power = entry_data.get(AUX_POWER)
    user_min_soc = entry_data.get(MIN_SOC)
    capacity = entry_data.get(BATTERY_CAPACITY)
    dawn_buffer = entry_data.get(DAWN_BUFFER)
    day_buffer = entry_data.get(DAY_BUFFER)
    # Added for 1.6.1
    charge_amps = entry_data.get(CHARGE_AMPS, 18)
    battery_volts = entry_data.get(BATTERY_VOLTS, 208)
    # Added for 1.7.0
    fox_modbus_host = entry_data.get(FOX_MODBUS_HOST, "")
    fox_modbus_port = entry_data.get(FOX_MODBUS_PORT, 502)
    fox_modbus_slave = entry_data.get(FOX_MODBUS_SLAVE, 247)

    session = async_get_clientsession(hass)
    solcast_client = SolcastApiClient(solcast_api_key, SOLCAST_URL, session)

    # Initialise controllers and services
    peak_utils = PeakPeriodUtils(eco_start_time, eco_end_time)

    forecast_controller = ForecastController(hass, solcast_client)
    average_controller = AverageController(
        hass, eco_start_time, eco_end_time, house_power, aux_power
    )
    schedule = Schedule(hass)
    battery_controller = BatteryController(
        hass,
        forecast_controller,
        average_controller,
        user_min_soc,
        capacity,
        dawn_buffer,
        day_buffer,
        eco_start_time,
        battery_soc,
        schedule,
        peak_utils,
    )

    _LOGGER.debug(f"Initialising {connection_type} service")
    if connection_type == FOX_CLOUD:
        cloud_client = FoxCloudApiClient(session, fox_api_key)
        fox_service = FoxCloudService(
            hass, cloud_client, eco_start_time, eco_end_time, user_min_soc
        )
    else:
        params = {CONNECTION_TYPE: connection_type}
        if connection_type == FOX_MODBUS_TCP:
            params.update({"host": fox_modbus_host, "port": fox_modbus_port})
        else:
            params.update({"port": fox_modbus_host, "baudrate": 9600})
        modbus_client = FoxModbus(hass, params)
        fox_service = FoxModbuservice(
            hass,
            modbus_client,
            fox_modbus_slave,
            eco_start_time,
            eco_end_time,
            user_min_soc,
        )

    charge_service = ChargeService(
        hass,
        battery_controller,
        forecast_controller,
        fox_service,
        peak_utils,
        eco_start_time,
        eco_end_time,
        battery_soc,
        user_min_soc,
        charge_amps,
        battery_volts,
    )

    hass.data[DOMAIN][entry.entry_id] = {
        "controllers": {
            "average": average_controller,
            "battery": battery_controller,
            "forecast": forecast_controller,
            "charge": charge_service,
        },
        "config": {
            "connection": (
                Connection.MODBUS
                if (connection_type in (FOX_MODBUS_TCP, FOX_MODBUS_SERIAL))
                else Connection.CLOUD
            )
        },
    }

    # Add callbacks into battery controller for updates
    forecast_controller.add_update_listener(battery_controller)
    average_controller.add_update_listener(battery_controller)

    hass.services.async_register(
        DOMAIN, "start_force_charge_now", fox_service.start_force_charge_now
    )
    hass.services.async_register(
        DOMAIN, "start_force_charge_off_peak", fox_service.start_force_charge_off_peak
    )
    hass.services.async_register(
        DOMAIN, "stop_force_charge", fox_service.stop_force_charge
    )
    hass.services.async_register(
        DOMAIN, "clear_schedule", battery_controller.clear_schedule
    )

    hass.data[DOMAIN][entry.entry_id]["unload"] = entry.add_update_listener(
        async_reload_entry
    )

    return True

[screenagerbe]

I'm afraid I don't use this anymore and difficult for me to test - but I believe these changes would work, if one of you would be able to make these changes and try on HA 2024.10.x

Thank you!! The integrations is loading again.

There are however a new warning and a new error:

Logger: homeassistant.helpers.frame
Source: helpers/frame.py:155
First occurred: 20:18:05 (4 occurrences)
Last logged: 20:18:05

Detected code that calls async_forward_entry_setup for integration, foxess_em with title: FoxESS - Energy Management and entry_id: 710[.......]65, which is deprecated and will stop working in Home Assistant 2025.6, await async_forward_entry_setups instead. Please report this issue.

and

This error originated from a custom integration.

Logger: custom_components.foxess_em.forecast.forecast_controller
Source: custom_components/foxess_em/forecast/forecast_controller.py:180
integration: FoxESS - Energy Management (documentation, issues)
First occurred: 20:18:22 (1 occurrences)
Last logged: 20:18:22

TypeError("'NoneType' object is not subscriptable")

[FozzieUK]

There are quite a few deprecation warnings now, it looks like the integration needs a good deal of housekeeping over the next 3 -6 months but it will keep going.

The second error is a problem, it looks like there is a problem in getting a response from the Solcast api - i’ll take a look as soon as I can.

[hawkcapl]

Hi

I’ve seen this as another possible HACs for battery and solar prediction https://springfall2008.github.io/batpred/

Not had chance to look at it but looks interesting.

cheers

[andyhawkes]

I'm afraid I don't use this anymore and difficult for me to test - but I believe these changes would work, if one of you would be able to make these changes and try on HA 2024.10.x

edit the file /homeassistant/custom_components/foxess_em/__init__.py

add this at line 11

import copy

so that it looks like this

import asyncio
import logging
from datetime import time
import copy

from custom_components.foxess_em.battery.schedule import Schedule

then find the line that starts with this def async def async_setup_entry(hass: HomeAssistant, entry: ConfigEntry): (around line 67)

and replace the whole function with this code.

async def async_setup_entry(hass: HomeAssistant, entry: ConfigEntry):
    """Set up this integration using UI."""
    if hass.data.get(DOMAIN) is None:
        hass.data.setdefault(DOMAIN, {})
        _LOGGER.info(STARTUP_MESSAGE)

    # Check FoxESS API Key before settings up the other platforms to prevent those being setup if it
    # fails, which will prevent setup working when the FoxESS API key is fixed.
    entry_options = copy.deepcopy(dict(entry.options))
    entry_data = copy.deepcopy(dict(entry.data))
    _LOGGER.debug(f"Options {entry.options}")
    _LOGGER.debug(f"Data {entry.data}")

    if entry_options != entry_data:
        # overwrite data with options
        entry_data = copy.deepcopy(dict(entry.options))
        _LOGGER.info("Config has been updated")
    _LOGGER.debug(f"_Data {entry_data}")

    connection_type = entry_data.get(CONNECTION_TYPE, FOX_MODBUS_TCP)
    fox_api_key = entry_data.get(FOX_API_KEY)
    if connection_type == FOX_CLOUD:
        if not fox_api_key:
            raise ConfigEntryAuthFailed(
                "FoxESSCloud must now be accessed vi API Keys. Please reconfigure."
            )

    for platform in PLATFORMS:
        if entry_options.get(platform, True):
            hass.async_add_job(
                hass.config_entries.async_forward_entry_setup(entry, platform)
            )

    solcast_api_key = entry_data.get(SOLCAST_API_KEY)

    eco_start_time = time.fromisoformat(entry_data.get(ECO_START_TIME))
    eco_end_time = time.fromisoformat(entry_data.get(ECO_END_TIME))
    house_power = entry_data.get(HOUSE_POWER)
    battery_soc = entry_data.get(BATTERY_SOC)
    aux_power = entry_data.get(AUX_POWER)
    user_min_soc = entry_data.get(MIN_SOC)
    capacity = entry_data.get(BATTERY_CAPACITY)
    dawn_buffer = entry_data.get(DAWN_BUFFER)
    day_buffer = entry_data.get(DAY_BUFFER)
    # Added for 1.6.1
    charge_amps = entry_data.get(CHARGE_AMPS, 18)
    battery_volts = entry_data.get(BATTERY_VOLTS, 208)
    # Added for 1.7.0
    fox_modbus_host = entry_data.get(FOX_MODBUS_HOST, "")
    fox_modbus_port = entry_data.get(FOX_MODBUS_PORT, 502)
    fox_modbus_slave = entry_data.get(FOX_MODBUS_SLAVE, 247)

    session = async_get_clientsession(hass)
    solcast_client = SolcastApiClient(solcast_api_key, SOLCAST_URL, session)

    # Initialise controllers and services
    peak_utils = PeakPeriodUtils(eco_start_time, eco_end_time)

    forecast_controller = ForecastController(hass, solcast_client)
    average_controller = AverageController(
        hass, eco_start_time, eco_end_time, house_power, aux_power
    )
    schedule = Schedule(hass)
    battery_controller = BatteryController(
        hass,
        forecast_controller,
        average_controller,
        user_min_soc,
        capacity,
        dawn_buffer,
        day_buffer,
        eco_start_time,
        battery_soc,
        schedule,
        peak_utils,
    )

    _LOGGER.debug(f"Initialising {connection_type} service")
    if connection_type == FOX_CLOUD:
        cloud_client = FoxCloudApiClient(session, fox_api_key)
        fox_service = FoxCloudService(
            hass, cloud_client, eco_start_time, eco_end_time, user_min_soc
        )
    else:
        params = {CONNECTION_TYPE: connection_type}
        if connection_type == FOX_MODBUS_TCP:
            params.update({"host": fox_modbus_host, "port": fox_modbus_port})
        else:
            params.update({"port": fox_modbus_host, "baudrate": 9600})
        modbus_client = FoxModbus(hass, params)
        fox_service = FoxModbuservice(
            hass,
            modbus_client,
            fox_modbus_slave,
            eco_start_time,
            eco_end_time,
            user_min_soc,
        )

    charge_service = ChargeService(
        hass,
        battery_controller,
        forecast_controller,
        fox_service,
        peak_utils,
        eco_start_time,
        eco_end_time,
        battery_soc,
        user_min_soc,
        charge_amps,
        battery_volts,
    )

    hass.data[DOMAIN][entry.entry_id] = {
        "controllers": {
            "average": average_controller,
            "battery": battery_controller,
            "forecast": forecast_controller,
            "charge": charge_service,
        },
        "config": {
            "connection": (
                Connection.MODBUS
                if (connection_type in (FOX_MODBUS_TCP, FOX_MODBUS_SERIAL))
                else Connection.CLOUD
            )
        },
    }

    # Add callbacks into battery controller for updates
    forecast_controller.add_update_listener(battery_controller)
    average_controller.add_update_listener(battery_controller)

    hass.services.async_register(
        DOMAIN, "start_force_charge_now", fox_service.start_force_charge_now
    )
    hass.services.async_register(
        DOMAIN, "start_force_charge_off_peak", fox_service.start_force_charge_off_peak
    )
    hass.services.async_register(
        DOMAIN, "stop_force_charge", fox_service.stop_force_charge
    )
    hass.services.async_register(
        DOMAIN, "clear_schedule", battery_controller.clear_schedule
    )

    hass.data[DOMAIN][entry.entry_id]["unload"] = entry.add_update_listener(
        async_reload_entry
    )

    return True

That seems to fix the loading issue, but it may have introduced a new problem - for the last couple of nights it looks like "Min SOC on grid" has not been reset to its default value after the overnight charge period, so I've incurred some charging costs at normal tariff rather than off peak.

Not the end of the world at this time of year, and better than no energy management, but a bit annoying.

I've not definitively proved that it is tied to the updated code, but the coincidence is at least suspicious.

[FozzieUK]

That seems to fix the loading issue, but it may have introduced a new problem - for the last couple of nights it looks like "Min SOC on grid" has not been reset to its default value after the overnight charge period, so I've incurred some charging costs at normal tariff rather than off peak.

Not the end of the world at this time of year, and better than no energy management, but a bit annoying.

I've not definitively proved that it is tied to the updated code, but the coincidence is at least suspicious. Hi @andyhawkes thanks for trying

Can I just ask a couple of questions so I can test for this -

For EM control do you use Fox cloud or Modbus ? Can you check what your 'normal' minSoC is set to in your EM config Do you have any error logs around the time of your eco_end ?

I've had to make quite a few other changes in addition to the breaking change mentioned above to make it run correctly and i'm working through testing it at the moment.

[andy-hawkes-deltatre]

@FozzieUK I'm using Modbus, and minSoC is configured as 10% in the EM config.

I tried to check the logs, but I updated HA a few hours ago, so I don't have the full log from last night - if it happens again I'll try to remember to pull the logs!

I really should have got into Python years ago when I was actually a developer rather than a manager!

[andy-hawkes-deltatre]

I also noticed that the charge period is left enabled - I can't remember if it used to do that or not though!

Also, where before the EM integration would manage the charging up to the desired minSoC (but no more), last night my battery charged to 100% between 02:00 and 04:00, then stayed there until 05:00 despite the EM setting the desired minSoC on Grid to 67%.

I manually reset minSoC on Grid to 10% at 09:25 when I spotted that it was still set to 67%, but the battery was still above that (~76%) at that time.

[FozzieUK]

I also noticed that the charge period is left enabled - I can't remember if it used to do that or not though!

Also, where before the EM integration would manage the charging up to the desired minSoC (but no more), last night my battery charged to 100% between 02:00 and 04:00, then stayed there until 05:00 despite the EM setting the desired minSoC on Grid to 67%.

I manually reset minSoC on Grid to 10% at 09:25 when I spotted that it was still set to 67%, but the battery was still above that (~76%) at that time.

Ok, thanks very much for that - the fix above just makes it start up again, in theory it should work as it used to be, but there are quite a few changes in code you don't have yet that correct other warnings and errors which may affect scheduling.

It does leave the charge period enabled, but it should have reset the minSoC at eco_end

I'll do some testing on my system overnight ;)

[FozzieUK]

This is the run for my latest dev version last night, worked fine - it set the charge times, minsoc etc.. correctly - and then controlled the charge once it hit minsoc, then released the minsoc locks at eco end.

I'll keep testing over the next few nights, but all looks good.

[siobhanellis]

Also had the startup problem. Used code you provided, and now running

[andy-hawkes-deltatre]

Mine is looking OK over the last couple of nights - the minSoC reset issue may have just been a local network glitch, as the wifi is occasionally unreliable in my garage where the inverter is located. It's hard to know if it's over-charging though as EM has set minSoC to 100% for the last couple of nights anyway - autumn is definitely here!