Only the electricity contract appears, but not the gas contract.

[weto91]

When I enter the Repsol credentials in the integration, only the electricity contract appears (which works perfectly), but in my case I also have a gas contract with Repsol, and it is not listed..

[ad-ha]

Hi, @weto91

Are you able to send me the DEBUG logs from your HA instance? (be careful with personal data that might appear on the logs).

I got that data before the latest changes to the API and it was all working well, so it might be just some minor changes from Repsol that broke it. As I only have an Electricity contract I cannot check it myself.

Once I get that, I will look into it and change whatever needs to be addressed.

Thanks

[ad-ha]

BTW @weto91

You may activate the debug logs incorporating this into configuration.yaml

logger:
  default: warning
  logs:
    # Repsol Luz y Gas
    custom_components.repsolluzygas: debug

[weto91]

Hello,

Thanks for the quickly reply. I have write the lines that you have indicated in configuration.yaml (then I restarted home assistant) , but I don't know where to download the registry, or if I have to do something before download it.

I've looked in settings/system/registry, but there is no entry for repsolluzygas.

Excuse my ignorance, I've only been using Home Assistant for a couple of weeks and I still don't have all the knowledge I would like.

Thanks in advantage.

Regards.

[weto91]

Hello,

Thanks for the quickly reply. I have write the lines that you have indicated in configuration.yaml (then I restarted home assistant) , but I don't know where to download the registry, or if I have to do something before download it.

I've looked in settings/system/registry, but there is no entry for repsolluzygas.

Excuse my ignorance, I've only been using Home Assistant for a couple of weeks and I still don't have all the knowledge I would like.

Thanks in advantage.

Regards.

Ok, i found it. repsolluzygas.log

I have modified what I believe was personal information by sequences of type: {HIDDEN_SOMETHING} I don't think I've left anything out...

In the log, I do see that it accepts the GAS contract without any problem, even though it does not show it to be able to select it and see the corresponding information...

Thanks!

[ad-ha]

No worries. Let me try to help.

You have to go into Settings >> System >> Logs. Make sure to check that it's showing Home Assistant Core logs. There search for Repsol and you shall get all entries.

If you still have no results, you have a "Load Full Log" button and you can search there. Alternatively, try to reload the integration.

And check the logs again.

Also, make sure that you do not have duplicated entries for the logs on configuration.yaml

Let me know the outcome

[weto91]

No worries. Let me try to help.

You have to go into Settings >> System >> Logs. Make sure to check that it's showing Home Assistant Core logs. There search for Repsol and you shall get all entries.

If you still have no results, you have a "Load Full Log" button and you can search there. Alternatively, try to reload the integration.

And check the logs again.

Also, make sure that you do not have duplicated entries for the logs on configuration.yaml

Let me know the outcome

Yes, I found it before. I left you the log in the previous comment :)

Thank you!

[ad-ha]

Got. Thanks, @weto91

I am away on vacation, but I will try to have a look as soon as I can. Looking into the log it shall be fairly simple, but I have to delve into it once I get some spare time.

If I get a fix in the meanwhile, I will let you know and release a new version.

[weto91]

Got. Thanks, @weto91

I am away on vacation, but I will try to have a look as soon as I can. Looking into the log it shall be fairly simple, but I have to delve into it once I get some spare time.

If I get a fix in the meanwhile, I will let you know and release a new version.

Nice!,

Thanks!! I'll look forward to it!

[ad-ha]

Hi @weto91

I think I found a solution for the issue. Although, as I do not have a Gas Contract with Repsol, I cannot fetch the data to see if it works.

To make it easier, would you be able to, please, test the following code on your instance and check if trying to add the integration again (as you have the electricity already, you just need to try to add again to check the gas contract, without deleting the existing integration).

Using File Editor or Visual Studio Code on Home Assistant, you just need to replace the code of these files:

• /homeassistant/custom_components/repsolluzygas/__init__.py
• /homeassistant/custom_components/repsolluzygas/config_flow.py

Btw, if you do not have File Editor or VS Code installed in HA, you can do it from the Add-On Store: https://my.home-assistant.io/redirect/supervisor

Once you do replace the code, restart HA and try to add the integration again, to see if the Gas contract pop-up during the configuration process.

If this works I will release a new version afterwards. Just want to avoid breaking something for everyone in case it does not work for some reason.

Here is the code that you need to use:

init.py

"""Integration for Repsol Luz y Gas."""
import aiohttp
import asyncio

from homeassistant.config_entries import ConfigEntry
from homeassistant.core import HomeAssistant
from homeassistant.exceptions import ConfigEntryNotReady
from homeassistant.helpers.update_coordinator import DataUpdateCoordinator, UpdateFailed
from aiohttp.client_exceptions import ClientError

from .const import (
    DOMAIN,
    LOGGER,
    LOGIN_URL,
    CONTRACTS_URL,
    HOUSES_URL,
    INVOICES_URL,
    COSTS_URL,
    NEXT_INVOICE_URL,
    VIRTUAL_BATTERY_HISTORY_URL,
    UPDATE_INTERVAL,
    LOGIN_HEADERS,
    CONTRACTS_HEADERS,
    COOKIES_CONST,
    LOGIN_DATA,
)

PLATFORMS: list[str] = ["sensor"]

async def async_setup(hass: HomeAssistant, config: dict) -> bool:
    return True

async def async_setup_entry(hass: HomeAssistant, entry: ConfigEntry) -> bool:
    session = aiohttp.ClientSession()
    client = RepsolLuzYGasAPI(session, entry.data["username"], entry.data["password"])

    async def async_update_data_start():
        try:
            return await client.fetch_all_data()
        except Exception as e:
            raise UpdateFailed(f"Error fetching data: {e}")

    coordinator = DataUpdateCoordinator(
        hass,
        LOGGER,
        name="repsolluzygas",
        update_method=async_update_data_start,
        update_interval=UPDATE_INTERVAL,
    )

    await coordinator.async_config_entry_first_refresh()

    hass.data.setdefault(DOMAIN, {})
    hass.data[DOMAIN][entry.entry_id] = {
        "api": client,
        "coordinator": coordinator,
    }

    for platform in ["sensor"]:
        hass.async_create_task(
            hass.config_entries.async_forward_entry_setup(entry, platform)
        )

    return True

async def async_unload_entry(hass: HomeAssistant, entry: ConfigEntry) -> bool:
    """Unload a config entry."""
    unload_ok = await hass.config_entries.async_unload_platforms(entry, ["sensor"])
    if unload_ok:
        hass.data[DOMAIN].pop(entry.entry_id)
    return unload_ok

async def async_reload_entry(hass: HomeAssistant, entry: ConfigEntry) -> None:
    await async_unload_entry(hass, entry)
    await async_setup_entry(hass, entry)

async def async_update_data(client):
    async def update_data():
        """Fetch data."""
        try:
            async with asyncio.timeout(10):
                data = await client.fetch_data()
                return data
        except (ClientError, asyncio.TimeoutError) as err:
            LOGGER.error("Error fetching Repsol Luz y Gas data: %s", err)
            raise UpdateFailed(f"Error fetching data: {err}") from err

    return update_data

class RepsolLuzYGasAPI:
    """Class to communicate with Repsol Luz y Gas API."""

    def __init__(self, session: aiohttp.ClientSession, username: str, password: str):
        """Initialize."""
        self.session = session
        self.username = username
        self.password = password
        self.uid = None
        self.signature = None
        self.timestamp = None

    cookies = COOKIES_CONST.copy()

    async def async_login(self):
        """Async login to Repsol API."""
        data = LOGIN_DATA.copy()
        data.update(
            {
                "loginID": self.username,
                "password": self.password,
            }
        )

        headers = LOGIN_HEADERS.copy()

        try:
            async with self.session.post(
                LOGIN_URL, headers=headers, cookies=self.cookies, data=data
            ) as response:
                if response.status == 200:
                    data = await response.json(content_type=None)
                    LOGGER.debug(f"Response: {data}")
                    self.uid = data["userInfo"]["UID"]
                    self.signature = data["userInfo"]["UIDSignature"]
                    self.timestamp = data["userInfo"]["signatureTimestamp"]
                else:
                    LOGGER.error(f"Unexpected response status: {response.status}")
                    return None
        except Exception as e:
            LOGGER.error(f"Error during login to Repsol API: {e}")
            return None

    async def async_get_contracts(self):
        """Retrieve contracts."""
        headers = CONTRACTS_HEADERS.copy()
        headers.update(
            {
                "UID": self.uid,
                "signature": self.signature,
                "signatureTimestamp": self.timestamp,
            }
        )

        url = CONTRACTS_URL

        contracts = {"house_id": None, "information": []}

        try:
            async with asyncio.timeout(10):
                async with self.session.get(
                    url, headers=headers, cookies=self.cookies
                ) as response:
                    LOGGER.debug(f"Headers: {headers}")
                    if response.status == 200:
                        data = await response.json()

                        LOGGER.debug("Contracts Data %s", data)

                        if data:  # Check if data is not empty
                            for house in data:
                                house_id = house["code"]
                                if not contracts["house_id"]:
                                    contracts["house_id"] = house_id

                                for contract in house.get("contracts", []):
                                    info = {
                                        "contract_id": contract["code"],
                                        "contractType": contract["contractType"],
                                        "cups": contract["cups"],
                                        "active": contract["status"] == "ACTIVE",
                                    }
                                    contracts["information"].append(info)

                            LOGGER.debug("Contracts Parsed %s", contracts)
                        else:
                            LOGGER.warning("No contract data received")
                    else:
                        LOGGER.error(
                            "Failed to fetch contracts data. HTTP Status: %s",
                            response.status,
                        )
                        return None
        except Exception as e:
            LOGGER.error("Error fetching contracts data: %s", e)
            return None

        return contracts

    async def async_get_houseDetails(self, house_id):
        """Fetch house details including contracts and SVA data."""
        headers = CONTRACTS_HEADERS.copy()
        headers.update(
            {
                "UID": self.uid,
                "signature": self.signature,
                "signatureTimestamp": self.timestamp,
            }
        )

        url = HOUSES_URL.format(house_id)

        try:
            async with asyncio.timeout(10):
                async with self.session.get(
                    url, headers=headers, cookies=self.cookies
                ) as response:
                    if response.status == 200:
                        response_data = await response.json()

                        LOGGER.debug("House Data %s", response_data)
                        return response_data
                    else:
                        LOGGER.error(
                            "Failed to fetch house data. HTTP Status: %s",
                            response.status,
                        )
                        return None

        except Exception as e:
            LOGGER.error("Error fetching house data: %s", e)
            return None

    async def async_get_invoices(self, house_id, contract_id):
        """Retrieve the latest invoice for a given contract."""
        headers = CONTRACTS_HEADERS.copy()
        headers.update(
            {
                "UID": self.uid,
                "signature": self.signature,
                "signatureTimestamp": self.timestamp,
            }
        )
        url = INVOICES_URL.format(house_id, contract_id)

        try:
            async with asyncio.timeout(10):
                async with self.session.get(
                    url, headers=headers, cookies=self.cookies
                ) as response:
                    if response.status == 200:
                        response_data = await response.json()

                        LOGGER.debug("Invoices Data %s", response_data)
                        return response_data
                    else:
                        LOGGER.error(
                            "Failed to fetch invoice data. HTTP Status: %s",
                            response.status,
                        )
                        return None

        except Exception as e:
            LOGGER.error("Error fetching invoice data: %s", e)
            return None

    async def async_get_costs(self, house_id, contract_id):
        """Retrieve cost data for a given contract."""
        headers = CONTRACTS_HEADERS.copy()
        headers.update(
            {
                "UID": self.uid,
                "signature": self.signature,
                "signatureTimestamp": self.timestamp,
            }
        )
        url = COSTS_URL.format(house_id, contract_id)
        data = {
            "totalDays": 0,
            "consumption": 0,
            "amount": 0,
            "amountVariable": 0,
            "amountFixed": 0,
            "averageAmount": 0,
        }

        try:
            async with asyncio.timeout(10):
                async with self.session.get(
                    url, headers=headers, cookies=self.cookies
                ) as response:
                    if response.status == 200:
                        response_data = await response.json()

                        for var in data.keys():
                            data[var] = response_data.get(var, 0)

                        LOGGER.debug("Costs Data %s", data)
                    else:
                        LOGGER.error(
                            "Failed to fetch costs data. HTTP Status: %s",
                            response.status,
                        )
        except Exception as e:
            LOGGER.error("Error fetching costs data: %s", e)

        return data

    async def async_get_next_invoice(self, house_id, contract_id):
        """Retrieve cost data for a given contract."""
        headers = CONTRACTS_HEADERS.copy()
        headers.update(
            {
                "UID": self.uid,
                "signature": self.signature,
                "signatureTimestamp": self.timestamp,
            }
        )
        url = NEXT_INVOICE_URL.format(house_id, contract_id)
        data = {
            "amount": 0,
            "amountVariable": 0,
            "amountFixed": 0,
        }

        try:
            async with asyncio.timeout(10):
                async with self.session.get(
                    url, headers=headers, cookies=self.cookies
                ) as response:
                    if response.status == 200:
                        response_data = await response.json()

                        for var in data.keys():
                            data[var] = response_data.get(var, 0)

                        LOGGER.debug("Next Invoice Data %s", data)
                    else:
                        LOGGER.debug(
                            "Failed to fetch next invoice data. HTTP Status: %s",
                            response.status,
                        )
        except Exception as e:
            LOGGER.debug("Error fetching next invoice data: %s", e)

        return data

    def extract_sva_ids(self, house_details):
        """Extract SVA IDs from house details response."""
        sva_ids = []
        for contract in house_details.get("contracts", []):
            for sva in contract.get("sva", []):
                sva_ids.append(sva["code"])
        return sva_ids

    async def async_get_virtual_battery_history(self, house_id, contract_id):
        headers = CONTRACTS_HEADERS.copy()
        headers.update(
            {
                "UID": self.uid,
                "signature": self.signature,
                "signatureTimestamp": self.timestamp,
            }
        )
        url = VIRTUAL_BATTERY_HISTORY_URL.format(house_id, contract_id)

        try:
            async with asyncio.timeout(10):
                async with self.session.get(
                    url, headers=headers, cookies=self.cookies
                ) as response:
                    if response.status == 200:
                        response_data = await response.json()

                        LOGGER.debug("Virtual Battery History Data %s", response_data)
                        return response_data
                    else:
                        LOGGER.error(
                            "Failed to fetch Virtual Battery History data. HTTP Status: %s",
                            response.status,
                        )
                        return None

        except Exception as e:
            LOGGER.error("Error fetching Virtual Battery History data: %s", e)
            return None

    async def async_update(self):
        """Asynchronously update data from the Repsol API."""
        data = {
            "consumption": 0,
            "amount": 0,
            "amountVariable": 0,
            "amountFixed": 0,
            "averageAmount": 0,
        }

        # Log in and get contracts asynchronously
        uid, signature, timestamp = await self.async_login()
        contracts = await self.async_get_contracts(uid, signature, timestamp)

        if "information" in contracts:
            for contract in contracts["information"]:
                if not contract.get("active", False):
                    continue

                # Get costs asynchronously
                response = await self.async_get_costs(
                    uid,
                    signature,
                    timestamp,
                    contracts["house_id"],
                    contract["contract_id"],
                )
                for var in data:
                    data[var] += response.get(var, 0)

                if response.get("totalDays", 0) > 0:
                    data["totalDays"] = response["totalDays"]
                    data["averageAmount"] = round(response["averageAmount"], 2)

                nextInvoice = await self.async_get_next_invoice(
                    uid,
                    signature,
                    timestamp,
                    contracts["house_id"],
                    contract["contract_id"],
                )
                for var in data:
                    data[var] += nextInvoice.get(var, 0)

                if nextInvoice.get("amount", 0) > 0:
                    data["nextInvoiceAmount"] = nextInvoice["amount"]
                    data["nextInvoiceAmountVariable"] = nextInvoice["amountVariable"]
                    data["nextInvoiceAmountFixed"] = nextInvoice["amountFixed"]

            if len(contracts["information"]) > 0:
                # Get the last invoice asynchronously
                last_contract = contracts["information"][-1]

                invoices = await self.async_get_invoices(
                    uid,
                    signature,
                    timestamp,
                    contracts["house_id"],
                    last_contract["contract_id"],
                )
                if invoices:
                    data["lastInvoiceAmount"] = invoices[0]["amount"]
                    data["lastInvoicePaid"] = invoices[0]["status"] == "PAID"

        self.data = data
        LOGGER.debug("Sensor Data %s", self.data)

    async def fetch_all_data(self):
        """Fetch and combine all necessary data from the API."""
        try:
            await self.async_login()
            contracts_data = await self.async_get_contracts()

            if not contracts_data:
                raise Exception("Failed to fetch contracts.")

            all_data = {}
            for contract in contracts_data.get("information", []):
                house_id = contracts_data["house_id"]
                contract_id = contract["contract_id"]
                house_data = await self.async_get_houseDetails(house_id)
                invoices_data = await self.async_get_invoices(house_id, contract_id)
                costs_data = await self.async_get_costs(house_id, contract_id)
                next_invoice_data = await self.async_get_next_invoice(
                    house_id, contract_id
                )
                virtual_battery_history_data = (
                    await self.async_get_virtual_battery_history(house_id, contract_id)
                )

                all_data[contract_id] = {
                    "contracts": contract,
                    "house_data": house_data,
                    "invoices": invoices_data,
                    "costs": costs_data,
                    "nextInvoice": next_invoice_data,
                    "virtual_battery_history": virtual_battery_history_data,
                }
            LOGGER.debug("Sensor Data %s", all_data)

            return all_data

        except Exception as e:
            LOGGER.error(f"Error fetching all data: {e}")
            raise

config_flow.py

from homeassistant import config_entries, core, exceptions
from homeassistant.core import HomeAssistant
import voluptuous as vol

from .const import (
    DOMAIN,
    LOGGER,
)
from . import RepsolLuzYGasAPI

class RepsolConfigFlow(config_entries.ConfigFlow, domain=DOMAIN):
    VERSION = 1
    CONNECTION_CLASS = config_entries.CONN_CLASS_CLOUD_POLL

    async def async_step_user(self, user_input=None):
        errors = {}
        if user_input is not None:
            # Initialize the API client and attempt to login
            api = RepsolLuzYGasAPI(
                self.hass.helpers.aiohttp_client.async_get_clientsession(),
                user_input["username"],
                user_input["password"],
            )
            try:
                await api.async_login()
                # Store the API client instance for use in the next step
                self.hass.data.setdefault(DOMAIN, {})["api"] = api
                # Store credentials for potential future use
                self.hass.data[DOMAIN]["credentials"] = user_input
                return await self.async_step_contract()
            except exceptions.HomeAssistantError:
                errors["base"] = "login_failed"

        data_schema = vol.Schema(
            {
                vol.Required("username"): str,
                vol.Required("password"): str,
            }
        )

        return self.async_show_form(
            step_id="user", data_schema=data_schema, errors=errors
        )

    async def async_step_contract(self, user_input=None):
        errors = {}
        api = self.hass.data[DOMAIN]["api"]
        contracts_data = await api.async_get_contracts()

        if not isinstance(contracts_data, dict) or "information" not in contracts_data:
            LOGGER.error("Unexpected contracts data structure: %s", contracts_data)
            errors["base"] = "invalid_contracts_data"
            return self.async_show_form(step_id="user", errors=errors)

        if user_input is not None:
            selected_contract = contracts_data["information"][
                int(user_input["contract_index"])
            ]

            # Check if the contract is already configured
            existing_ids = {
                entry.data["contract_id"] for entry in self._async_current_entries()
            }
            if selected_contract["contract_id"] in existing_ids:
                errors["base"] = "already_configured"
                return self.async_show_form(
                    step_id="contract",
                    data_schema=self._get_contracts_schema(contracts_data),
                    errors=errors,
                )

            self.hass.data[DOMAIN]["contract_id"] = selected_contract["contract_id"]
            self.hass.data[DOMAIN]["house_id"] = contracts_data["house_id"]
            return self.async_create_entry(
                title="Repsol Luz y Gas",
                data={
                    **self.hass.data[DOMAIN]["credentials"],
                    "contract_id": self.hass.data[DOMAIN]["contract_id"],
                    "house_id": self.hass.data[DOMAIN]["house_id"],
                },
            )

        return self.async_show_form(
            step_id="contract",
            data_schema=self._get_contracts_schema(contracts_data),
            errors=errors,
        )

    def _get_contracts_schema(self, contracts_data):
        """Generate dynamic schema for contract selection based on available contracts."""
        return vol.Schema(
            {
                vol.Required("contract_index"): vol.In(
                    {
                        i: f'{contract["contractType"]} - {contract["cups"]}'
                        for i, contract in enumerate(contracts_data["information"])
                    }
                ),
            }
        )

[weto91]

Ok, now I can add the gas contract, but it doesn't add devices or entities to it.

Regards!

[ad-ha]

Great. So, if that now works, we can check on the sensors (I was already expecting that they could fail, after the changes from the API).

For that, I need you to send me the logs again with the info from the Gas contract. It shall be reading the invoices and all.

Without that I cannot map the sensors to the info from the server.

Can you please send the updated logs, checking that you get the gas info?

Thanks a lot for the help

[weto91]

Hi, here is the log. I think it can help you =) home-assistant_2024-08-22T16-53-18.380Z.log

[ad-ha]

Alright, let me have a look into it.

A quick question, since you hide all data, is this information related to one electricity invoice and the other with one gas invoice? Is the idContract different or is it the same?

2024-08-22 17:21:34.925 DEBUG (MainThread) [custom_components.repsolluzygas.const] Invoices Data [{'idContract': '{ID}', 'startDate': '{date}', 'endDate': '{date}', 'amount': {amount}, 'debt': 0, 'status': 'PAID', 'invoiceCode': '{code}', 'invoiceNumber': '{number}', 'subsizedConsumptionAmount': 0, 'subsizedConsumptionPercentage': 0}]
2024-08-22 17:21:34.931 DEBUG (MainThread) [custom_components.repsolluzygas.const] Invoices Data [{'idContract': '{ID}', 'startDate': '{date}', 'endDate': '{date}', 'amount': {amount}, 'debt': 0, 'status': 'PAID', 'invoiceCode': '{code}', 'invoiceNumber': '{number}', 'subsizedConsumptionAmount': 0, 'subsizedConsumptionPercentage': 0}]

Would you be able to go into the Area de Cliente website on Repsol and send me the developr tools info that appears when you look into the gas invoices and gas data?

Similar to this (in this case is an example with Electricity)

If you can post the whole structure of the response that will help a lot!

Thanks again for your time and help.

[weto91]

Those two lines are from the same contract, the electricity one.

Regarding the development tools, I use Brave, and it seems that the interface is not the same, if you tell me where I have to click... of course.

However, keep in mind that GAS does not have a connected meter, so it only allows you to view and download the latest invoices, it apparently has no more identities to show.

[weto91]

Esas dos líneas son del mismo contrato, la eléctrica.

En cuanto a las herramientas de desarrollo, utilizo Brave, y parece que la interfaz no es la misma, si me dices dónde tengo que hacer clic...

Sin embargo, tenga en cuenta que GAS no tiene un medidor conectado, por lo que solo le permite ver y descargar las últimas facturas, aparentemente no tiene más identidades para mostrar.

I think that the only identities that could be obtained are the date of the last invoice, whether or not it is paid, the total price of that invoice, fixed price and variable price.

[weto91]

I can see this data for the GAS contract in the logs:

'amount': 0, 'amountVariable': {XXXX}, 'amountFixed': {XXXX}}, 'virtual_battery_history': None}, '{XXXX}': {'contracts': {'contract_id': '{XXXX}', 'contractType': 'GAS', 'cups': '{XXXX}', 'active': True}

I think that is all can be obtained...

[ad-ha]

I use Brave, and it seems that the interface is not the same, if you tell me where I have to click... of course.

Not sure how Brave handles that nor their development tools. What I sent you was from Chrome.

My guess is that there is more data available, at least from the logs I have from when I had Gas (way before the new API changes).

If you can get the data from the website, whatever you see there, we may get the same thing from the API, I just need to map the sensor fields to the correct API mapping and endpoints.

Cheers

[ad-ha]

@weto91 were you able to get those logs and info?

Without that I cannot really move on and get this fixed, as I do not have a Gas contract. So if you can, I will highly appreciate it.

On another note, if you prefer, we can chat privately on any other platform to try to get this sorted. Let me know and we can work it out.

[weto91]

Hello, sorry. I was on vacations 😅. Can you share me your email? I think is better and faster.

[ad-ha]

Hello, sorry. I was on vacations 😅. Can you share me your email? I think is better and faster.

Sent you an email. Cheers