Can't select correct electric meter for tariff comparison.

caiusseverus commented 1 week ago

Describe the bug

When setting up a new tariff comparison for electricity, the integration automatically sets up the entities for the old non-smart meter serial number. There is no way to choose the new smart meter.

For example: octopus_energy_electricity_k99e081*_10125812***_previous_accumulative_cost_comparison is what is set up, where k99e0 is the old meter serial number, and has no data associated with it. The new serial number start 24L301**** etc.

Reproduction steps

Set up a new electricity tariff comparison while there is an old non-smart meter still showing on the api.

Expected behaviour

It should be possible to choose the correct meter during set up.

Tariff Code

E-1R-AGILE-24-04-03-A

Integration Version

11.1.1

Home Assistant Version

2024.6.3

Fresh Install?

Not specified

Home Assistant Logs

{
  "home_assistant": {
    "installation_type": "Home Assistant Container",
    "version": "2024.6.3",
    "dev": false,
    "hassio": false,
    "virtualenv": false,
    "python_version": "3.12.2",
    "docker": true,
    "arch": "x86_64",
    "timezone": "Europe/London",
    "os_name": "Linux",
    "os_version": "5.15.0-112-generic",
    "run_as_root": true
  },
  "custom_components": {
    "geo_home": {
      "documentation": "https://www.home-assistant.io/integrations/geo_home",
      "version": "1.12.0",
      "requirements": []
    },
    "hacs": {
      "documentation": "https://hacs.xyz/docs/configuration/start",
      "version": "1.34.0",
      "requirements": [
        "aiogithubapi>=22.10.1"
      ]
    },
    "thermal_comfort": {
      "documentation": "https://github.com/dolezsa/thermal_comfort/blob/master/README.md",
      "version": "2.2.2",
      "requirements": []
    },
    "tplink_deco": {
      "documentation": "https://github.com/amosyuen/ha-tplink-deco",
      "version": "3.6.2",
      "requirements": [
        "pycryptodome>=3.12.0"
      ]
    },
    "tdarr": {
      "documentation": "https://github.com/itchannel/tdarr_ha",
      "version": "1.17",
      "requirements": []
    },
    "localtuya": {
      "documentation": "https://github.com/rospogrigio/localtuya/",
      "version": "5.2.1",
      "requirements": []
    },
    "smart_thermostat": {
      "documentation": "https://github.com/ScratMan/HASmartThermostat",
      "version": "2024.2.0",
      "requirements": []
    },
    "octopus_energy": {
      "documentation": "https://bottlecapdave.github.io/HomeAssistant-OctopusEnergy",
      "version": "11.1.1",
      "requirements": []
    }
  },
  "integration_manifest": {
    "domain": "octopus_energy",
    "name": "Octopus Energy",
    "codeowners": [
      "bottlecapdave"
    ],
    "config_flow": true,
    "dependencies": [
      "repairs",
      "recorder"
    ],
    "documentation": "https://bottlecapdave.github.io/HomeAssistant-OctopusEnergy",
    "homekit": {},
    "iot_class": "cloud_polling",
    "issue_tracker": "https://github.com/BottlecapDave/HomeAssistant-OctopusEnergy/issues",
    "ssdp": [],
    "version": "11.1.1",
    "zeroconf": [],
    "is_built_in": false
  },
  "setup_times": {
    "null": {
      "setup": 0.0013465899974107742
    },
    "56bd4d97ef19d78ac1ca45236281f1ef": {
      "config_entry_setup": 0.014516340917907655
    },
    "6d7117232a0f1bba2eb85e83c025f996": {
      "config_entry_setup": 0.010311831021681428
    },
    "0c169be521d23599edde0b6c01013c3c": {
      "config_entry_setup": 0.023579925997182727
    },
    "6729f88d33288a07f32ce99190963113": {
      "wait_import_platforms": -4.795867890003137,
      "wait_base_component": -0.0007888980908319354,
      "config_entry_setup": 5.420553270028904
    }
  },
  "data": {
    "id": "A-A255****",
    "octoplus_enrolled": true,
    "electricity_meter_points": [
      {
        "mpan": "**REDACTED**",
        "meters": [
          {
            "serial_number": "**REDACTED**",
            "is_export": false,
            "is_smart_meter": false,
            "device_id": null,
            "manufacturer": "GEC/ABB J11BA",
            "model": null,
            "firmware": null
          },
          {
            "serial_number": "**REDACTED**",
            "is_export": false,
            "is_smart_meter": true,
            "device_id": "**REDACTED**",
            "manufacturer": "1063 - Landis and Gyr",
            "model": "00050204",
            "firmware": "38040602"
          }
        ],
        "agreements": [
          {
            "start": "2021-09-25T23:00:00+00:00",
            "end": "2023-03-31T23:00:00+00:00",
            "tariff_code": "E-1R-AVRO-VAR-21-09-26-A",
            "product_code": "AVRO-VAR-21-09-26"
          },
          {
            "start": "2023-03-31T23:00:00+00:00",
            "end": "2024-06-16T23:00:00+00:00",
            "tariff_code": "E-1R-VAR-22-11-01-A",
            "product_code": "VAR-22-11-01"
          },
          {
            "start": "2024-06-16T23:00:00+00:00",
            "end": "2025-06-16T23:00:00+00:00",
            "tariff_code": "E-1R-AGILE-24-04-03-A",
            "product_code": "AGILE-24-04-03"
          }
        ]
      }
    ],
    "gas_meter_points": [
      {
        "mprn": "**REDACTED**",
        "meters": [
          {
            "serial_number": "**REDACTED**",
            "consumption_units": "m\u00b3",
            "is_smart_meter": true,
            "device_id": "**REDACTED**",
            "manufacturer": "1063 - Landis and Gyr",
            "model": "47720101",
            "firmware": "03035522"
          }
        ],
        "agreements": [
          {
            "start": "2021-09-25T23:00:00+00:00",
            "end": "2023-03-31T23:00:00+00:00",
            "tariff_code": "G-1R-AVRO-VAR-21-09-26-A",
            "product_code": "AVRO-VAR-21-09-26"
          },
          {
            "start": "2023-03-31T23:00:00+00:00",
            "end": null,
            "tariff_code": "G-1R-VAR-22-11-01-A",
            "product_code": "VAR-22-11-01"
          }
        ]
      }
    ]
  }
}

Confirmation

[X] I confirm that I cannot find my solution within the documentation
[X] I confirm that I cannot find my solution within the FAQ

BottlecapDave commented 1 week ago

Hello and sorry you're seeing this issue. Am I correct in assuming the integration also provides other entities (e.g. previous consumption) for your old inactive meter?

caiusseverus commented 1 week ago

Yes it does. The integration shows three meters in total - the old electric, new electric and gas.

The Landis and Gyr ones are the smart meters and the GEC is the old non-smart one. There are sensor entities for each meter, although the old meter obviously doesn't show any usage data. It still has entries for price etc.

I should add that all the other entities seem to be working fine and have appropriate data.

BottlecapDave commented 1 week ago

Ok thanks. I'll have a look at trying to filter out the inactive meters (which seem to have active from/to fields). It will be a little bit of a stab in the dark as I don't have any inactive meters, so might release a beta for you to try.

caiusseverus commented 1 week ago

OK no problem - I can test a beta when it's ready.

BottlecapDave commented 1 week ago

I've created a beta release at https://github.com/BottlecapDave/HomeAssistant-OctopusEnergy/releases/tag/v11.2.1-beta-01, which should filter out your inactive meters. The entities for your old meter(s) will still be present, but you can safely delete them.

caiusseverus commented 6 days ago

It doesn't seem to have fixed it unless I'm doing something wrong. I updated the integration with HACS and completely restarted Home Assistant. HACS now shows the beta version 11.2.1-beta-01. The integration itself though still says 11.2.0 - is that correct for the beta, or should it have updated? Not sure if I've missed a step somewhere.

Anyway assuming it has updated as it should, I get the same result as before - the only meter option is the old one, and if you complete the setup it creates entities with the old serial number still.

BottlecapDave commented 6 days ago

The integration itself though still says 11.2.0 - is that correct for the beta, or should it have updated?

Yeah I forgot to update the version in the manifest file :(

Could you please provide the diagnostic data again. It should now include active_from and active_to fields for your meters.

caiusseverus commented 5 days ago

Here is the diagnostic data - it does contain active_from and active_to, however the active_to fields do not have dates in them.

{
  "home_assistant": {
    "installation_type": "Home Assistant Container",
    "version": "2024.6.4",
    "dev": false,
    "hassio": false,
    "virtualenv": false,
    "python_version": "3.12.2",
    "docker": true,
    "arch": "x86_64",
    "timezone": "Europe/London",
    "os_name": "Linux",
    "os_version": "5.15.0-112-generic",
    "run_as_root": true
  },
  "custom_components": {
    "geo_home": {
      "documentation": "https://www.home-assistant.io/integrations/geo_home",
      "version": "1.12.0",
      "requirements": []
    },
    "hacs": {
      "documentation": "https://hacs.xyz/docs/configuration/start",
      "version": "1.34.0",
      "requirements": [
        "aiogithubapi>=22.10.1"
      ]
    },
    "thermal_comfort": {
      "documentation": "https://github.com/dolezsa/thermal_comfort/blob/master/README.md",
      "version": "2.2.2",
      "requirements": []
    },
    "tplink_deco": {
      "documentation": "https://github.com/amosyuen/ha-tplink-deco",
      "version": "3.6.2",
      "requirements": [
        "pycryptodome>=3.12.0"
      ]
    },
    "tdarr": {
      "documentation": "https://github.com/itchannel/tdarr_ha",
      "version": "1.17",
      "requirements": []
    },
    "localtuya": {
      "documentation": "https://github.com/rospogrigio/localtuya/",
      "version": "5.2.1",
      "requirements": []
    },
    "smart_thermostat": {
      "documentation": "https://github.com/ScratMan/HASmartThermostat",
      "version": "2024.2.0",
      "requirements": []
    },
    "octopus_energy": {
      "documentation": "https://bottlecapdave.github.io/HomeAssistant-OctopusEnergy",
      "version": "11.2.0",
      "requirements": []
    }
  },
  "integration_manifest": {
    "domain": "octopus_energy",
    "name": "Octopus Energy",
    "codeowners": [
      "bottlecapdave"
    ],
    "config_flow": true,
    "dependencies": [
      "repairs",
      "recorder"
    ],
    "documentation": "https://bottlecapdave.github.io/HomeAssistant-OctopusEnergy",
    "homekit": {},
    "iot_class": "cloud_polling",
    "issue_tracker": "https://github.com/BottlecapDave/HomeAssistant-OctopusEnergy/issues",
    "ssdp": [],
    "version": "11.2.0",
    "zeroconf": [],
    "is_built_in": false
  },
  "setup_times": {
    "null": {
      "setup": 0.0011432701721787453
    },
    "56bd4d97ef19d78ac1ca45236281f1ef": {
      "config_entry_setup": 0.011284935986623168
    },
    "20c7817be5c9062fbd411e13172baedd": {
      "config_entry_setup": 0.0018320700619369745
    },
    "ac04bd7f04289537df733f3ea9612fb8": {
      "config_entry_setup": 0.0017598818521946669
    },
    "6729f88d33288a07f32ce99190963113": {
      "wait_import_platforms": -6.124503059079871,
      "wait_base_component": -0.0007561598904430866,
      "config_entry_setup": 6.796012866077945
    }
  },
  "data": {
    "id": "A-A255BF23",
    "octoplus_enrolled": true,
    "electricity_meter_points": [
      {
        "mpan": "**REDACTED**",
        "meters": [
          {
            "active_from": {
              "__type": "<class 'datetime.date'>",
              "isoformat": "1999-09-23"
            },
            "active_to": null,
            "serial_number": "**REDACTED**",
            "is_export": false,
            "is_smart_meter": false,
            "device_id": null,
            "manufacturer": "GEC/ABB J11BA",
            "model": null,
            "firmware": null
          },
          {
            "active_from": {
              "__type": "<class 'datetime.date'>",
              "isoformat": "2024-06-10"
            },
            "active_to": null,
            "serial_number": "**REDACTED**",
            "is_export": false,
            "is_smart_meter": true,
            "device_id": "**REDACTED**",
            "manufacturer": "1063 - Landis and Gyr",
            "model": "00050204",
            "firmware": "38040602"
          }
        ],
        "agreements": [
          {
            "start": "2021-09-25T23:00:00+00:00",
            "end": "2023-03-31T23:00:00+00:00",
            "tariff_code": "E-1R-AVRO-VAR-21-09-26-A",
            "product_code": "AVRO-VAR-21-09-26"
          },
          {
            "start": "2023-03-31T23:00:00+00:00",
            "end": "2024-06-16T23:00:00+00:00",
            "tariff_code": "E-1R-VAR-22-11-01-A",
            "product_code": "VAR-22-11-01"
          },
          {
            "start": "2024-06-16T23:00:00+00:00",
            "end": "2025-06-16T23:00:00+00:00",
            "tariff_code": "E-1R-AGILE-24-04-03-A",
            "product_code": "AGILE-24-04-03"
          }
        ]
      }
    ],
    "gas_meter_points": [
      {
        "mprn": "**REDACTED**",
        "meters": [
          {
            "active_from": {
              "__type": "<class 'datetime.date'>",
              "isoformat": "2024-05-07"
            },
            "active_to": null,
            "serial_number": "**REDACTED**",
            "consumption_units": "m\u00b3",
            "is_smart_meter": true,
            "device_id": "**REDACTED**",
            "manufacturer": "1063 - Landis and Gyr",
            "model": "47720101",
            "firmware": "03035522"
          }
        ],
        "agreements": [
          {
            "start": "2021-09-25T23:00:00+00:00",
            "end": "2023-03-31T23:00:00+00:00",
            "tariff_code": "G-1R-AVRO-VAR-21-09-26-A",
            "product_code": "AVRO-VAR-21-09-26"
          },
          {
            "start": "2023-03-31T23:00:00+00:00",
            "end": null,
            "tariff_code": "G-1R-VAR-22-11-01-A",
            "product_code": "VAR-22-11-01"
          }
        ]
      }
    ]
  }
}

BottlecapDave commented 4 days ago

hmm that would explain why it wasn't getting filtered before when I was only requesting active meters. You might need to send an email to hello@octopus.energy or post on the forums (if you have access) stating your old meter is still reporting as active via the Octopus Energy APIs.

caiusseverus commented 4 days ago

OK I'll get in touch with them to see if they can remove it. I don't know why it still shows, as the old gas meter has gone.

caiusseverus commented 13 hours ago

I spoke to Octopus and they said that the old meter had got caught up in their system somewhere which is why it hadn't been removed. I sent them a photo of the final reading and they've issued a bill and now it no longer shows on the website. I waited a day and checked the api and it no longer shows there either.

I'm now able to create a tariff comparison and it uses the correct meter.

Happy to close this as the immediate problem is resolved - this is only likely to happen during the transition between a non-smart and smart meter, but when the process is completed the issue should no longer arise.

BottlecapDave commented 3 hours ago

Thanks for getting back and glad the issue is now solved. I'll look at sorting the meters by activeFrom date so that the more recent meters appear first. There's not a lot I can though with the "inactive" meter until OE report it being inactive though.

BottlecapDave / HomeAssistant-OctopusEnergy