Radarr Integration loads without error but all sensors Unavailable

[DavidFW1960]

The problem

As per the title the Radarr Integration is not giving any errors and is loading but all sensors show as unavailable

What is version of Home Assistant Core has the issue?

2021.6.x

What was the last working version of Home Assistant Core?

one of the 2021.6 betas and certainly 2021.5.5

What type of installation are you running?

Home Assistant Supervised

Integration causing the issue

Radarr

Link to integration documentation on our website

https://www.home-assistant.io/integrations/radarr/

Example YAML snippet

sensor:
  - platform: radarr
    api_key: !secret my_radarr_api
    host: !secret my_radarr_host
    port: !secret my_radarr_port
    monitored_conditions:
      - movies
      - upcoming
      - commands
      - status
    days: 10

### Anything in the logs that might be useful for us?

```txt
nothing at all

Additional information

No response

[DavidFW1960]

So I know what the problem is. They have removed the old API endpoint in version 4 nightlies and now /v3 is needed. I tried making a custom_component and editing the endpoints:

ENDPOINTS = {
    "diskspace": "{0}://{1}:{2}/{3}api/v3/diskspace",
    "upcoming": "{0}://{1}:{2}/{3}api/v3/calendar?start={4}&end={5}",
    "movies": "{0}://{1}:{2}/{3}api/v3/movie",
    "commands": "{0}://{1}:{2}/{3}api/v3/command",
    "status": "{0}://{1}:{2}/{3}api/v3/system/status",

I then get errors so seems like there is something else in core?:

Logger: homeassistant.components.sensor
Source: custom_components/radarr/sensor.py:166
Integration: Sensor (documentation, issues)
First occurred: 3:51:29 PM (2 occurrences)
Last logged: 3:51:30 PM

Error adding entities for domain sensor with platform radarr
Error while setting up radarr platform for sensor
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 383, in async_add_entities
    await asyncio.gather(*tasks)
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 588, in _async_add_entity
    await entity.add_to_platform_finish()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 615, in add_to_platform_finish
    self.async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 368, in async_write_ha_state
    self._async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 405, in _async_write_ha_state
    extra_state_attributes = self.extra_state_attributes
  File "/config/custom_components/radarr/sensor.py", line 166, in extra_state_attributes
    attributes[to_key(movie)] = movie["downloaded"]
KeyError: 'downloaded'

[DavidFW1960]

Radarr support say the old API has been deprecated for some time and the new API will work with all current supported releases of Radarr - current v3 stable as well as the V4 nightlies.

[mcevansw]

Same issue here, number 3 since last nights update, will follow up with any progress I may have

[DavidFW1960]

ok well I fixed this but I fear no way a PR will be accepted because it's using yaml still. I can't get the command condition working so if you have command in your monitored get rid of it. My config:

  - platform: radarr
    api_key: !secret my_radarr_api
    host: !secret my_radarr_host
    port: !secret my_radarr_port
    monitored_conditions:
      - movies
      - upcoming
#      - commands
      - status
    days: 10

Need to copy contents of whole folder to config/custom_components and edit manifest.json to include a version number. Then replace sensor.py with this:

"""Support for Radarr."""
from datetime import datetime, timedelta
import logging
import time

import requests
import voluptuous as vol

from homeassistant.components.sensor import PLATFORM_SCHEMA, SensorEntity
from homeassistant.const import (
    CONF_API_KEY,
    CONF_HOST,
    CONF_MONITORED_CONDITIONS,
    CONF_PORT,
    CONF_SSL,
    DATA_BYTES,
    DATA_EXABYTES,
    DATA_GIGABYTES,
    DATA_KILOBYTES,
    DATA_MEGABYTES,
    DATA_PETABYTES,
    DATA_TERABYTES,
    DATA_YOTTABYTES,
    DATA_ZETTABYTES,
    HTTP_OK,
)
import homeassistant.helpers.config_validation as cv
from homeassistant.util import dt as dt_util

_LOGGER = logging.getLogger(__name__)

CONF_DAYS = "days"
CONF_INCLUDED = "include_paths"
CONF_UNIT = "unit"
CONF_URLBASE = "urlbase"

DEFAULT_HOST = "localhost"
DEFAULT_PORT = 7878
DEFAULT_URLBASE = ""
DEFAULT_DAYS = "1"
DEFAULT_UNIT = DATA_GIGABYTES

SCAN_INTERVAL = timedelta(minutes=10)

SENSOR_TYPES = {
    "diskspace": ["Disk Space", DATA_GIGABYTES, "mdi:harddisk"],
    "upcoming": ["Upcoming", "Movies", "mdi:television"],
    "wanted": ["Wanted", "Movies", "mdi:television"],
    "movies": ["Movies", "Movies", "mdi:television"],
 #   "commands": ["Commands", "Commands", "mdi:code-braces"],
    "status": ["Status", "Status", "mdi:information"],
}

ENDPOINTS = {
    "diskspace": "{0}://{1}:{2}/{3}api/v3/diskspace",
    "upcoming": "{0}://{1}:{2}/{3}api/v3/calendar?unmonitored=false&start={4}&end={5}",
    "movies": "{0}://{1}:{2}/{3}api/v3/movie",
 #   "commands": "{0}://{1}:{2}/{3}api/v3/command",
    "status": "{0}://{1}:{2}/{3}api/v3/system/status",
}

# Support to Yottabytes for the future, why not
BYTE_SIZES = [
    DATA_BYTES,
    DATA_KILOBYTES,
    DATA_MEGABYTES,
    DATA_GIGABYTES,
    DATA_TERABYTES,
    DATA_PETABYTES,
    DATA_EXABYTES,
    DATA_ZETTABYTES,
    DATA_YOTTABYTES,
]
PLATFORM_SCHEMA = PLATFORM_SCHEMA.extend(
    {
        vol.Required(CONF_API_KEY): cv.string,
        vol.Optional(CONF_DAYS, default=DEFAULT_DAYS): cv.string,
        vol.Optional(CONF_HOST, default=DEFAULT_HOST): cv.string,
        vol.Optional(CONF_INCLUDED, default=[]): cv.ensure_list,
        vol.Optional(CONF_MONITORED_CONDITIONS, default=["movies"]): vol.All(
            cv.ensure_list, [vol.In(list(SENSOR_TYPES))]
        ),
        vol.Optional(CONF_PORT, default=DEFAULT_PORT): cv.port,
        vol.Optional(CONF_SSL, default=False): cv.boolean,
        vol.Optional(CONF_UNIT, default=DEFAULT_UNIT): vol.In(BYTE_SIZES),
        vol.Optional(CONF_URLBASE, default=DEFAULT_URLBASE): cv.string,
    }
)

def setup_platform(hass, config, add_entities, discovery_info=None):
    """Set up the Radarr platform."""
    conditions = config.get(CONF_MONITORED_CONDITIONS)
    add_entities([RadarrSensor(hass, config, sensor) for sensor in conditions], True)

class RadarrSensor(SensorEntity):
    """Implementation of the Radarr sensor."""

    def __init__(self, hass, conf, sensor_type):
        """Create Radarr entity."""

        self.conf = conf
        self.host = conf.get(CONF_HOST)
        self.port = conf.get(CONF_PORT)
        self.urlbase = conf.get(CONF_URLBASE)
        if self.urlbase:
            self.urlbase = f"{self.urlbase.strip('/')}/"
        self.apikey = conf.get(CONF_API_KEY)
        self.included = conf.get(CONF_INCLUDED)
        self.days = int(conf.get(CONF_DAYS))
        self.ssl = "https" if conf.get(CONF_SSL) else "http"
        self._state = None
        self.data = []
        self.type = sensor_type
        self._name = SENSOR_TYPES[self.type][0]
        if self.type == "diskspace":
            self._unit = conf.get(CONF_UNIT)
        else:
            self._unit = SENSOR_TYPES[self.type][1]
        self._icon = SENSOR_TYPES[self.type][2]
        self._available = False

    @property
    def name(self):
        """Return the name of the sensor."""
        return "{} {}".format("Radarr", self._name)

    @property
    def state(self):
        """Return sensor state."""
        return self._state

    @property
    def available(self):
        """Return sensor availability."""
        return self._available

    @property
    def unit_of_measurement(self):
        """Return the unit of the sensor."""
        return self._unit

    @property
    def extra_state_attributes(self):
        """Return the state attributes of the sensor."""
        attributes = {}
        if self.type == "upcoming":
            for movie in self.data:
                attributes[to_key(movie)] = get_release_date(movie)
 #       elif self.type == "commands":
 #           for command in self.data:
 #               attributes[command["name"]] = command["state"]
        elif self.type == "diskspace":
            for data in self.data:
                free_space = to_unit(data["freeSpace"], self._unit)
                total_space = to_unit(data["totalSpace"], self._unit)
                percentage_used = (
                    0 if total_space == 0 else free_space / total_space * 100
                )
                attributes[data["path"]] = "{:.2f}/{:.2f}{} ({:.2f}%)".format(
                    free_space, total_space, self._unit, percentage_used
                )
        elif self.type == "movies":
            for movie in self.data:
                attributes[to_key(movie)] = movie["hasFile"]
        elif self.type == "status":
            attributes = self.data

        return attributes

    @property
    def icon(self):
        """Return the icon of the sensor."""
        return self._icon

    def update(self):
        """Update the data for the sensor."""
        time_zone = dt_util.get_time_zone(self.hass.config.time_zone)
        start = get_date(time_zone)
        end = get_date(time_zone, self.days)
        try:
            res = requests.get(
                ENDPOINTS[self.type].format(
                    self.ssl, self.host, self.port, self.urlbase, start, end
                ),
                headers={"X-Api-Key": self.apikey},
                timeout=10,
            )
        except OSError:
            _LOGGER.warning("Host %s is not available", self.host)
            self._available = False
            self._state = None
            return

        if res.status_code == HTTP_OK:
            if self.type in ["upcoming", "movies"]:        #, "commands"]:
                self.data = res.json()
                self._state = len(self.data)
            elif self.type == "diskspace":
                # If included paths are not provided, use all data
                if self.included == []:
                    self.data = res.json()
                else:
                    # Filter to only show lists that are included
                    self.data = list(
                        filter(lambda x: x["path"] in self.included, res.json())
                    )
                self._state = "{:.2f}".format(
                    to_unit(sum([data["freeSpace"] for data in self.data]), self._unit)
                )
            elif self.type == "status":
                self.data = res.json()
                self._state = self.data["version"]
            self._available = True

def get_date(zone, offset=0):
    """Get date based on timezone and offset of days."""
    day = 60 * 60 * 24
    return datetime.date(datetime.fromtimestamp(time.time() + day * offset, tz=zone))

def get_release_date(data):
    """Get release date."""
    date = data.get("physicalRelease")
    if not date:
        date = data.get("inCinemas")
    return date

def to_key(data):
    """Get key."""
    return "{} ({})".format(data["title"], data["year"])

def to_unit(value, unit):
    """Convert bytes to give unit."""
    return value / 1024 ** BYTE_SIZES.index(unit)

OR you can copy my sensor.py and manifest.json from here: https://github.com/DavidFW1960/home-assistant/tree/master/custom_components/radarr

I also patched Maykar's upcoming media component to add api v3 (one minor change) and made a PR for it

[DavidFW1960]

@ronanmu Any chance you can take a look at and maybe fix this?

[towerhand]

Can anyone please fix this integration?

[DavidFW1960]

Can anyone please fix this integration?

Did you try the above fix? It's working for me (except for the commands sensor)

[towerhand]

I haven’t had time to mess with it but I should be able to try it in a couple weeks.

[marine1988]

I have the same bug on mine config

[tkdrob]

I wonder if people are still using radarr v2. If we can forget about v2, it will be easier to add config flow. Is anyone watching this issue working on that?

[DavidFW1960]

no one so far seems to care about this..

[tkdrob]

no one so far seems to care about this..

Except possible me. I would have to either find a good existing library or make my own.

[DavidFW1960]

Well the current integration works for old API.. It's only V3 that doesn't work and my custom hack of the core component works for v3.. just not the old commands function.

Except possible me. I would have to either find a good existing library or make my own.

Why when the hack works is a new library needed? Is that needed to use the config flow? I did look at the Sonarr component which was updated a while ago and uses the config flow but it's just beyond my skills.

I was also referring to the devs and code owner for this integration being uninterested in fixing it not people responding here...

[DeftNerd]

The Sonarr integration uses the sonarr ^1 Python library, but there isn't an identical library for Radarr which would make code copy&paste possible.

Instead, I think that building a new Radarr integration based on pyarr ^3 would be the best solution. The library has one pull request that'll cause breaking changes ^5, but once that's done, I'm considering taking up the mantle and building a new plugin myself.

What's exciting is that the pyarr library currently supports Sonarr & Radarr, but Readarr^6, Lidarr^7 and Prowlarr^8 are coming soon.

This means that a successful Home Assistant plugin for Radarr based on the pyarr library should be relatively easily modified to provide support for Sonarr, Lidarr, Readarr, and Prowlarr.

References:

[tkdrob]

I am also considering writing the radarr config flow.

[tkdrob]

The Sonarr integration uses the sonarr 12 Python library, but there isn't an identical library for Radarr which would make code copy&paste possible.

Instead, I think that building a new Radarr integration based on pyarr 34 would be the best solution. The library has one pull request that'll cause breaking changes 5, but once that's done, I'm considering taking up the mantle and building a new plugin myself.

What's exciting is that the pyarr library currently supports Sonarr & Radarr, but Readarr6, Lidarr7 and Prowlarr8 are coming soon.

This means that a successful Home Assistant plugin for Radarr based on the pyarr library should be relatively easily modified to provide support for Sonarr, Lidarr, Readarr, and Prowlarr.

References:

Footnotes

1. https://pypi.org/project/sonarr/ ↩
2. https://github.com/ctalkington/python-sonarr ↩
3. https://pypi.org/project/pyarr/ ↩
4. https://github.com/totaldebug/pyarr ↩
5. https://github.com/totaldebug/pyarr/tree/80-add_readarr_support ↩
6. Add Readarr support totaldebug/pyarr#80 ↩
7. Add Lidarr support totaldebug/pyarr#81 ↩
8. Add Prowlarr support totaldebug/pyarr#82 ↩

The issue with pyarr is it does not throw exceptions for things like authentication and its not async. We like that here. You will now see that I have a PR open with a fresh new library. 🚀

[DavidFW1960]

Well 2021.12.0b0 completely breaks the integration so good news if a new one is coming.

[tkdrob]

Its broken if you are running custom. Something was changed in the imports.

[DavidFW1960]

Yeah I was running a modified version as per above as a custom component because it didn't support the latest API

[DavidFW1960]

ok I loaded the 2 files you changed in latest 'tweak' and the integration loads with no errors when I restart HA. When I go to integrations and add Radarr it says

[DavidFW1960]

Can you confirm from this post that I made the correct modifications in the correct file?

https://github.com/home-assistant/core/pull/60847#issuecomment-987500103

[tkdrob]

You would have to share with me the folder you made.

[DavidFW1960]

It's just in config/custom_components?radarr

[tkdrob]

Check discord. I sent you my copy.

[DavidFW1960]

Check discord. I sent you my copy.

Are you by any chance also using Sabnzbd? This is erroring in 2021.12.0.x as well. I raised an issue there https://github.com/home-assistant/core/issues/61306 but it seems an upstream library is the culprit for that.

[tkdrob]

Check discord. I sent you my copy.

Are you by any chance also using Sabnzbd? This is erroring in 2021.12.0.x as well. I raised an issue there #61306 but it seems an upstream library is the culprit for that.

No. But I may look into it. I'm currently working no some new integrations.

[DavidFW1960]

Thanks. It's really weird because just using an API Call I think I can get the same data as the integration anyway by using a rest sensor so I dunno why HA is doing it the way that it currently is.

[tkdrob]

Thanks. It's really weird because just using an API Call I think I can get the same data as the integration anyway by using a rest sensor so I dunno why HA is doing it the way that it currently is.

A change has been made on how timeouts are handled. The timeout context is no longer used with aiohttp. That's why.

Actually, the loop argument is more the problem here and is deprecated.

[Archmonger]

@DeftNerd Our latest V3 release for PyArr is now live.

An upcoming release PyArr may will add async, but the API structure won't change. It's still undecided whether we'll maintain separate async/sync versions of PyArr, or just have everything be async moving forward.

[tkdrob]

@DeftNerd Our latest V3 release for PyArr is now live.

V4 of PyArr may convert the library to async, but the API structure won't change. It's still undecided whether we'll maintain separate async/sync versions of PyArr, or just have everything be async moving forward.

I actually recently made aiopyarr

[Archmonger]

@tkdrob The API is a little bit different than how I planned to structure it on totaldebug/pyarr but kudos to the fork!

Do you mind if we adopt your tests configuration into our branch? We've been putting off developing tests for PyArr for a bit too long.

Also side note, our latest pull request did bring in a few built-in exception types.

[tkdrob]

@tkdrob The API is a little bit different than how I planned to structure it on totaldebug/pyarr but kudos to the fork!

Do you mind if we adopt your tests configuration into our branch? We've been putting off developing tests for PyArr for a bit too long.

Not at all, go ahead. All I ask is for a mention. There is a bug I noticed with the host config ignoring custom ports so I'll have to fix that.

[furrymayo]

Hey, I am having a similar issue. If you guys are anywhere near getting this resolved I wouldn't mind beta testing whatever you got.

[DavidFW1960]

Hey, I am having a similar issue. If you guys are anywhere near getting this resolved I wouldn't mind beta testing whatever you got.

You can download a working version as a custom component from my repo

[tkdrob]

Hey, I am having a similar issue. If you guys are anywhere near getting this resolved I wouldn't mind beta testing whatever you got.

You can download a working version as a custom component from my repo

Keep in mind that if you use this custom component, to remove it and re-setup the integration when it is officially released. The config entries will be different.

[DavidFW1960]

Hey, I am having a similar issue. If you guys are anywhere near getting this resolved I wouldn't mind beta testing whatever you got.

You can download a working version as a custom component from my repo

Keep in mind that if you use this custom component, to remove it and re-setup the integration when it is officially released. The config entries will be different.

Yes. Thanks for pointing that out. It serves a purpose for now though and I'll keep an eye on the release notes too.

[hellcry37]

Some time ago radarr failed to work, then I just remove it. Today I've tried to se it up again, everything was correct but sensors are all unavailable.

Home Assistant Core-2022.4.1

Radarr v4.0.5.5981 server.

[DavidFW1960]

Radarr v4.0.5.5981 server.

The reason is HA is only supporting v2 API still. This radarr is using API v3.

There has been a PR to fix this and add GUI confir since many months still not merged,

[hellcry37]

yeah this is the case for many other integrations

[ghost]

The PR seems to be failing black and pip check. Do these need to be fixed before they'll accept the PR?

[hellcry37]

no idea but clearly ppl don't seem to bother to much with this

[tkdrob]

The PR seems to be failing black and pip check. Do these need to be fixed before they'll accept the PR?

The target branch for the PR is not dev, so that does not matter. It's whenever @balloob wants to review it.

no idea but clearly ppl don't seem to bother to much with this

It's quite the opposite, actually. Remember that this is volunteer work and the members have their own agendas of what needs to be done for each release. Each release is huge. Just look at the last release. It gets better and better. I prefer they stick to that plan.

Anyone is free to move anything that is being proposed into custom_components and run it that way.

[ghost]

Is there some trick I'm missing to get it to work from custom components? I cloned your radarr folder and slide it into the custom_components but it's not an option.

[DavidFW1960]

Where did you clone it from?

[github-actions[bot]]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

[DavidFW1960]

This still is not working

[saschabrockel]

Like @DavidFW1960 says it is still not working.

[marine1988]

still not working on last version

[pseudomix]

Still not Working on last version

[tkdrob]

Not until next release, but be sure to read the posted issues. A lot of changes are being made to modernize things but still a WIP.