Found device but is not active

[Scaredycrow]

I'm seeing this in the log file when I try to run this:

Found device "Living room" (serial=XXX-AU-XXXXXXXX) but is not active; skipping

Model is a Dyson Pure Heat + Cool. It's turned on and working. Discoverable and controllable via the mobile app.

My credentials are linked in the config.ini, when I leave them out or have them incorrect the service fails to start (as expected)

When browsing the prometheus metrics for this service, I see metrics only for the prometheus process eg. python_gc_objects_collected_total{generation="0"} 705.0 but none for the dyson metrics:

# HELP dyson_humidity_percent Relative humidity (percentage)
# TYPE dyson_humidity_percent gauge
# HELP dyson_temperature_celsius Ambient temperature (celsius)
# TYPE dyson_temperature_celsius gauge
# HELP dyson_volatile_organic_compounds_units Level of Volatile organic compounds
# TYPE dyson_volatile_organic_compounds_units gauge
# HELP dyson_dust_units Level of Dust
# TYPE dyson_dust_units gauge
# HELP dyson_fan_mode Current mode of the fan
# TYPE dyson_fan_mode gauge
# HELP dyson_fan_state Current running state of the fan
# TYPE dyson_fan_state gauge
# HELP dyson_fan_speed_units Current speed of fan (-1 = AUTO)
# TYPE dyson_fan_speed_units gauge
# HELP dyson_oscillation_mode Current oscillation mode
# TYPE dyson_oscillation_mode gauge
# HELP dyson_focus_mode Current focus mode
# TYPE dyson_focus_mode gauge
# HELP dyson_heat_mode Current heat mode
# TYPE dyson_heat_mode gauge
# HELP dyson_heat_state Current heat state
# TYPE dyson_heat_state gauge
# HELP dyson_heat_target_celsius Heat target temperature (celsius)
# TYPE dyson_heat_target_celsius gauge
# HELP dyson_quality_target_units Quality target for fan
# TYPE dyson_quality_target_units gauge
# HELP dyson_filter_life_seconds Remaining filter life (seconds)
# TYPE dyson_filter_life_seconds gauge

Am I missing a step in the setup?

[seanrees]

Hi there,

The 'active' flag is returned by the underlying Dyson API via libpurecool[1]. I am not actually sure what makes a device "active" versus not, my assumption was that it was a timer that would age out offline/defunct devices.

Would you mind doing an experiment? Could you modify line 221 of main.py from:

client.monitor(metrics.update)

to

client.monitor(metrics.update, only_active=False)

That will disable the check you're running into. If it works after that, I'm happy to add a command-line switch for cases like this.

Thanks for the filing the issue!

[1] https://github.com/etheralm/libpurecool/blob/e797babbcb03dd6b0a9fb9b471087ff1ad1885eb/libpurecool/dyson_device.py#L86

[Scaredycrow]

Thanks for responding.

Testing your suggestion, log output says:

INFO Reading "/etc/prometheus-dyson/config.ini"
INFO Monitoring "Living room" (serial=XXX-AU-XXXXXXXX)
WARNING Received unknown update from "Living room" (serial=XXX-AU-XXXXXXX): <class 'libpurecool.dyson_pure_state_v2.DysonPureHotCoolV2State'>; ignoring
WARNING Received unknown update from "Living room" (serial=XXX-AU-XXXXXXXX): <class 'libpurecool.dyson_pure_state_v2.DysonEnvironmentalSensorV2State'>; ignoring

The behaviour in terms of the metrics returned is still the same (only python / process metrics returned, none of the dyson metrics are populating)

Looks like maybe libpurecool is not handling the Dyson Heat + Cool unit?

[Scaredycrow]

Extra info:

• Ubuntu Server 20.04.1 virtual machine on Hyper-V (Windows 10 version)
• Python 3.8.5 (I did have to adapt some commands to be run with "python3" instead of just "python". This included PIP needing to run as PIP3. Not sure if that has an impact?

[seanrees]

Thanks for all the info!

The key hint is in this logging message:

WARNING Received unknown update from "Living room" (serial=XXX-AU-XXXXXXXX): <class 'libpurecool.dyson_pure_state_v2.DysonEnvironmentalSensorV2State'>; ignoring

The gap here is actually in the exporter. It looks like your device is a newer generation of the Pure Hot+Cool, and reports out a new format, which libpurecool calls "V2" (for version 2). My exporter doesn't handle this yet. I had a quick look and it doesn't look terribly difficult to add support, would you be willing to hang on a few days and try out an experimental branch?

[Scaredycrow]

Yeah that would be great. I'm in no specific rush, this is just for some home dashboarding.

[seanrees]

Cool, I hacked at it a bit today and it turned out to be reasonably straightforward (though there are cleanups I'd like to do :-)). Can you give the 'pure_state_v2' branch a try? (git pull, git checkout pure_state_v2)

There are 2 new flags as well:

1. --only_monitor_active_devices=false (should prevent the first error you saw, means you can revert the change from upthread)
2. --log_level=DEBUG (should provide additional debugging info in case things go awry)

Ideally if this works, you should see all metrics populated, except those with a "V1 devices only" annotation.

[Scaredycrow]

No joy I'm afraid:

python3 /opt/prometheus-dyson/bin/main.py --config /etc/prometheus-dyson/config.ini --only_monitor_active_devices=false --log_level=DEBUG                                                                                                   2021/01/07 05:05:42       INFO Starting up on port=8091
2021/01/07 05:05:42       INFO Reading "/etc/prometheus-dyson/config.ini"
2021/01/07 05:05:42      DEBUG Disabling insecure request warnings since dyson are using a self signed certificate.
2021/01/07 05:05:42      DEBUG Starting new HTTPS connection (1): appapi.cp.dyson.com:443
2021/01/07 05:05:44      DEBUG https://appapi.cp.dyson.com:443 "POST /v1/userregistration/authenticate?country=AU HTTP/1.1" 200 None
2021/01/07 05:05:44      DEBUG Starting new HTTPS connection (1): appapi.cp.dyson.com:443
2021/01/07 05:05:44      DEBUG https://appapi.cp.dyson.com:443 "GET /v1/provisioningservice/manifest HTTP/1.1" 200 2
2021/01/07 05:05:44      DEBUG Starting new HTTPS connection (1): appapi.cp.dyson.com:443
2021/01/07 05:05:45      DEBUG https://appapi.cp.dyson.com:443 "GET /v2/provisioningservice/manifest HTTP/1.1" 200 None
2021/01/07 05:05:45       INFO Found device "Living room" (serial=XXX-AU-YYYYYYYY) but is not active; skipping

leading to:

# HELP python_gc_objects_collected_total Objects collected during gc
# TYPE python_gc_objects_collected_total counter
python_gc_objects_collected_total{generation="0"} 710.0
python_gc_objects_collected_total{generation="1"} 88.0
python_gc_objects_collected_total{generation="2"} 0.0
# HELP python_gc_objects_uncollectable_total Uncollectable object found during GC
# TYPE python_gc_objects_uncollectable_total counter
python_gc_objects_uncollectable_total{generation="0"} 0.0
python_gc_objects_uncollectable_total{generation="1"} 0.0
python_gc_objects_uncollectable_total{generation="2"} 0.0
# HELP python_gc_collections_total Number of times this generation was collected
# TYPE python_gc_collections_total counter
python_gc_collections_total{generation="0"} 77.0
python_gc_collections_total{generation="1"} 7.0
python_gc_collections_total{generation="2"} 0.0
# HELP python_info Python platform information
# TYPE python_info gauge
python_info{implementation="CPython",major="3",minor="8",patchlevel="5",version="3.8.5"} 1.0
# HELP process_virtual_memory_bytes Virtual memory size in bytes.
# TYPE process_virtual_memory_bytes gauge
process_virtual_memory_bytes 2.26455552e+08
# HELP process_resident_memory_bytes Resident memory size in bytes.
# TYPE process_resident_memory_bytes gauge
process_resident_memory_bytes 3.3746944e+07
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1.60999594277e+09
# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
# TYPE process_cpu_seconds_total counter
process_cpu_seconds_total 0.14
# HELP process_open_fds Number of open file descriptors.
# TYPE process_open_fds gauge
process_open_fds 6.0
# HELP process_max_fds Maximum number of open file descriptors.
# TYPE process_max_fds gauge
process_max_fds 1024.0
# HELP dyson_humidity_percent Relative humidity (percentage)
# TYPE dyson_humidity_percent gauge
# HELP dyson_temperature_celsius Ambient temperature (celsius)
# TYPE dyson_temperature_celsius gauge
# HELP dyson_volatile_organic_compounds_units Level of Volatile organic compounds
# TYPE dyson_volatile_organic_compounds_units gauge
# HELP dyson_dust_units Level of Dust (V1 units only)
# TYPE dyson_dust_units gauge
# HELP dyson_pm25_units Level of PM2.5 particulate matter (V2 units only)
# TYPE dyson_pm25_units gauge
# HELP dyson_pm10_units Level of PM10 particulate matter (V2 units only)
# TYPE dyson_pm10_units gauge
# HELP dyson_nitrogen_oxide_units Level of nitrogen oxides (NOx, V2 units only)
# TYPE dyson_nitrogen_oxide_units gauge
# HELP dyson_fan_mode Current mode of the fan
# TYPE dyson_fan_mode gauge
# HELP dyson_fan_state Current running state of the fan
# TYPE dyson_fan_state gauge
# HELP dyson_fan_speed_units Current speed of fan (-1 = AUTO)
# TYPE dyson_fan_speed_units gauge
# HELP dyson_oscillation_mode Current oscillation mode
# TYPE dyson_oscillation_mode gauge
# HELP dyson_heat_mode Current heat mode
# TYPE dyson_heat_mode gauge
# HELP dyson_heat_state Current heat state
# TYPE dyson_heat_state gauge
# HELP dyson_heat_target_celsius Heat target temperature (celsius)
# TYPE dyson_heat_target_celsius gauge
# HELP dyson_focus_mode Current focus mode (V1 units only)
# TYPE dyson_focus_mode gauge
# HELP dyson_quality_target_units Quality target for fan (V1 units only)
# TYPE dyson_quality_target_units gauge
# HELP dyson_filter_life_seconds Remaining HEPA filter life (seconds, V1 units only)
# TYPE dyson_filter_life_seconds gauge
# HELP dyson_continuous_monitoring Monitor air quality continuously (V2 units only)
# TYPE dyson_continuous_monitoring gauge
# HELP dyson_carbon_filter_life_percent Percent remaining of carbon filter (V2 units only)
# TYPE dyson_carbon_filter_life_percent gauge
# HELP dyson_hepa_filter_life_percent Percent remaining of HEPA filter (V2 units only)
# TYPE dyson_hepa_filter_life_percent gauge
# HELP dyson_night_mode Night mode (V2 units only)
# TYPE dyson_night_mode gauge
# HELP dyson_night_mode_fan_speed_units Night mode fan speed (V2 units only)
# TYPE dyson_night_mode_fan_speed_units gauge
# HELP dyson_oscillation_angle_low_degrees Low oscillation angle (V2 units only)
# TYPE dyson_oscillation_angle_low_degrees gauge
# HELP dyson_oscillation_angle_high_degrees High oscillation angle (V2 units only)
# TYPE dyson_oscillation_angle_high_degrees gauge
# HELP dyson_front_direction_mode Airflow direction from front (V2 units only)
# TYPE dyson_front_direction_mode gauge

[seanrees]

Sorry, I messed up on the flags there for you. :-( I didn't realise when I replied last that the bool converter in Python's argparse would interpret any string as True; so even if you say "false", it's doing the opposite of what you expect.

Try: --only_include_active_devices=

In your invocation, it would look like this:

python3 /opt/prometheus-dyson/bin/main.py --config /etc/prometheus-dyson/config.ini --only_monitor_active_devices= --log_level=DEBUG

Sorry again for the confusion. That was my bad.

[Scaredycrow]

Well caught! That's fixed it. Now to play with Grafana.

# HELP python_gc_objects_collected_total Objects collected during gc
# TYPE python_gc_objects_collected_total counter
python_gc_objects_collected_total{generation="0"} 763.0
python_gc_objects_collected_total{generation="1"} 88.0
python_gc_objects_collected_total{generation="2"} 0.0
# HELP python_gc_objects_uncollectable_total Uncollectable object found during GC
# TYPE python_gc_objects_uncollectable_total counter
python_gc_objects_uncollectable_total{generation="0"} 0.0
python_gc_objects_uncollectable_total{generation="1"} 0.0
python_gc_objects_uncollectable_total{generation="2"} 0.0
# HELP python_gc_collections_total Number of times this generation was collected
# TYPE python_gc_collections_total counter
python_gc_collections_total{generation="0"} 78.0
python_gc_collections_total{generation="1"} 7.0
python_gc_collections_total{generation="2"} 0.0
# HELP python_info Python platform information
# TYPE python_info gauge
python_info{implementation="CPython",major="3",minor="8",patchlevel="5",version="3.8.5"} 1.0
# HELP process_virtual_memory_bytes Virtual memory size in bytes.
# TYPE process_virtual_memory_bytes gauge
process_virtual_memory_bytes 5.28719872e+08
# HELP process_resident_memory_bytes Resident memory size in bytes.
# TYPE process_resident_memory_bytes gauge
process_resident_memory_bytes 3.434496e+07
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1.61008788303e+09
# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
# TYPE process_cpu_seconds_total counter
process_cpu_seconds_total 0.11
# HELP process_open_fds Number of open file descriptors.
# TYPE process_open_fds gauge
process_open_fds 9.0
# HELP process_max_fds Maximum number of open file descriptors.
# TYPE process_max_fds gauge
process_max_fds 1024.0
# HELP dyson_humidity_percent Relative humidity (percentage)
# TYPE dyson_humidity_percent gauge
dyson_humidity_percent{name="Living room",serial="XXX-AU-YYYYYYYY"} 55.0
# HELP dyson_temperature_celsius Ambient temperature (celsius)
# TYPE dyson_temperature_celsius gauge
dyson_temperature_celsius{name="Living room",serial="XXX-AU-YYYYYYYY"} 24.30000000000001
# HELP dyson_volatile_organic_compounds_units Level of Volatile organic compounds
# TYPE dyson_volatile_organic_compounds_units gauge
dyson_volatile_organic_compounds_units{name="Living room",serial="XXX-AU-YYYYYYYY"} 3.0
# HELP dyson_dust_units Level of Dust (V1 units only)
# TYPE dyson_dust_units gauge
# HELP dyson_pm25_units Level of PM2.5 particulate matter (V2 units only)
# TYPE dyson_pm25_units gauge
dyson_pm25_units{name="Living room",serial="XXX-AU-YYYYYYYY"} 0.0
# HELP dyson_pm10_units Level of PM10 particulate matter (V2 units only)
# TYPE dyson_pm10_units gauge
dyson_pm10_units{name="Living room",serial="XXX-AU-YYYYYYYY"} 0.0
# HELP dyson_nitrogen_oxide_units Level of nitrogen oxides (NOx, V2 units only)
# TYPE dyson_nitrogen_oxide_units gauge
dyson_nitrogen_oxide_units{name="Living room",serial="XXX-AU-YYYYYYYY"} 7.0
# HELP dyson_fan_mode Current mode of the fan
# TYPE dyson_fan_mode gauge
dyson_fan_mode{dyson_fan_mode="AUTO",name="Living room",serial="XXX-AU-YYYYYYYY"} 1.0
dyson_fan_mode{dyson_fan_mode="FAN",name="Living room",serial="XXX-AU-YYYYYYYY"} 0.0
dyson_fan_mode{dyson_fan_mode="OFF",name="Living room",serial="XXX-AU-YYYYYYYY"} 0.0
# HELP dyson_fan_state Current running state of the fan
# TYPE dyson_fan_state gauge
dyson_fan_state{dyson_fan_state="FAN",name="Living room",serial="XXX-AU-YYYYYYYY"} 0.0
dyson_fan_state{dyson_fan_state="OFF",name="Living room",serial="XXX-AU-YYYYYYYY"} 1.0
# HELP dyson_fan_speed_units Current speed of fan (-1 = AUTO)
# TYPE dyson_fan_speed_units gauge
dyson_fan_speed_units{name="Living room",serial="XXX-AU-YYYYYYYY"} -1.0
# HELP dyson_oscillation_mode Current oscillation mode
# TYPE dyson_oscillation_mode gauge
dyson_oscillation_mode{dyson_oscillation_mode="ON",name="Living room",serial="XXX-AU-YYYYYYYY"} 0.0
dyson_oscillation_mode{dyson_oscillation_mode="OFF",name="Living room",serial="XXX-AU-YYYYYYYY"} 1.0
# HELP dyson_heat_mode Current heat mode
# TYPE dyson_heat_mode gauge
dyson_heat_mode{dyson_heat_mode="HEAT",name="Living room",serial="XXX-AU-YYYYYYYY"} 1.0
dyson_heat_mode{dyson_heat_mode="OFF",name="Living room",serial="XXX-AU-YYYYYYYY"} 0.0
# HELP dyson_heat_state Current heat state
# TYPE dyson_heat_state gauge
dyson_heat_state{dyson_heat_state="HEAT",name="Living room",serial="XXX-AU-YYYYYYYY"} 0.0
dyson_heat_state{dyson_heat_state="OFF",name="Living room",serial="XXX-AU-YYYYYYYY"} 1.0
# HELP dyson_heat_target_celsius Heat target temperature (celsius)
# TYPE dyson_heat_target_celsius gauge
dyson_heat_target_celsius{name="Living room",serial="XXX-AU-YYYYYYYY"} 24.0
# HELP dyson_focus_mode Current focus mode (V1 units only)
# TYPE dyson_focus_mode gauge
# HELP dyson_quality_target_units Quality target for fan (V1 units only)
# TYPE dyson_quality_target_units gauge
# HELP dyson_filter_life_seconds Remaining HEPA filter life (seconds, V1 units only)
# TYPE dyson_filter_life_seconds gauge
# HELP dyson_continuous_monitoring Monitor air quality continuously (V2 units only)
# TYPE dyson_continuous_monitoring gauge
dyson_continuous_monitoring{dyson_continuous_monitoring="ON",name="Living room",serial="XXX-AU-YYYYYYYY"} 1.0
dyson_continuous_monitoring{dyson_continuous_monitoring="OFF",name="Living room",serial="XXX-AU-YYYYYYYY"} 0.0
# HELP dyson_carbon_filter_life_percent Percent remaining of carbon filter (V2 units only)
# TYPE dyson_carbon_filter_life_percent gauge
dyson_carbon_filter_life_percent{name="Living room",serial="XXX-AU-YYYYYYYY"} 43.0
# HELP dyson_hepa_filter_life_percent Percent remaining of HEPA filter (V2 units only)
# TYPE dyson_hepa_filter_life_percent gauge
dyson_hepa_filter_life_percent{name="Living room",serial="XXX-AU-YYYYYYYY"} 43.0
# HELP dyson_night_mode Night mode (V2 units only)
# TYPE dyson_night_mode gauge
dyson_night_mode{dyson_night_mode="ON",name="Living room",serial="XXX-AU-YYYYYYYY"} 1.0
dyson_night_mode{dyson_night_mode="OFF",name="Living room",serial="XXX-AU-YYYYYYYY"} 0.0
# HELP dyson_night_mode_fan_speed_units Night mode fan speed (V2 units only)
# TYPE dyson_night_mode_fan_speed_units gauge
dyson_night_mode_fan_speed_units{name="Living room",serial="XXX-AU-YYYYYYYY"} 4.0
# HELP dyson_oscillation_angle_low_degrees Low oscillation angle (V2 units only)
# TYPE dyson_oscillation_angle_low_degrees gauge
dyson_oscillation_angle_low_degrees{name="Living room",serial="XXX-AU-YYYYYYYY"} 171.0
# HELP dyson_oscillation_angle_high_degrees High oscillation angle (V2 units only)
# TYPE dyson_oscillation_angle_high_degrees gauge
dyson_oscillation_angle_high_degrees{name="Living room",serial="XXX-AU-YYYYYYYY"} 216.0
# HELP dyson_front_direction_mode Airflow direction from front (V2 units only)
# TYPE dyson_front_direction_mode gauge
dyson_front_direction_mode{dyson_front_direction_mode="ON",name="Living room",serial="XXX-AU-YYYYYYYY"} 0.0
dyson_front_direction_mode{dyson_front_direction_mode="OFF",name="Living room",serial="XXX-AU-YYYYYYYY"} 1.0

[seanrees]

Great news!

OK, what I'll do is clean up the code and push it to the main branch. That'll take me a few days most likely, but the experimental branch should work just fine in the interim. :-)

[Scaredycrow]

Thanks for your responsiveness. This has turned out great.

[seanrees]

Glad to hear! My next steps: I'll make this fix a bit more proper and commit to the main branch, at which point I'll close this issue out.

If you'd like to share your Grafana dashboard JSON file, feel free to send me a pull request :) I could imagine a directory structure like this would look nice:

contrib/grafana-scaredycrow/
   dashboard.json
   screenshot.png

[Scaredycrow]

Sounds good, though I'll wait till the VOC issue is sorted out first. More than happy to contribute my dashboard as an example for v2 devices.

[seanrees]

OK -- I've just pushed a cleaner version of V2 support to the main branch. Two major changes:

• I fixed the sharp edge around --only_include_active_devices -- the flag is now: --include_inactive_devices and no longer requires the trailing equal sign :-) An updated invocation for you would look like python3 /opt/prometheus-dyson/bin/main.py --config /etc/prometheus-dyson/config.ini --include_inactive_devices --log_level=DEBUG
• A metric name change: dyson_continuous_monitoring got renamed to dyson_continuous_monitoring_mode for consistency.

That change also has a fix for #5 as well, it will scale VOC to [0,10].

If you're using the Debian build, then note the version has changed from 0.0.1 to 0.0.2.

[Scaredycrow]

Confirming this is resolved and noting that --include_inactive_devices MUST be used, at least for my device, or else no dyson metrics will be returned.

Possibility exists that this advice will apply to all "v2" devices, but that would need a large sample size!

[seanrees]

Thanks for all your help with the bug report and trying out the experimental V2 support. Going to close this out but let me know if you run into other issues.