sen5x temperature_compensation parameters do not seem to adjust reported temperature

test32443 commented 2 years ago

The problem

I have both sen55 and scd40 connected to same m5stack. I am able to apply temperature offset to scd40, see it reported in initialization logs (see below) and can see the offset applied to the readings. For sen55, even putting high values for temperature offset do not seem to produce a measurable change in temperature readings. Also, I do not see temperature compensation reported for sen55 in initialization logs (see below).

Which version of ESPHome has the issue?

2022.6.2

What type of installation are you using?

Home Assistant Add-on

Which version of Home Assistant has the issue?

2022.6.7

What platform are you using?

ESP32

Board

m5stack core

Component causing the issue

sen5x

Example YAML snippet

sensor:
  - platform: sen5x
    id: sen54
    pm_1_0:
      name: "PM1"
      id: sensor_pm1
      accuracy_decimals: 1
    pm_2_5:
      name: "PM2.5"
      id: sensor_pm25
      accuracy_decimals: 1
    pm_10_0:
      name: "PM10"
      id: sensor_pm10
      accuracy_decimals: 1
    temperature:
      name: "Temperature"
      id: sensor_temperature
      accuracy_decimals: 1
    humidity:
      name: "Humidity"
      id: sensor_humidity
      accuracy_decimals: 0
    voc:
      name: "VOC"
      id: sensor_tvoc
      algorithm_tuning:
        index_offset: 100
        learning_time_offset_hours: 72
        learning_time_gain_hours: 72
        gating_max_duration_minutes: 180
        std_initial: 50
        gain_factor: 230
    nox:
      name: "NOX"
      id: sensor_nox
      algorithm_tuning:
        index_offset: 100
        learning_time_offset_hours: 12
        learning_time_gain_hours: 12
        gating_max_duration_minutes: 180
        std_initial: 50
        gain_factor: 230
    temperature_compensation:
      offset: 20.0
      normalized_offset_slope: 1.0
      time_constant: 1
    acceleration_mode: low
    store_baseline: true
    address: 0x69
    update_interval: 10s
  - platform: scd4x
    co2:
      name: "CO2"
      id: sensor_co2
    temperature:
      name: "Temperature"
      id: sensor_temperature1
    humidity:
      name: "Humidity"
      id: sensor_humidity1
    temperature_offset: 9.0C

Anything in the logs that might be useful for us?

[14:13:35][C][sen5x:227]: sen5x:
[14:13:35][C][sen5x:228]:   Address: 0x69
[14:13:35][C][sen5x:251]:   Productname: SEN55
[14:13:35][C][sen5x:252]:   Firmware version: 2
[14:13:35][C][sen5x:253]:   Serial number 01.67.52
[14:13:35][C][sen5x:255]:   Auto auto cleaning interval 604800 seconds
[14:13:35][C][sen5x:260]:   Low RH/T acceleration mode
[14:13:35][C][sen5x:270]:   Update Interval: 10.0s
[14:13:35][C][sen5x:271]:   PM  1.0 'PM1'
[14:13:35][C][sen5x:271]:     Device Class: 'pm1'
[14:13:35][C][sen5x:271]:     State Class: 'measurement'
[14:13:35][C][sen5x:271]:     Unit of Measurement: 'µg/m³'
[14:13:35][C][sen5x:271]:     Accuracy Decimals: 1
[14:13:35][C][sen5x:271]:     Icon: 'mdi:chemical-weapon'
[14:13:35][C][sen5x:272]:   PM  2.5 'PM2.5'
[14:13:35][C][sen5x:272]:     Device Class: 'pm25'
[14:13:35][C][sen5x:272]:     State Class: 'measurement'
[14:13:35][C][sen5x:272]:     Unit of Measurement: 'µg/m³'
[14:13:35][C][sen5x:272]:     Accuracy Decimals: 1
[14:13:35][C][sen5x:272]:     Icon: 'mdi:chemical-weapon'
[14:13:35][C][sen5x:274]:   PM 10.0 'PM10'
[14:13:35][C][sen5x:274]:     Device Class: 'pm10'
[14:13:36][C][sen5x:274]:     State Class: 'measurement'
[14:13:36][C][sen5x:274]:     Unit of Measurement: 'µg/m³'
[14:13:36][C][sen5x:274]:     Accuracy Decimals: 1
[14:13:36][C][sen5x:274]:     Icon: 'mdi:chemical-weapon'
[14:13:36][C][sen5x:275]:   Temperature 'Temperature'
[14:13:36][C][sen5x:275]:     Device Class: 'temperature'
[14:13:36][C][sen5x:275]:     State Class: 'measurement'
[14:13:36][C][sen5x:275]:     Unit of Measurement: '°C'
[14:13:36][C][sen5x:275]:     Accuracy Decimals: 1
[14:13:36][C][sen5x:275]:     Icon: 'mdi:thermometer'
[14:13:36][C][sen5x:276]:   Humidity 'Humidity'
[14:13:36][C][sen5x:276]:     Device Class: 'humidity'
[14:13:36][C][sen5x:276]:     State Class: 'measurement'
[14:13:36][C][sen5x:276]:     Unit of Measurement: '%'
[14:13:36][C][sen5x:276]:     Accuracy Decimals: 0
[14:13:36][C][sen5x:276]:     Icon: 'mdi:water-percent'
[14:13:36][C][sen5x:277]:   VOC 'VOC'
[14:13:36][C][sen5x:277]:     Device Class: 'volatile_organic_compounds'
[14:13:36][C][sen5x:277]:     State Class: 'measurement'
[14:13:36][C][sen5x:277]:     Unit of Measurement: ''
[14:13:36][C][sen5x:277]:     Accuracy Decimals: 0
[14:13:36][C][sen5x:277]:     Icon: 'mdi:radiator'
[14:13:36][C][sen5x:278]:   NOx 'NOX'
[14:13:36][C][sen5x:278]:     Device Class: 'nitrous_oxide'
[14:13:36][C][sen5x:278]:     State Class: 'measurement'
[14:13:36][C][sen5x:278]:     Unit of Measurement: ''
[14:13:36][C][sen5x:278]:     Accuracy Decimals: 0
[14:13:36][C][sen5x:278]:     Icon: 'mdi:radiator'
[14:13:36][C][scd4x:094]: scd4x:
[14:13:36][C][scd4x:095]:   Address: 0x62
[14:13:36][C][scd4x:112]:   Automatic self calibration: ON
[14:13:36][C][scd4x:121]:   Ambient pressure compensation disabled
[14:13:36][C][scd4x:122]:   Altitude compensation: 0m
[14:13:36][C][scd4x:127]:   Measurement mode: periodic (5s)
[14:13:36][C][scd4x:139]:   Temperature offset: 9.00 °C
[14:13:36][C][scd4x:140]:   Update Interval: 60.0s
[14:13:36][C][scd4x:141]:   CO2 'CO2'
[14:13:36][C][scd4x:141]:     Device Class: 'carbon_dioxide'
[14:13:36][C][scd4x:141]:     State Class: 'measurement'
[14:13:36][C][scd4x:141]:     Unit of Measurement: 'ppm'
[14:13:36][C][scd4x:141]:     Accuracy Decimals: 0
[14:13:36][C][scd4x:141]:     Icon: 'mdi:molecule-co2'
[14:13:36][C][scd4x:142]:   Temperature 'Temperature'
[14:13:36][C][scd4x:142]:     Device Class: 'temperature'
[14:13:36][C][scd4x:142]:     State Class: 'measurement'
[14:13:36][C][scd4x:142]:     Unit of Measurement: '°C'
[14:13:36][C][scd4x:142]:     Accuracy Decimals: 2
[14:13:36][C][scd4x:142]:     Icon: 'mdi:thermometer'
[14:13:36][C][scd4x:143]:   Humidity 'Humidity'
[14:13:36][C][scd4x:143]:     Device Class: 'humidity'
[14:13:36][C][scd4x:143]:     State Class: 'measurement'
[14:13:36][C][scd4x:143]:     Unit of Measurement: '%'
[14:13:36][C][scd4x:143]:     Accuracy Decimals: 2
[14:13:36][C][scd4x:143]:     Icon: 'mdi:water-percent'

Additional information

No response

ebrithiljonas commented 2 years ago

I have the exact same problem, using the same ESPHome Version (v2022.6.2) with also an ESP32. My config for the sensor looks like this:

sensor:
  - platform: sen5x
    id: sen55
    pm_1_0:
      name: "{$sensor_prefix} PM <1µm Weight concentration"
      id: sensor_pm_1_0
      accuracy_decimals: 1
      filters:
        - median:
            window_size: &window_size 15
            send_every: &send_every 15
            send_first_at: &send_first_at 15
      on_value:
        then:
          - script.execute: update
    pm_2_5:
      name: "{$sensor_prefix} PM <2.5µm Weight concentration"
      id: sensor_pm_2_5
      accuracy_decimals: 1
      filters:
        - median:
            window_size: *window_size
            send_every: *send_every
            send_first_at: *send_first_at
    pm_4_0:
      name: "{$sensor_prefix} PM <4µm Weight concentration"
      id: sensor_pm_4_0
      accuracy_decimals: 1
      filters:
        - median:
            window_size: *window_size
            send_every: *send_every
            send_first_at: *send_first_at
    pm_10_0:
      name: "{$sensor_prefix} PM <10µm Weight concentration"
      id: sensor_pm_10_0
      accuracy_decimals: 1
      filters:
        - median:
            window_size: *window_size
            send_every: *send_every
            send_first_at: *send_first_at
    temperature:
      name: "{$sensor_prefix} Temperature"
      id: sensor_temperature
      accuracy_decimals: 1
      filters:
        - median:
            window_size: *window_size
            send_every: *send_every
            send_first_at: *send_first_at
    humidity:
      name: "{$sensor_prefix} Humidity"
      id: sensor_humidity
      accuracy_decimals: 0
      filters:
        - median:
            window_size: *window_size
            send_every: *send_every
            send_first_at: *send_first_at
    voc:
      name: "{$sensor_prefix} VOC"
      id: sensor_voc
      filters:
        - median:
            window_size: *window_size
            send_every: *send_every
            send_first_at: *send_first_at
    nox:
      name: "{$sensor_prefix} NOx"
      id: sensor_nox
      filters:
        - median:
            window_size: *window_size
            send_every: *send_every
            send_first_at: *send_first_at
    temperature_compensation:
      offset: 5
      normalized_offset_slope: 0
      time_constant: 1
    acceleration_mode: low
    store_baseline: true
    address: 0x69
    update_interval: 1s

AwokenByMyFears commented 1 year ago

I am also experiencing this issue. Any solutions?

AwokenByMyFears commented 1 year ago

Actually I just found a way that works...

sensor:
  - platform: sen5x
    id: kitchen_sen55
    temperature:
      name: "Kitchen Temperature"
      filters:
        - offset: -2.78

Using this I was able to apply an offset to my temp to get it to be correct. Hope this helps the others of you who can't get the built in way to work.

test32443 commented 1 year ago

Actually I just found a way that works...
sensor:
  - platform: sen5x
    id: kitchen_sen55
    temperature:
      name: "Kitchen Temperature"
      filters:
        - offset: -2.78
Using this I was able to apply an offset to my temp to get it to be correct. Hope this helps the others of you who can't get the built in way to work.

This is a nice workaround, but I think it only addresses the issue for the display purposes - humidity calculation would still likely be wrong because it is using internal sensor value.

AwokenByMyFears commented 1 year ago

Actually I just found a way that works...
sensor:
  - platform: sen5x
    id: kitchen_sen55
    temperature:
      name: "Kitchen Temperature"
      filters:
        - offset: -2.78
Using this I was able to apply an offset to my temp to get it to be correct. Hope this helps the others of you who can't get the built in way to work.
This is a nice workaround, but I think it only addresses the issue for the display purposes - humidity calculation would still likely be wrong because it is using internal sensor value.

I believe it will still work as the method is altering what the sensor is reporting to home assistant (as confirmed by the log files). But give it a try and see if it works.

test32443 commented 1 year ago

It will work for display purposes, which is great on its own, however humidity is calculated internally within sensor and it is impacted by the temperature it reports. I have Sen5x and scd4x side by side and after filter adjustment on SEN5x, both report similar temperatures, but humidity values are quite different. I think it is happening because the filter offset adjusts temperature outside of the sensor and therefore not having impact on humidity calculation.

AwokenByMyFears commented 1 year ago

It will work for display purposes, which is great on its own, however humidity is calculated internally within sensor and it is impacted by the temperature it reports. I have Sen5x and scd4x side by side and after filter adjustment on SEN5x, both report similar temperatures, but humidity values are quite different. I think it is happening because the filter offset adjusts temperature outside of the sensor and therefore not having impact on humidity calculation.

I see what you're saying. I didn't think about how the humidity was calculated based on the temp sensor. So yeah, this would only work to correct the temp sensor. Unless you can figure out the correct offset for the humidity too ;)

Zahradka-CZ commented 1 year ago

Hi, I have the same problem that is described here. On the manufacturer's website, you can download Sensirion_Temperature_Acceleration_and_Compensation_Instructions_SEN In this manual it is mentioned that you need to get into STAR mode (Sensirion Temperature Acceleration Routine). Could someone implement this mode in the code for ESP32? After running this mode, temperature compensation would already be accepted.

Translated with www.DeepL.com/Translator (free version)

amagr0 commented 1 year ago

I have the SEN55 sensor and I also having problems with temperature and humidity readings. I'm not using the sensor inside anything, so it should not be necessary to compensate it.

I solve this with filters for temperature and humidity, seams to work fine for now.

AwokenByMyFears commented 1 year ago

@martgras, I believe you were the person who added the integration. Is there anything you can do to help us sort this out? Thanks!

cegan09 commented 1 year ago

Found this thread while setting up a couple of these sensors. It does appear to not work for either the ESP8266 or ESP32. No combination of compensation settings appears to have any impact on the reported values. Based on the link above it looks like the setting isn't actually being applied to the sensor.

matt7aylor commented 1 year ago

I had the same issue. I've added a PR that should fix it if anyone wants to try themselves before it is (hopefully) merged: https://github.com/esphome/esphome/pull/4901

test32443 commented 1 year ago

Hello matt7aylor, what do you pass for temp compensation in your code? Whenever I pass anything other than 0.0 for temp compensation, I get "[E][sen5x:395]: set temperature_compensation failed. Err=2" message.

Here are the parameters I am trying to pass: temperature_compensation: offset: 0.0 normalized_offset_slope: 0 time_constant: 0

matt7aylor commented 1 year ago

There is another problem in the original code as far as I can tell with the slope, in that it doesn't seem to support negative numbers (I've got a fix that is working for me, will add it to that PR). But if you use something like:

    temperature_compensation:
      offset: -2.0
      normalized_offset_slope: 0
      time_constant: 0

do you still get the same error?

test32443 commented 1 year ago

If I keep the original code from your PR and use the config you posted (with offset set to -2.0), I still get the error: [07:56:42][D][sen5x:106]: Productname SEN55 [07:56:42][D][sen5x:134]: Firmware version 2 [07:56:42][E][sen5x:395]: set temperature_compensation failed. Err=2 [07:56:42][D][sen5x:221]: Sensor initialized

matt7aylor commented 1 year ago

Hmm, this is the first time I've ever done anything with esphome/C++ or indeed a Sen5x sensor so we may hit the limits of my useful knowledge quite soon.

On the plus side at least it is trying to set the value, whereas I'm guessing before that it didn't even get to that part?

The use of uint16 in a few places where we really do want signed integers does worry me a bit, for instance if I add this logging above line 394 in esphome/components/sen5x/sen5x.cpp:

  ESP_LOGD(TAG, "Temperature compensation: offset %d, normalized_offset_slope %d, time_constant %d", 
           params[0], params[1], params[2]);

then I get: [D][sen5x:415]: Temperature compensation: offset 65136, normalized_offset_slope 0, time_constant 0 when I really should get: [D][sen5x:218]: Temperature compensation: offset -400, normalized_offset_slope 0, time_constant 0 (for -2.0), but it's essentially overflowing twice I guess so it cancels out in mine, maybe it isn't in yours? Perhaps you could try the same logging and show the result and also just try a positive number to see if that makes any difference.

(Also I've now added a fix, that works for me, to allow negative slope values, see: https://github.com/esphome/esphome/pull/4901 )

test32443 commented 1 year ago

It fails even with a positive compensation value - for example for offset=2.0 : [09:43:29][D][sen5x:106]: Productname SEN55 [09:43:29][D][sen5x:134]: Firmware version 2 [09:43:29][D][sen5x:394]: Temperature compensation: offset 400, normalized_offset_slope 0, time_constant 0 [09:43:29][E][sen5x:215]: Error starting continuous measurements. [09:43:29][E][component:113]: Component sen5x.sensor was marked as failed.

matt7aylor commented 1 year ago

That is a different error? In that log it seems like maybe the temperature compensation did not produce an error but the measurement init (line 215) command did, is that accurate? I haven't changed anything that should affect the init command, though in other places in the code there are delay(20); lines between commands, you could try putting one between the commands (on line 206 say) and see if that makes any difference.

Is everything else working as expected? The communication with the sensor is reliable? Do you have any issues setting VOC/NOX tuning parameters? I'm not sure why you should have problems when I don't, we are using the same sensiron product/firmware version according to my logs:

[D][sen5x:106]: Productname SEN55
[D][sen5x:134]: Firmware version 2

but without some clue as to how to reproduce the problem you are seeing it is difficult for me to know what to suggest.

test32443 commented 1 year ago

Looking at the SEN5x document (https://sensirion.com/media/documents/6791EFA0/62A1F68F/Sensirion_Datasheet_Environmental_Node_SEN5x.pdf ), it seems that parameters should be followed with checksum value - is it even being calculated/passed correctly?

test32443 commented 1 year ago

It is very strange - just adding ESP_LOGD entry at line 394, changes the output from [10:58:26][E][sen5x:396]: set temperature_compensation failed. Err=2 to [10:55:41][E][sen5x:215]: Error starting continuous measurements. [10:55:41][E][component:113]: Component sen5x.sensor was marked as failed. and then sensor does not even operate.

If I comment out ESP_LOGD at line 394, I get back to set temperature_compensation failed. Err=2.

I do have VOC tuning parameters set as following: voc: name: "VOC" id: sensor_tvoc algorithm_tuning: index_offset: 100 learning_time_offset_hours: 72 learning_time_gain_hours: 72 gating_max_duration_minutes: 180 std_initial: 50 gain_factor: 230

test32443 commented 1 year ago

Quick update - putting delay(20) after each parameter write did solve the problem:

 if (this->voc_tuning_params_.has_value())
    this->write_tuning_parameters_(SEN5X_CMD_VOC_ALGORITHM_TUNING, this->voc_tuning_params_.value());
  delay(20);
  if (this->nox_tuning_params_.has_value())
    this->write_tuning_parameters_(SEN5X_CMD_NOX_ALGORITHM_TUNING, this->nox_tuning_params_.value());
  delay(20);
  if (this->temperature_compensation_.has_value())
    this->write_temperature_compensation_(this->temperature_compensation_.value());
  delay(20);

Even if I change it to delay(10), I am getting error message, but with delay(20) it seem to work. It is possible that our ESP platforms are different - I am using M5Stack core, but for now, the issue seems to be resolved - thank you so much!

Could I recommend adding these delay(20) statements to your PR?

matt7aylor commented 1 year ago

Excellent, great that we could get to the fix together, you are right it is probably our different ESP platforms, some difference in timings between platforms was my theory with the delay too, I'm pleased it was accurate.

I've added an extra delay for each of those write commands in the PR, let me know if that works for you. Maybe make a comment on the PR if it is all now working, perhaps that will help whoever comes to review it.

test32443 commented 1 year ago

I put the comment on PR specifying that it resolved the issue for me - hope it helps to get it reviewed and approved.

cegan09 commented 1 year ago

I'd like to try the fix as well, but I admit I'm not sure how to force the modified files instead of the official ones. Any chance I can get a quick guide or pointer to a how to?

test32443 commented 1 year ago

The way I did it was to go to PR files here, download sen5x.cpp, sensor.py and sen5x.h into your esphome folder (mine was at c:\users\\AppData\Roaming\Python\Python39\site-packages\esphome\components\sen5x ) backing up older versions and re-deploying my ESPHome project to device.

cegan09 commented 1 year ago

Thanks, I'll have to figure out how to do it for my device. I'm running esphome inside home assistant. I swapped the files for the PR versions, but no change. Likely I just haven't done it correctly.

matt7aylor commented 1 year ago

You can also compile/install the esphome firmware separately from the esphome install in your home assistant (I have esphome running in a docker container but built/flashed on a different PC with a copy of the yaml and it still works fine with homeassistant etc.). Rough steps for a linux machine (I don't know much about windows):

clone the esphome git repository (in this case the fork with my changes https://github.com/matt7aylor/esphome/tree/sen5x-fix-compensation), enter the root directory and checkout the sen5x-fix-compensation branch
Alternatively: Download the files directly without git https://github.com/matt7aylor/esphome/archive/refs/heads/sen5x-fix-compensation.zip , extract and enter the root directory
create a python virtual environment to work in python -m venv venv
source the virtual environment e.g. source venv/bin/activate
install the esphome dependencies to the venv pip install .
copy your current esphome .yaml to the directory e.g. aqtest.yaml
compile esphome aqtest.yaml compile
flash to connected ESP e.g. esptool.py -p /dev/ttyUSB0 -b 921600 write_flash 0x0 .esphome/build/aqtest/.pioenvs/aqtest/firmware-factory.bin (change port for your port)
[view UART logs e.g. screen /dev/ttyUSB0 115200]

(Presented as a rough guide only, don't follow any steps you don't understand the principle/how to adapt them.)

cegan09 commented 1 year ago

Thanks. I'll see if I can get a test environment setup on a linux machine and try it out.

cegan09 commented 1 year ago

Well, that took far too long to get a linux environment setup with a python version over 3.9...

I can confirm that the fix appears to work for me as well, both positive and negative offsets. Side note, when figuring out the offset, it helps to remember that the sensor speaks in °C, not °F. Coffee hadn't kicked in yet, spend a few minutes trying to figure out why the offset kept overshooting.

AwokenByMyFears commented 8 months ago

https://esphome.io/changelog/2023.12.0.html

"sen5x fix temperature compensation and gas tuning (breaking-change)"

Looks like this finally got merged into a production branch. Thanks for fixing it, @matt7aylor !

esphome / issues