Non null flat line joining when no data

[Thlerry]

Describe the bug Graph joins non null values, instead of showing zero / null. I had no power for several hours today between 7am and 12:30pm roughly. So I would have expected the graph to show a dip at zero during that time, but it seems plotly joins the last known value, with the first new one, even several hours apart. Is there a way to prevent this, and show the fact that there was no value during that time, and keep the graph at zero ? See screenshot below

Screenshots

yaml

type: custom:plotly-graph
entities:
  - entity: sensor.tb_house_consumption_kw
    name: Grid
    fill: tozeroy
    line:
      color: '#77aaff'
      shape: spline
    show_value: false
    extend_to_present: false
    on_legend_click: {}
    on_click: {}
    on_dblclick: {}
    filters:
      - resample: 10m
  - entity: sensor.tb_solaredge_ac_power
    name: Solar
    fill: tozeroy
    line:
      shape: spline
      color: '#ff8800'
    show_value: false
    extend_to_present: false
    on_legend_click: {}
    on_click: {}
    on_dblclick: {}
    filters:
      - resample: 10m
  - entity: sensor.tb_house_solar_only_consumption_kw
    name: Self
    fill: tozeroy
    line:
      color: '#00eeee'
      shape: spline
    show_value: false
    extend_to_present: false
    on_legend_click: {}
    on_click: {}
    on_dblclick: {}
    filters:
      - resample: 10m
hours_to_show: current_day
refresh_interval: 1
config:
  scrollZoom: false
  displayModeBar: false
  staticPlot: true

Additional context When this happened, I had no electricity during the gap, so HA was not running at all, so nothing should have been recorded in the DB ?

[Thlerry]

In the console I see a lot of the following errors:

[Thlerry]

I clear the console, and did a refresh and this is what I see:

[Thlerry]

Not sure how to get the information you need to debug ? Would you be able to help ?

[dbuezas]

Check out this section: https://github.com/dbuezas/lovelace-plotly-graph-card#debugging

With it you can dump the data to the console for analysis

[dbuezas]

The issue may be the resample filter

[dbuezas]

I looked at my implementation of resampling, and it should actually add a bunch of nulls, but try without it to narrow this down :)

[Thlerry]

With resampling:

Without Resampling:

[Thlerry]

here is the graph with nofill / no resampling. This looks correct.

[Thlerry]

And this is nofill but with resampling. So it seems one issue is with the resampling, and one issue with the fill option ?

[dbuezas]

I see. Maybe you can map all the null values to the number zero before resampling?

[dbuezas]

This would give a reasonable result after resampling, even if you use fill

[Thlerry]

Is there an easy way to do that, maybe with a filter ?

Would that work ?

filters:
  - map_y_numbers: 'y == null ? 0 : y'
  - resample: 2m

[dbuezas]

Yes, But don't use map_y_numbers, that one will already eliminate the nulls

Use map_y instead

[Thlerry]

OK, I gave it a go, but still not working.

Here is a simplified graph with various options. Any option used in filters: seems to mess up the line / fill.

[dbuezas]

Try

map_y: isNaN(+y) ? 0 : y

Maybe y is "unavailable" or something like that. You may want to check out the "debugging" section in the readme to see the data in the console. Way easier to debug these kinds of issues.

BTW, map_y_numbers is expected to remove nulls, since they are not numbers

edit: fixed typo isNan => isNaN

[Thlerry]

I am getting this error with isNan function ?

[Thlerry]

I think checking against null or undefined works because the graph changes. It starts from 0 after the gap, but still links the last valid value to 0. (See below)

How do you use the debugger to check further ? I looked at the debugging section but not sure what to do in the debugger (not used to it)

[Thlerry]

Were you able to reproduce it ? You can take any sensor, stop HA for a few minutes to create a gap in values, and restart HA. Pretty easy.

[dbuezas]

just a typo, I updated my message above

[dbuezas]

Oh I see, you are right, it is working. So it is not that the sensor has "unavailable" datapoints, they are just missing. In this case, joining the lines is actually reasonable behaviour. What you can do is add an extra fake datapoint

filters:
  - fn: |- 
      ({xs, ys}) => ({
          xs: xs.flatMap((x, i) => ys[i+1] === undefined ? [x,x]:[x]),
          ys: ys.flatMap((y, i) => ys[i+1] === undefined ? [y,undefined]:[y])
      })

This will insert an extra undefined to the ys array, but set its date to that of the last known value. This should make the gap as wide as the whole missing chunk

[Thlerry]

I tried to add the fn above, but the syntax doesn't seem correct, and HA keeps spinning. I tried to put everything on the same line, and still doesn't work:

And the multiline version not working:

[dbuezas]

Add an extra indentation (2 spaces) to all lines inside -fn

[dbuezas]

(ok, that's equivalent to the single line) Now add the extra filter that maps undefineds to zero. You now only have the "filll" issue, which isn't a bug bit just how plotly behaves