transform would get supercharged with access to history item

[fcassirer]

Good stuff @RomRider. Thanks for putting this together.

I have a use case where I am trying to chart daily usage of my xfinity internet service. I am trying to create a bar chart of daily usage for a given 'month so far' and an indication if I am on track to stay under my bandwidth cap (1229 GB/mo). I have sensors that have a daily history of usage as well as an array of 6 prior months total usage.

So, for any given day/datapoint, I need to:

• calculate the perday bandwidth amount (bandwidth_cap/days_in_the_month)
• determine what day of the current cycle this data represents (data_timestamp is timestamp of when data was recorded)
• calculate thresholds of usage based on (number of days * perday)
• color a bar or line segment with red, yellow, or green based on say under 70% (green), 70-90% (yellow), and >90% red.
• I need to calculate this for every historical point.

The result should be a set of colorized bars that would show each day individually based on the day into the period. Note that day 1 could be over but day 2 could be under so the bars would be red then green if day 1 was a busy day and day 2 wasn't.

I have the config-template-card which allows me to use scripting over the entire apexcharts-card, however, this is only good for the current (today) instantiation of the entity.

I am looking at the transform option to provide this info, however, there is no access to the actual history item, instead, we get the historical 'x' value which is the state of the item and full access to the hass.states['sensor.xfinity_usage'] object, but this is again the current value, not the actual item that is being pulled from the history.

I think if there were an option to directly access the object structure item being transformed it would allow this capability.

So this line and this one

I think would just need to pass in the current working item to make it available as a named entity as 'x' or 'hass'.

Here is what a sample graph would look like, ideally the green bars would show based on that data points current usage not the usage as of the day it is being displayed. The following yaml was attempting to draw a line graph overlay that would show the 70% and 90% threshold lines, but it became clear that data isn't available. Note, this example is setting color based on the current state, not previous state.

using:

type: 'custom:config-template-card'
variables:
  THIS: 'states[''sensor.xfinity_usage'']'
  tthold: |
    (thresh,THIS) => {
      const date = new Date(THIS.attributes.raw.usageMonths[6].endDate);
      console.log(date);
      const daysmon = date.getDate();
      console.log("daysmon="+daysmon)
      console.log(THIS.attributes.data_timestamp);
      const today = new Date(THIS.attributes.data_timestamp * 1000).getDate();
      console.log(">>" + today);
      const daysleft = daysmon - today ;
      const perday = 1229/daysmon;
      console.log("usage= "+THIS.state);
      return today * perday * thresh;
    }
entities:
  - sensor.xfinity_usage
card:
  type: 'custom:apexcharts-card'
  header:
    show: true
    title: 'Xfinity Usage'
    show_states: false
    colorize_states: true
  update_interval: 12h
  experimental:
    hidden_by_default: true
    color_threshold: true
  apex_config:
    xaxis:
      labels:
        format: M/d
    yaxis:
      min: 0
      max: '${tthold(1,THIS)}'
    legend:
      show: false
    hidden_by_default: true
    color_threshold: true
  span:
    start: day
    offset: '-5d'
  graph_span: 7d
  series:
    - entity: sensor.xfinity_usage
      type: column
      name: Current
      extend_to_end: false
      fill_raw: last
      group_by:
        func: last
        duration: 1d
      show:
        legend_value: false
        datalabels: true
      color_threshold:
        - color: Red
          value: '${tthold(0.9,THIS)}'
        - color: Yellow
          value: '${tthold(0.7,THIS)}'
        - color: Green
          value: 0
    - entity: sensor.xfinity_usage
      transform: "
      console.log('this='+Object.keys(this));
      var THIS = hass.states['sensor.xfinity_usage']; 
      const thresh = 0.7;
      console.log('entity='+THIS.attributes.data_timestamp);
      const date = new Date(THIS.attributes.raw.usageMonths[6].endDate);
      console.log(date);
      const daysmon = date.getDate();
      console.log('daysmon='+daysmon);
      console.log(THIS.attributes.data_timestamp);
      const today = new Date(THIS.attributes.data_timestamp * 1000).getDate();
      console.log('>>' + today);
      const daysleft = daysmon - today ;
      const perday = 1229/daysmon;
      console.log('perday='+perday); 
      console.log('today='+today);
      console.log('thresh='+thresh);
      console.log('x='+x);

      return x * today * perday * thresh;
      "
      type: line
      name: Current
      extend_to_end: false
      fill_raw: last
      group_by:
        func: last
        duration: 1d 
      show:
        legend_value: false
        datalabels: true

As a reference, here what the xfinity_usage sensor looks like, there is essentially some data in the outer block like data_timestamp, usage, and total and then there are 6 months of past usage summaries with usageMonths[6] being this month so far and also contains the start/end date of the current period.

{'data_timestamp': 1612890369,
              'friendly_name': 'Xfinity Usage',
              'raw': {'courtesyAllowed': 1,
                      'courtesyRemaining': 1,
                      'courtesyUsed': 0,
                      'displayUsage': True,
                      'inPaidOverage': False,
                      'usageMonths': [{'additionalBlocksUsed': 0.0,
                                       'additionalCostPerBlock': 10.0,
                                       'additionalIncluded': 0.0,
                                       'additionalPercentUsed': 0.0,
                                       'additionalRemaining': 0.0,
                                       'additionalUnitsPerBlock': 50.0,
                                       'additionalUsed': 0.0,
                                       'allowableUsage': 1229.0,
                                       'billableOverage': 0.0,
                                       'currentCreditAmount': 0,
                                       'devices': [{'id': 'B0:39:56:99:E4:E1',
                                                    'usage': 931.0}],
                                       'displayUsage': True,
                                       'endDate': '08/31/2020',
                                       'homeUsage': 931.0,
                                       'maxCreditAmount': 0,
                                       'overageCharges': 0.0,
                                       'overageUsed': 0.0,
                                       'policy': 'limited',
                                       'policyName': '1.2 Terabyte Data Plan',
                                       'startDate': '08/01/2020',
                                       'totalUsage': 931.0,
                                       'unitOfMeasure': 'GB',
                                       'wifiUsage': 0.0},
                                      {'additionalBlocksUsed': 0.0,
                                       'additionalCostPerBlock': 10.0,
                                       'additionalIncluded': 0.0,
                                       'additionalPercentUsed': 0.0,
                                       'additionalRemaining': 0.0,
                                       'additionalUnitsPerBlock': 50.0,
                                       'additionalUsed': 0.0,
                                       'allowableUsage': 1229.0,
                                       'billableOverage': 0.0,
                                       'currentCreditAmount': 0,
                                       'devices': [{'id': 'B0:39:56:99:E4:E1',
                                                    'usage': 987.0}],
                                       'displayUsage': True,
                                       'endDate': '09/30/2020',
                                       'homeUsage': 987.0,
                                       'maxCreditAmount': 0,
                                       'overageCharges': 0.0,
                                       'overageUsed': 0.0,
                                       'policy': 'limited',
                                       'policyName': '1.2 Terabyte Data Plan',
                                       'startDate': '09/01/2020',
                                       'totalUsage': 987.0,
                                       'unitOfMeasure': 'GB',
                                       'wifiUsage': 0.0},
                                      {'additionalBlocksUsed': 0.0,
                                       'additionalCostPerBlock': 10.0,
                                       'additionalIncluded': 0.0,
                                       'additionalPercentUsed': 0.0,
                                       'additionalRemaining': 0.0,
                                       'additionalUnitsPerBlock': 50.0,
                                       'additionalUsed': 0.0,
                                       'allowableUsage': 1229.0,
                                       'billableOverage': 0.0,
                                       'currentCreditAmount': 0,
                                       'devices': [{'id': 'B0:39:56:99:E4:E1',
                                                    'usage': 730.0}],
                                       'displayUsage': True,
                                       'endDate': '10/31/2020',
                                       'homeUsage': 730.0,
                                       'maxCreditAmount': 0,
                                       'overageCharges': 0.0,
                                       'overageUsed': 0.0,
                                       'policy': 'limited',
                                       'policyName': '1.2 Terabyte Data Plan',
                                       'startDate': '10/01/2020',
                                       'totalUsage': 730.0,
                                       'unitOfMeasure': 'GB',
                                       'wifiUsage': 0.0},
                                      {'additionalBlocksUsed': 0.0,
                                       'additionalCostPerBlock': 10.0,
                                       'additionalIncluded': 0.0,
                                       'additionalPercentUsed': 0.0,
                                       'additionalRemaining': 0.0,
                                       'additionalUnitsPerBlock': 50.0,
                                       'additionalUsed': 0.0,
                                       'allowableUsage': 1229.0,
                                       'billableOverage': 0.0,
                                       'currentCreditAmount': 0,
                                       'devices': [{'id': 'B0:39:56:99:E4:E1',
                                                    'usage': 850.0}],
                                       'displayUsage': True,
                                       'endDate': '11/30/2020',
                                       'homeUsage': 850.0,
                                       'maxCreditAmount': 0,
                                       'overageCharges': 0.0,
                                       'overageUsed': 0.0,
                                       'policy': 'limited',
                                       'policyName': '1.2 Terabyte Data Plan',
                                       'startDate': '11/01/2020',
                                       'totalUsage': 850.0,
                                       'unitOfMeasure': 'GB',
                                       'wifiUsage': 0.0},
                                      {'additionalBlocksUsed': 0.0,
                                       'additionalCostPerBlock': 10.0,
                                       'additionalIncluded': 0.0,
                                       'additionalPercentUsed': 0.0,
                                       'additionalRemaining': 0.0,
                                       'additionalUnitsPerBlock': 50.0,
                                       'additionalUsed': 0.0,
                                       'allowableUsage': 1229.0,
                                       'billableOverage': 0.0,
                                       'currentCreditAmount': 0,
                                       'devices': [{'id': 'B0:39:56:99:E4:E1',
                                                    'usage': 882.0}],
                                       'displayUsage': True,
                                       'endDate': '12/31/2020',
                                       'homeUsage': 882.0,
                                       'maxCreditAmount': 0,
                                       'overageCharges': 0.0,
                                       'overageUsed': 0.0,
                                       'policy': 'limited',
                                       'policyName': '1.2 Terabyte Data Plan',
                                       'startDate': '12/01/2020',
                                       'totalUsage': 882.0,
                                       'unitOfMeasure': 'GB',
                                       'wifiUsage': 0.0},
                                      {'additionalBlocksUsed': 0.0,
                                       'additionalCostPerBlock': 10.0,
                                       'additionalIncluded': 0.0,
                                       'additionalPercentUsed': 0.0,
                                       'additionalRemaining': 0.0,
                                       'additionalUnitsPerBlock': 50.0,
                                       'additionalUsed': 0.0,
                                       'allowableUsage': 1229.0,
                                       'billableOverage': 0.0,
                                       'currentCreditAmount': 0,
                                       'devices': [{'id': 'B0:39:56:99:E4:E1',
                                                    'usage': 1099.0}],
                                       'displayUsage': True,
                                       'endDate': '01/31/2021',
                                       'homeUsage': 1099.0,
                                       'maxCreditAmount': 0,
                                       'overageCharges': 0.0,
                                       'overageUsed': 0.0,
                                       'policy': 'limited',
                                       'policyName': '1.2 Terabyte Data Plan',
                                       'startDate': '01/01/2021',
                                       'totalUsage': 1099.0,
                                       'unitOfMeasure': 'GB',
                                       'wifiUsage': 0.0},
                                      {'additionalBlocksUsed': 0.0,
                                       'additionalCostPerBlock': 10.0,
                                       'additionalIncluded': 0.0,
                                       'additionalPercentUsed': 0.0,
                                       'additionalRemaining': 0.0,
                                       'additionalUnitsPerBlock': 50.0,
                                       'additionalUsed': 0.0,
                                       'allowableUsage': 1229.0,
                                       'billableOverage': 0.0,
                                       'currentCreditAmount': 0,
                                       'devices': [{'id': 'B0:39:56:99:E4:E1',
                                                    'usage': 205.0}],
                                       'displayUsage': True,
                                       'endDate': '02/28/2021',
                                       'homeUsage': 205.0,
                                       'maxCreditAmount': 0,
                                       'overageCharges': 0.0,
                                       'overageUsed': 0.0,
                                       'policy': 'limited',
                                       'policyName': '1.2 Terabyte Data Plan',
                                       'startDate': '02/01/2021',
                                       'totalUsage': 205.0,
                                       'unitOfMeasure': 'GB',
                                       'wifiUsage': 0.0}]},
              'total': 1229.0,
              'unit_of_measurement': 'GB',
              'units': 'GB',
              'used': 205.0
}

Thanks in advance -Fred

[RomRider]

Hey,

I didn't read through all the code nor try to do the maths on your behalf but it seems you have thought it through. 😊

If what you need is access the full state of the current timestamp (with all the attributes etc...) on top of just the state value for each history entry, I can make this happen. So you'd get x as the state or attribute (but you can ignore it) and something like stateObj with the whole content of the entity for the history entry currently being transformed.

Would that work for you?

[fcassirer]

Yes, absolutely, that would provide some options for how to present the data. More broadly, maybe a general mechanism similar to the config-template-card might be a way to add more consistent scripting across the entire card, i.e, not reinventing what args get passed into template, data_generator, and no doubt future yet to be named features ;-) In my example, there is a function called tthold() that is global to the card from config-template-card, would it be useful to allow the transform/data_generator access to that context? Just thinking out loud ...

Thanks!

[RomRider]

One thing at a time but thanks for the enthusiasm 😊 I'll add support for retrieving the full object in transform asap!

[github-actions[bot]]

:tada: This issue has been resolved in version 1.7.0-dev.2 :tada:

The release is available on GitHub release

Your semantic-release bot :package::rocket:

[fcassirer]

Thank you! This works well. Here is a graph that can now plot the % usage for each day:

All I had to do was change the data_timestamp reference from the current xfinity_usage object to the entity as it existed the day it was recorded:

transform: >-
        ...
        console.log(entity.attributes.data_timestamp); const today = new
        Date(entity.attributes.data_timestamp * 1000).getDate();

The red and yellow lines represent the 70% and 90% usage levels for that day of the cycle. Now that we have this field I think much more interesting graphs can be created.

Thanks again for the quick turnaround! -Fred

[github-actions[bot]]

:tada: This issue has been resolved in version 1.7.0 :tada:

The release is available on GitHub release

Your semantic-release bot :package::rocket: