v3 - Sometimes sorting order gets mixed up on page reload

[stas-sl]

Occasionally, when the page reloads, the sorting order defined by web_server_sorting_weight gets mixed up. This issue becomes more noticeable with an increasing number of sensors and more frequent updates.

https://github.com/user-attachments/assets/dff1791f-d979-4a6c-b0ca-880cc3af7d1a

esphome:
  name: test
  platformio_options:
    upload_speed: 1500000
    board_build.f_cpu: 240000000

esp32:
  board: esp32dev
  framework:
    type: esp-idf

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  power_save_mode: none
  fast_connect: true

logger:
  level: ERROR

api:
  password: !secret api_password

ota:
  platform: esphome

web_server:
  port: 80
  version: 3

number:
  - platform: template
    name: 1
    id: number1
    optimistic: true
    min_value: 0
    max_value: 100
    step: 1
    web_server_sorting_weight: 1
  - platform: template
    name: 2
    id: number2
    optimistic: true
    min_value: 0
    max_value: 100
    step: 1
    web_server_sorting_weight: 2
  - platform: template
    name: 3
    id: number3
    optimistic: true
    min_value: 0
    max_value: 100
    step: 1
    web_server_sorting_weight: 3
  - platform: template
    name: 4
    id: number4
    optimistic: true
    min_value: 0
    max_value: 100
    step: 1
    web_server_sorting_weight: 4
  - platform: template
    name: 5
    id: number5
    optimistic: true
    min_value: 0
    max_value: 100
    step: 1
    web_server_sorting_weight: 5

interval:
  - interval: 100ms
    then:
      - lambda: |-
          auto call = id(number1)->make_call();
          call.number_increment(true);
          call.perform();
  - interval: 100ms
    then:
      - lambda: |-
          auto call = id(number2)->make_call();
          call.number_increment(true);
          call.perform();
  - interval: 100ms
    then:
      - lambda: |-
          auto call = id(number3)->make_call();
          call.number_increment(true);
          call.perform();
  - interval: 100ms
    then:
      - lambda: |-
          auto call = id(number4)->make_call();
          call.number_increment(true);
          call.perform();
  - interval: 100ms
    then:
      - lambda: |-
          auto call = id(number5)->make_call();
          call.number_increment(true);
          call.perform();

[RFDarter]

hi, can you test it with

  framework:
    type: arduino

and see if it's the same

[stas-sl]

Hi, yes, the same result, I also tried a different browser (Chrome) as I previously tested in Edge, but the issue persists:

https://github.com/user-attachments/assets/15b260aa-b866-423e-a0cd-0885467d843d

[RFDarter]

hm, same on FF. I logged the /events but I cant make any sense of it why the order gets messed up the way it does

order-glich-states.txt

[RFDarter]

A normal behaviour would be if you refresh the page the esp would send a state event for every component with DETAIL_ALL which will look like this:

event: state
data: {"id":"number-1","name":"1","icon":"","entity_category":0,"min_value":"0","max_value":"100","step":"1","mode":0,"sorting_weight":1,"value":"84","state":"84"}

if the state changes the event would look like this:

data: {"id":"number-1","value":"86","state":"86"}```

[stas-sl]

I haven’t checked the contents of /events myself, but based on the file you attached, it seems the server (esp) is sending everything correctly. However, the order gets mixed up on the client side.

[RFDarter]

I think I figured it out.

On this example the order it got displayed was 5 1 2 3 4

and the events got received in this order

 event: state
data: {"id":"number-5","value":"84","state":"84"}

event: state
data: {"id":"number-1","name":"1","icon":"","entity_category":0,"min_value":"0","max_value":"100","step":"1","mode":0,"sorting_weight":1,"value":"84","state":"84"}

event: state
data: {"id":"number-2","name":"2","icon":"","entity_category":0,"min_value":"0","max_value":"100","step":"1","mode":0,"sorting_weight":2,"value":"84","state":"84"}

event: state
data: {"id":"number-3","name":"3","icon":"","entity_category":0,"min_value":"0","max_value":"100","step":"1","mode":0,"sorting_weight":3,"value":"84","state":"84"}

event: state
data: {"id":"number-1","value":"85","state":"85"}

event: state
data: {"id":"number-4","name":"4","icon":"","entity_category":0,"min_value":"0","max_value":"100","step":"1","mode":0,"sorting_weight":4,"value":"84","state":"84"}

event: state
data: {"id":"number-4","value":"85","state":"85"}

event: state
data: {"id":"number-5","name":"5","icon":"","entity_category":0,"min_value":"0","max_value":"100","step":"1","mode":0,"sorting_weight":5,"value":"84","state":"84"}

number 5 was the first event received on refresh and since it did not include DETAIL_ALL and thus no sorting_weight it got put on the list in the first place. Then it got 1, 2 ,3 but with DETAIL_ALL. So 1, 2 ,3 got sorted correctly and put in the list after 5. Then it received an state update on number 1 and since 1 is already in the list and sorted it just updated the state. Then it got 4 with DETAIL_ALL with got put in the list and got sorted. After that 4 got an state update.

Then comes the interesting part. It received number 5 with DETAIL_ALL, but since 5 is allready in the list it ignores the sorting weight and it should even ignore the max and min value but since you set it to 0 and 100 it is not noticed since it defaults to 0 and 100 if no min and max is received.

[RFDarter]

You can see it pretty good here. The Order it got displayed was 4 2 1 5 3

And here is the events stream:

event: state
data: {"id":"number-4","value":"453","state":"453"}

event: state
data: {"id":"number-2","value":"453","state":"453"}

event: state
data: {"id":"number-1","name":"1","icon":"","entity_category":0,"min_value":"50","max_value":"500","step":"1","mode":0,"sorting_weight":1,"value":"452","state":"452"}

event: state
data: {"id":"number-1","value":"453","state":"453"}

event: state
data: {"id":"number-5","value":"453","state":"453"}

event: state
data: {"id":"number-2","name":"2","icon":"","entity_category":0,"min_value":"50","max_value":"500","step":"1","mode":0,"sorting_weight":2,"value":"453","state":"453"}

event: state
data: {"id":"number-3","value":"453","state":"453"}

event: state
data: {"id":"number-3","name":"3","icon":"","entity_category":0,"min_value":"50","max_value":"500","step":"1","mode":0,"sorting_weight":3,"value":"453","state":"453"}

event: state
data: {"id":"number-4","name":"4","icon":"","entity_category":0,"min_value":"50","max_value":"500","step":"1","mode":0,"sorting_weight":4,"value":"453","state":"453"}

event: state
data: {"id":"number-5","name":"5","icon":"","entity_category":0,"min_value":"50","max_value":"500","step":"1","mode":0,"sorting_weight":5,"value":"453","state":"453"}

[stas-sl]

It seems like a reasonable explanation. Why not update all properties, including min/max/sorting_weight, whenever new data arrives - whether it comes with the initial data or later. And if sorting_weight is present, trigger re-sorting?

[RFDarter]

It turns out all properties get updated on arrival of new data

Object.assign(this.entities[idx], data);

But sorting of the list is only trigged on the first occurrence of the entity.

I guess we could check if sorting_weight is in the data and then trigger sorting, but then if no sorting_weight is set at all in the config which would sort the entity in alphabetical order the order would still be messed up and jump around based on the order it received the events.

I think a cleaner solution would be to split up the events in a init event or something like that and the state event and only react to state events of entity's that set a init event already.

EDIT: Like i expected even with no sorting_weight set the order gets messed up.

https://github.com/user-attachments/assets/f0503180-782d-4c9b-a684-40a432efce8a

[stas-sl]

It seems that re-sorting should also be triggered when the name property is received, or any property that affects the sorting order, such as entity_category. From the logs, it could be seen that only three properties are sent when the state or value changes: id, state, and value. This causes the item to be placed randomly initially, and when the name property arrives later, it updates without triggering a re-sort, so the item remains in its initial position.

I believe, your suggestion to ignore all state changes until the “init” message is received could also work. However, I’m unsure of the best method to distinguish these messages. Checking for the presence of a specific property?

[RFDarter]

I believe, your suggestion to ignore all state changes until the “init” message is received could also work. However, I’m unsure of the best method to distinguish these messages. Checking for the presence of a specific property?

I decided to simply let the esp send a init_entity event instead of a state event when iterating to the list of entity's and the Browser side react to this init_entity event and put the entity in the list and sort it. Only if the entity is in that list it will react to state events sent form that entity.

The "Problem" with this solution is that if the Browser misses that init_entity event, the entity will never show up even if the esp is sending state events.