[Bug] Upgrade from 3.4.1 to 3.4.2 - No data from entity views on dashboards

[andy-5]

Describe the bug

Since upgrading from 3.4.1PE to 3.4.2PE we have several dashboards where no data is shown in charts (previously, they worked fine). As far as I can tell, all failing charts have an entity view defined as the data source for the entity alias.

Also, when accessing some (but not all) of the failing dashboards, the thingsboard.log contains multiple errors like this:

com.fasterxml.jackson.databind.JsonMappingException: Null key for a Map not allowed in JSON (use a converting NullKeySerializer?) (through reference chain: org.thingsboard.server.service.telemetry.cmd.v2.EntityDataUpdate["update"]->java.util.ArrayList[0]->org.thingsboard.server.common.data.query.EntityData["timeseries"])
        at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:283)
        at com.fasterxml.jackson.databind.SerializerProvider.mappingException(SerializerProvider.java:1358)
        at com.fasterxml.jackson.databind.SerializerProvider.reportMappingProblem(SerializerProvider.java:1254)
        at com.fasterxml.jackson.databind.ser.impl.FailingSerializer.serialize(FailingSerializer.java:32)
        at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFieldsUsing(MapSerializer.java:895)
        at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeWithoutTypeInfo(MapSerializer.java:762)
        at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:720)
        at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:35)
        at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:728)
        at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:774)
        at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178)
        at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
        at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
        at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
        at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:728)
        at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:774)
        at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178)
        at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
        at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
        at com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4568)
        at com.fasterxml.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:3821)
        at org.thingsboard.server.service.telemetry.DefaultTelemetryWebSocketService.sendWsMsg(DefaultTelemetryWebSocketService.java:854)
        at org.thingsboard.server.service.telemetry.DefaultTelemetryWebSocketService.sendWsMsg(DefaultTelemetryWebSocketService.java:332)
        at org.thingsboard.server.service.telemetry.DefaultTelemetryWebSocketService.sendWsMsg(DefaultTelemetryWebSocketService.java:326)
        at org.thingsboard.server.service.subscription.TbAbstractSubCtx.sendWsMsg(TbAbstractSubCtx.java:356)
        at org.thingsboard.server.service.subscription.DefaultTbEntityDataSubscriptionService.lambda$handleGetTsCmd$20(DefaultTbEntityDataSubscriptionService.java:610)
        at com.google.common.util.concurrent.AbstractTransformFuture$TransformFuture.doTransform(AbstractTransformFuture.java:242)
        at com.google.common.util.concurrent.AbstractTransformFuture$TransformFuture.doTransform(AbstractTransformFuture.java:232)
        at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

It seems some telemetry data could not be serialized to JSON. But as this error is only logged for some failing dashboards, I'm not sure if it is related.

Your Server Environment

• own setup
  • local infrastructure
  • ThingsBoard Version 3.4.2PE
  • OS Name and Version: Ubuntu 22.04.1 LTS

Your Client Environment

Desktop:

• OS: Windows
• Browser Firefox (but also Edge)
• Version 107

To Reproduce Steps to reproduce the behavior:

1. Define an entity view for some timeseries keys of a device
2. Create a new dashboard, add a chart widget
3. Define an entity alias using the entity view from 1.
4. Select timeseries keys for the chart
5. Save chart
6. Nothing is shown in chart widget on dashboard, even if there is data for the selected timeseries in the selected time range

Expected behavior I expect the chart widget to show the timeseries data.

If the entity alias of the dashboard is changed to the device (instead of the entity view), everything is shown correctly.

[Backdraft007]

Offen it is a problem with browser cache after an upgrade. Please clear the browser cache with CTRL+F5 and try again.

[andy-5]

Thanks for the response. Unfortunately, clearing the browser cache did not help.

With the browser developer tools open (and cache disabled), I can see that the Websocket communication to wss://our.server/api/ws/plugins/telemetry seems to be the problem.

When the data source is an entity view, the data for the dashboard widget is requested with

{"attrSubCmds":[],"tsSubCmds":[],"historyCmds":[],"entityDataCmds":[{"cmdId":1,"tsCmd":{"keys":["MyKey1","MyKey2"],"startTs":1670369280000,"timeWindow":36120000,"interval":120000,"limit":301,"agg":"AVG"}}],"entityDataUnsubscribeCmds":[],"alarmDataCmds":[],"alarmDataUnsubscribeCmds":[],"entityCountCmds":[],"entityCountUnsubscribeCmds":[]}

but there is no answer - and therefore no data is displayed in the widget.

But if I change the data source (entity alias) to the corresponding device, I can see a similar request

{"attrSubCmds":[],"tsSubCmds":[],"historyCmds":[],"entityDataCmds":[{"cmdId":1,"tsCmd":{"keys":["MyKey1","MyKey2"],"startTs":1670370000000,"timeWindow":36120000,"interval":120000,"limit":301,"agg":"AVG"}}],"entityDataUnsubscribeCmds":[],"alarmDataCmds":[],"alarmDataUnsubscribeCmds":[],"entityCountCmds":[],"entityCountUnsubscribeCmds":[]}

followed by regular data updates like the following (note: I removed most of the data values):

{
    "cmdId": 1,
    "data": null,
    "update": [
        {
            "entityId": {
                "entityType": "DEVICE",
                "id": "bad2e900-1df9-11ed-90a5-ed665098dc40"
            },
            "readAttrs": true,
            "readTs": true,
            "latest": {
                "ENTITY_FIELD": {
                    "name": {
                        "ts": 1670406023250,
                        "value": "test-name"
                    },
                    "additionalInfo": {
                        "ts": 1670406023250,
                        "value": "{\"gateway\":false,\"overwriteActivityTime\":false,\"description\":\"test\"}"
                    },
                    "label": {
                        "ts": 1670406023250,
                        "value": "test"
                    }
                }
            },
            "timeseries": {
                "MyKey1": [
                    {
                        "ts": 1670370060000,
                        "value": "-0.8",
                        "count": 2
                    },
                    {
                        "ts": 1670370180000,
                        "value": "-0.8",
                        "count": 2
                    },
                    [...]
                ],
                "MyKey2": [
                    {
                        "ts": 1670370060000,
                        "value": "11.0",
                        "count": 1
                    },
                    {
                        "ts": 1670370420000,
                        "value": "11.0",
                        "count": 1
                    },
                    [...]
                ]
            },
            "aggLatest": {}
        }
    ],
    "errorCode": 0,
    "errorMsg": null,
    "allowedEntities": 10000,
    "cmdUpdateType": "ENTITY_DATA"
}

So I think it is a server-side problem. The server sends device telemetry data over the Websocket connection, but not entity view telemetry data.

[andy-5]

FYI, I just replicated this on demo.thingsboard.io - so it is not related to our server installation.

Entity view telemetry data is not shown on dashboards (because no updates are received through the Websocket connection).

[Webbeh]

Hi.

I have the same issue with 3.4.2 PE.

Oddly enough, not all widgets seem to be affected.

All of those graphs were working pre-update.

[den1jkee]

Hi, Thank you for your feedback. We are always improving our project and have already fixed this bug in the 3.4.3 version of ThingsBoard.

Please follow our Release Notes to see all new additions to the platform in the subsequent releases and bug fixes.

Best regards, ThingsBoard Team.

[andy-5]

Thank you for the update. We've just upgraded and I can confirm that this issue is solved with TB 3.4.3.