moxious / triage

testing triage actions for issues
0 stars 1 forks source link

Log panel infinite loop of empty logs #331

Open tonypowa opened 4 months ago

tonypowa commented 4 months ago

What happened?

Grafana 10.4.0 (latest as of writing)

I am streaming log messages to a log panel from an MQTT datasource. It works fine for a few minutes, and then the log panel starts spooling out empty logs at a very high rate:

image

The actual logs are only being sent once per second, and always contain some kind of content.

Hitting refresh on the page temporarily corrects the issue, but it quickly comes back.

I have other types of visualization working with MQTT, so I think the problem is in the log panel, not in the MQTT datasource.

What did you expect to happen?

Actual log messages from my MQTT topic are displayed, and not an infinite scroll of empty logs.

Did this work before?

This is the first time I've tried the log panel with the MQTT data source.

How do we reproduce it?

Periodically send some kind of log message through MQTT, set up the MQTT datasource, create a log panel and select the log topic.

Is the bug inside a dashboard panel?

| Key | Value |

|--|--|

| Panel | logs @ 10.0.12 |

| Grafana | 10.4.0 (03f502a94d17f7dc4e6c34acdf8428aedd986e4c) // Open Source |

Panel debug snapshot dashboard ```json { "panels": [ { "datasource": { "type": "grafana", "uid": "grafana" }, "gridPos": { "h": 13, "w": 15, "x": 0, "y": 0 }, "id": 2, "options": { "showTime": true, "showLabels": false, "showCommonLabels": false, "wrapLogMessage": false, "prettifyLogMessage": false, "enableLogDetails": true, "dedupStrategy": "none", "sortOrder": "Descending" }, "pluginVersion": "10.0.12", "targets": [ { "refId": "A", "datasource": { "type": "grafana", "uid": "grafana" }, "queryType": "snapshot", "snapshot": [ { "schema": { "meta": { "typeVersion": [ 0, 0 ], "channel": "ds/e92d200a-67e3-4a4b-9f5d-f7612be011d8/500ms/logs/ferret" }, "fields": [ { "name": "Time", "type": "time", "config": {} }, { "name": "log", "type": "string", "config": {} }, { "name": "stream", "type": "string", "config": {} }, { "name": "time", "type": "string", "config": {} } ] }, "data": { "values": [ [ 1710854030807, 1710854031810, 1710854032813, 1710854033815, 1710854034819, 1710854035822, 1710854036823, 1710854037825, 1710854038828, 1710854039830, 1710854040834, 1710854041836, 1710854042839, 1710854043841, 1710854044843, 1710854045846, 1710854046848, 1710854047852, 1710854048855, 1710854049858, 1710854050902, 1710854051864, 1710854052867, 1710854053868, 1710854054871, 1710854055874, 1710854056876, 1710854057879, 1710854058882, 1710854059884, 1710854060887, 1710854061889, 1710854062892, 1710854063895, 1710854064898, 1710854065900, 1710854066902, 1710854067905, 1710854068907, 1710854069910, 1710854070913, 1710854071915, 1710854072920, 1710854073922, 1710854074925, 1710854075927, 1710854076930, 1710854077933, 1710854078935, 1710854079938, 1710854080942, 1710854081944, 1710854082947, 1710854083951, 1710854084953, 1710854085957, 1710854086959, 1710854087962, 1710854088964, 1710854089968, 1710854090970, 1710854091973, 1710854092975, 1710854093978, 1710854094979, 1710854095982, 1710854096984, 1710854097988, 1710854098990, 1710854099992, 1710854100995, 1710854101997, 1710854103000, 1710854104002, 1710854105005, 1710854106008, 1710854107012, 1710854108015, 1710854109018, 1710854110021, 1710854111022, 1710854112067, 1710854113028, 1710854114030, 1710854115033, 1710854116035, 1710854117038, 1710854118041, 1710854119043, 1710854120046, 1710854121048, 1710854122051, 1710854123054, 1710854124057, 1710854125059, 1710854126062, 1710854127065, 1710854128068, 1710854129070, 1710854130073, 1710854131075, 1710854132078, 1710854133081, 1710854134083, 1710854135087, 1710854136090, 1710854137091, 1710854138094, 1710854139097, 1710854140099, 1710854141101, 1710854142104, 1710854143106, 1710854144109, 1710854145111, 1710854146114, 1710854147115, 1710854148118, 1710854149120, 1710854150123, 1710854151126, 1710854152129, 1710854153131, 1710854154134, 1710854155136, 1710854156139, 1710854157142, 1710854158145, 1710854159147, 1710854160150, 1710854161152, 1710854162154, 1710854163155, 1710854164158, 1710854165160, 1710854166163, 1710854167165, 1710854168168, 1710854169171, 1710854170174, 1710854171218, 1710854172179, 1710854173182, 1710854174184, 1710854175187, 1710854176189, 1710854177191, 1710854178193, 1710854179196, 1710854180198, 1710854181201, 1710854182203, 1710854183207, 1710854184210, 1710854185213, 1710854186215, 1710854187218, 1710854188220, 1710854189223, 1710854190226, 1710854191229, 1710854192231, 1710854193233, 1710854194236, 1710854195239, 1710854196242, 1710854197244, 1710854198247, 1710854199250, 1710854200253, 1710854201255, 1710854202258, 1710854203260, 1710854204262, 1710854205265, 1710854206267, 1710854207272, 1710854208275, 1710854209278, 1710854210281, 1710854211284, 1710854212287, 1710854213289, 1710854214292, 1710854215294, 1710854216296, 1710854217299, 1710854218302, 1710854219304, 1710854220307, 1710854221309, 1710854222312, 1710854223314, 1710854224317, 1710854225321, 1710854226323, 1710854227325, 1710854228328, 1710854229331, 1710854230334, 1710854231382, 1710854232341, 1710854233343, 1710854234346, 1710854235351, 1710854236354, 1710854237356, 1710854238360, 1710854239362, 1710854240368, 1710854241370, 1710854242371, 1710854243380, 1710854244381, 1710854245384, 1710854246387, 1710854247390, 1710854248392, 1710854249395, 1710854250398 ], [ "INFO 2024-03-19T08:13:50.395 ferret test 1301", "INFO 2024-03-19T08:13:51.397 ferret test 1302", "INFO 2024-03-19T08:13:52.398 ferret test 1303", "INFO 2024-03-19T08:13:53.400 ferret test 1304", "INFO 2024-03-19T08:13:54.402 ferret test 1305", "INFO 2024-03-19T08:13:55.405 ferret test 1306", "INFO 2024-03-19T08:13:56.405 ferret test 1307", "INFO 2024-03-19T08:13:57.407 ferret test 1308", "INFO 2024-03-19T08:13:58.408 ferret test 1309", "INFO 2024-03-19T08:13:59.409 ferret test 1310", "INFO 2024-03-19T08:14:00.410 ferret test 1311", "INFO 2024-03-19T08:14:01.411 ferret test 1312", "INFO 2024-03-19T08:14:02.413 ferret test 1313", "INFO 2024-03-19T08:14:03.415 ferret test 1314", "INFO 2024-03-19T08:14:04.416 ferret test 1315", "INFO 2024-03-19T08:14:05.417 ferret test 1316", "INFO 2024-03-19T08:14:06.419 ferret test 1317", "INFO 2024-03-19T08:14:07.420 ferret test 1318", "INFO 2024-03-19T08:14:08.421 ferret test 1319", "INFO 2024-03-19T08:14:09.423 ferret test 1320", "INFO 2024-03-19T08:14:10.425 ferret test 1321", "INFO 2024-03-19T08:14:11.427 ferret test 1322", "INFO 2024-03-19T08:14:12.428 ferret test 1323", "INFO 2024-03-19T08:14:13.430 ferret test 1324", "INFO 2024-03-19T08:14:14.431 ferret test 1325", "INFO 2024-03-19T08:14:15.433 ferret test 1326", "INFO 2024-03-19T08:14:16.434 ferret test 1327", "INFO 2024-03-19T08:14:17.436 ferret test 1328", "INFO 2024-03-19T08:14:18.437 ferret test 1329", "INFO 2024-03-19T08:14:19.439 ferret test 1330", "INFO 2024-03-19T08:14:20.441 ferret test 1331", "INFO 2024-03-19T08:14:21.442 ferret test 1332", "INFO 2024-03-19T08:14:22.444 ferret test 1333", "INFO 2024-03-19T08:14:23.445 ferret test 1334", "INFO 2024-03-19T08:14:24.447 ferret test 1335", "INFO 2024-03-19T08:14:25.448 ferret test 1336", "INFO 2024-03-19T08:14:26.449 ferret test 1337", "INFO 2024-03-19T08:14:27.451 ferret test 1338", "INFO 2024-03-19T08:14:28.453 ferret test 1339", "INFO 2024-03-19T08:14:29.454 ferret test 1340", "INFO 2024-03-19T08:14:30.455 ferret test 1341", "INFO 2024-03-19T08:14:31.457 ferret test 1342", "INFO 2024-03-19T08:14:32.459 ferret test 1343", "INFO 2024-03-19T08:14:33.460 ferret test 1344", "INFO 2024-03-19T08:14:34.461 ferret test 1345", "INFO 2024-03-19T08:14:35.463 ferret test 1346", "INFO 2024-03-19T08:14:36.464 ferret test 1347", "INFO 2024-03-19T08:14:37.465 ferret test 1348", "INFO 2024-03-19T08:14:38.467 ferret test 1349", "INFO 2024-03-19T08:14:39.469 ferret test 1350", "INFO 2024-03-19T08:14:40.471 ferret test 1351", "INFO 2024-03-19T08:14:41.472 ferret test 1352", "INFO 2024-03-19T08:14:42.473 ferret test 1353", "INFO 2024-03-19T08:14:43.475 ferret test 1354", "INFO 2024-03-19T08:14:44.476 ferret test 1355", "INFO 2024-03-19T08:14:45.478 ferret test 1356", "INFO 2024-03-19T08:14:46.479 ferret test 1357", "INFO 2024-03-19T08:14:47.481 ferret test 1358", "INFO 2024-03-19T08:14:48.483 ferret test 1359", "INFO 2024-03-19T08:14:49.485 ferret test 1360", "INFO 2024-03-19T08:14:50.487 ferret test 1361", "INFO 2024-03-19T08:14:51.489 ferret test 1362", "INFO 2024-03-19T08:14:52.490 ferret test 1363", "INFO 2024-03-19T08:14:53.491 ferret test 1364", "INFO 2024-03-19T08:14:54.493 ferret test 1365", "INFO 2024-03-19T08:14:55.494 ferret test 1366", "INFO 2024-03-19T08:14:56.496 ferret test 1367", "INFO 2024-03-19T08:14:57.497 ferret test 1368", "INFO 2024-03-19T08:14:58.499 ferret test 1369", "INFO 2024-03-19T08:14:59.500 ferret test 1370", "INFO 2024-03-19T08:15:00.501 ferret test 1371", "INFO 2024-03-19T08:15:01.503 ferret test 1372", "INFO 2024-03-19T08:15:02.505 ferret test 1373", "INFO 2024-03-19T08:15:03.506 ferret test 1374", "INFO 2024-03-19T08:15:04.507 ferret test 1375", "INFO 2024-03-19T08:15:05.509 ferret test 1376", "INFO 2024-03-19T08:15:06.510 ferret test 1377", "INFO 2024-03-19T08:15:07.512 ferret test 1378", "INFO 2024-03-19T08:15:08.513 ferret test 1379", "INFO 2024-03-19T08:15:09.515 ferret test 1380", "INFO 2024-03-19T08:15:10.517 ferret test 1381", "INFO 2024-03-19T08:15:11.519 ferret test 1382", "INFO 2024-03-19T08:15:12.520 ferret test 1383", "INFO 2024-03-19T08:15:13.521 ferret test 1384", "INFO 2024-03-19T08:15:14.523 ferret test 1385", "INFO 2024-03-19T08:15:15.525 ferret test 1386", "INFO 2024-03-19T08:15:16.525 ferret test 1387", "INFO 2024-03-19T08:15:17.527 ferret test 1388", "INFO 2024-03-19T08:15:18.528 ferret test 1389", "INFO 2024-03-19T08:15:19.529 ferret test 1390", "INFO 2024-03-19T08:15:20.530 ferret test 1391", "INFO 2024-03-19T08:15:21.533 ferret test 1392", "INFO 2024-03-19T08:15:22.534 ferret test 1393", "INFO 2024-03-19T08:15:23.536 ferret test 1394", "INFO 2024-03-19T08:15:24.538 ferret test 1395", "INFO 2024-03-19T08:15:25.539 ferret test 1396", "INFO 2024-03-19T08:15:26.542 ferret test 1397", "INFO 2024-03-19T08:15:27.544 ferret test 1398", "INFO 2024-03-19T08:15:28.545 ferret test 1399", "INFO 2024-03-19T08:15:29.547 ferret test 1400", "INFO 2024-03-19T08:15:30.548 ferret test 1401", "INFO 2024-03-19T08:15:31.549 ferret test 1402", "INFO 2024-03-19T08:15:32.551 ferret test 1403", "INFO 2024-03-19T08:15:33.552 ferret test 1404", "INFO 2024-03-19T08:15:34.553 ferret test 1405", "INFO 2024-03-19T08:15:35.555 ferret test 1406", "INFO 2024-03-19T08:15:36.557 ferret test 1407", "INFO 2024-03-19T08:15:37.559 ferret test 1408", "INFO 2024-03-19T08:15:38.560 ferret test 1409", "INFO 2024-03-19T08:15:39.562 ferret test 1410", "INFO 2024-03-19T08:15:40.564 ferret test 1411", "INFO 2024-03-19T08:15:41.565 ferret test 1412", "INFO 2024-03-19T08:15:42.568 ferret test 1413", "INFO 2024-03-19T08:15:43.569 ferret test 1414", "INFO 2024-03-19T08:15:44.571 ferret test 1415", "INFO 2024-03-19T08:15:45.573 ferret test 1416", "INFO 2024-03-19T08:15:46.574 ferret test 1417", "INFO 2024-03-19T08:15:47.576 ferret test 1418", "INFO 2024-03-19T08:15:48.577 ferret test 1419", "INFO 2024-03-19T08:15:49.579 ferret test 1420", "INFO 2024-03-19T08:15:50.581 ferret test 1421", "INFO 2024-03-19T08:15:51.582 ferret test 1422", "INFO 2024-03-19T08:15:52.584 ferret test 1423", "INFO 2024-03-19T08:15:53.588 ferret test 1424", "INFO 2024-03-19T08:15:54.589 ferret test 1425", "INFO 2024-03-19T08:15:55.591 ferret test 1426", "INFO 2024-03-19T08:15:56.593 ferret test 1427", "INFO 2024-03-19T08:15:57.594 ferret test 1428", "INFO 2024-03-19T08:15:58.596 ferret test 1429", "INFO 2024-03-19T08:15:59.597 ferret test 1430", "INFO 2024-03-19T08:16:00.599 ferret test 1431", "INFO 2024-03-19T08:16:01.601 ferret test 1432", "INFO 2024-03-19T08:16:02.602 ferret test 1433", "INFO 2024-03-19T08:16:03.604 ferret test 1434", "INFO 2024-03-19T08:16:04.605 ferret test 1435", "INFO 2024-03-19T08:16:05.607 ferret test 1436", "INFO 2024-03-19T08:16:06.609 ferret test 1437", "INFO 2024-03-19T08:16:07.609 ferret test 1438", "INFO 2024-03-19T08:16:08.611 ferret test 1439", "INFO 2024-03-19T08:16:09.613 ferret test 1440", "INFO 2024-03-19T08:16:10.614 ferret test 1441", "INFO 2024-03-19T08:16:11.615 ferret test 1442", "INFO 2024-03-19T08:16:12.617 ferret test 1443", "INFO 2024-03-19T08:16:13.617 ferret test 1444", "INFO 2024-03-19T08:16:14.619 ferret test 1445", "INFO 2024-03-19T08:16:15.620 ferret test 1446", "INFO 2024-03-19T08:16:16.621 ferret test 1447", "INFO 2024-03-19T08:16:17.623 ferret test 1448", "INFO 2024-03-19T08:16:18.625 ferret test 1449", "INFO 2024-03-19T08:16:19.625 ferret test 1450", "INFO 2024-03-19T08:16:20.627 ferret test 1451", "INFO 2024-03-19T08:16:21.628 ferret test 1452", "INFO 2024-03-19T08:16:22.629 ferret test 1453", "INFO 2024-03-19T08:16:23.631 ferret test 1454", "INFO 2024-03-19T08:16:24.631 ferret test 1455", "INFO 2024-03-19T08:16:25.633 ferret test 1456", "INFO 2024-03-19T08:16:26.635 ferret test 1457", "INFO 2024-03-19T08:16:27.636 ferret test 1458", "INFO 2024-03-19T08:16:28.637 ferret test 1459", "INFO 2024-03-19T08:16:29.639 ferret test 1460", "INFO 2024-03-19T08:16:30.641 ferret test 1461", "INFO 2024-03-19T08:16:31.643 ferret test 1462", "INFO 2024-03-19T08:16:32.644 ferret test 1463", "INFO 2024-03-19T08:16:33.646 ferret test 1464", "INFO 2024-03-19T08:16:34.647 ferret test 1465", "INFO 2024-03-19T08:16:35.649 ferret test 1466", "INFO 2024-03-19T08:16:36.651 ferret test 1467", "INFO 2024-03-19T08:16:37.653 ferret test 1468", "INFO 2024-03-19T08:16:38.653 ferret test 1469", "INFO 2024-03-19T08:16:39.655 ferret test 1470", "INFO 2024-03-19T08:16:40.656 ferret test 1471", "INFO 2024-03-19T08:16:41.657 ferret test 1472", "INFO 2024-03-19T08:16:42.659 ferret test 1473", "INFO 2024-03-19T08:16:43.660 ferret test 1474", "INFO 2024-03-19T08:16:44.662 ferret test 1475", "INFO 2024-03-19T08:16:45.663 ferret test 1476", "INFO 2024-03-19T08:16:46.664 ferret test 1477", "INFO 2024-03-19T08:16:47.666 ferret test 1478", "INFO 2024-03-19T08:16:48.667 ferret test 1479", "INFO 2024-03-19T08:16:49.669 ferret test 1480", "INFO 2024-03-19T08:16:50.669 ferret test 1481", "INFO 2024-03-19T08:16:51.671 ferret test 1482", "INFO 2024-03-19T08:16:52.673 ferret test 1483", "INFO 2024-03-19T08:16:53.675 ferret test 1484", "INFO 2024-03-19T08:16:54.677 ferret test 1485", "INFO 2024-03-19T08:16:55.679 ferret test 1486", "INFO 2024-03-19T08:16:56.680 ferret test 1487", "INFO 2024-03-19T08:16:57.681 ferret test 1488", "INFO 2024-03-19T08:16:58.682 ferret test 1489", "INFO 2024-03-19T08:16:59.684 ferret test 1490", "INFO 2024-03-19T08:17:00.685 ferret test 1491", "INFO 2024-03-19T08:17:01.687 ferret test 1492", "INFO 2024-03-19T08:17:02.689 ferret test 1493", "INFO 2024-03-19T08:17:03.691 ferret test 1494", "INFO 2024-03-19T08:17:04.692 ferret test 1495", "INFO 2024-03-19T08:17:05.694 ferret test 1496", "INFO 2024-03-19T08:17:06.696 ferret test 1497", "INFO 2024-03-19T08:17:07.699 ferret test 1498", "INFO 2024-03-19T08:17:08.700 ferret test 1499", "INFO 2024-03-19T08:17:09.701 ferret test 1500", "INFO 2024-03-19T08:17:10.703 ferret test 1501", "INFO 2024-03-19T08:17:11.705 ferret test 1502", "INFO 2024-03-19T08:17:12.707 ferret test 1503", "INFO 2024-03-19T08:17:13.708 ferret test 1504", "INFO 2024-03-19T08:17:14.709 ferret test 1505", "INFO 2024-03-19T08:17:15.711 ferret test 1506", "INFO 2024-03-19T08:17:16.712 ferret test 1507", "INFO 2024-03-19T08:17:17.713 ferret test 1508", "INFO 2024-03-19T08:17:18.715 ferret test 1509", "INFO 2024-03-19T08:17:19.717 ferret test 1510", "INFO 2024-03-19T08:17:20.720 ferret test 1511", "INFO 2024-03-19T08:17:21.722 ferret test 1512", "INFO 2024-03-19T08:17:22.724 ferret test 1513", "INFO 2024-03-19T08:17:23.725 ferret test 1514", "INFO 2024-03-19T08:17:24.727 ferret test 1515", "INFO 2024-03-19T08:17:25.729 ferret test 1516", "INFO 2024-03-19T08:17:26.730 ferret test 1517", "INFO 2024-03-19T08:17:27.732 ferret test 1518", "INFO 2024-03-19T08:17:28.733 ferret test 1519", "INFO 2024-03-19T08:17:29.736 ferret test 1520" ], [ "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt", "mqtt" ], [ "2024-03-19T13:13:50.395829Z", "2024-03-19T13:13:51.397546Z", "2024-03-19T13:13:52.399400Z", "2024-03-19T13:13:53.401457Z", "2024-03-19T13:13:54.403611Z", "2024-03-19T13:13:55.405345Z", "2024-03-19T13:13:56.406021Z", "2024-03-19T13:13:57.407463Z", "2024-03-19T13:13:58.409059Z", "2024-03-19T13:13:59.409970Z", "2024-03-19T13:14:00.410614Z", "2024-03-19T13:14:01.412099Z", "2024-03-19T13:14:02.414074Z", "2024-03-19T13:14:03.415540Z", "2024-03-19T13:14:04.417131Z", "2024-03-19T13:14:05.418148Z", "2024-03-19T13:14:06.419559Z", "2024-03-19T13:14:07.421118Z", "2024-03-19T13:14:08.421973Z", "2024-03-19T13:14:09.423594Z", "2024-03-19T13:14:10.425785Z", "2024-03-19T13:14:11.427368Z", "2024-03-19T13:14:12.428937Z", "2024-03-19T13:14:13.430478Z", "2024-03-19T13:14:14.431866Z", "2024-03-19T13:14:15.433380Z", "2024-03-19T13:14:16.435036Z", "2024-03-19T13:14:17.437158Z", "2024-03-19T13:14:18.438151Z", "2024-03-19T13:14:19.439647Z", "2024-03-19T13:14:20.441297Z", "2024-03-19T13:14:21.442808Z", "2024-03-19T13:14:22.444343Z", "2024-03-19T13:14:23.445959Z", "2024-03-19T13:14:24.447352Z", "2024-03-19T13:14:25.448787Z", "2024-03-19T13:14:26.449971Z", "2024-03-19T13:14:27.452143Z", "2024-03-19T13:14:28.453816Z", "2024-03-19T13:14:29.454709Z", "2024-03-19T13:14:30.456229Z", "2024-03-19T13:14:31.457877Z", "2024-03-19T13:14:32.459687Z", "2024-03-19T13:14:33.461126Z", "2024-03-19T13:14:34.461976Z", "2024-03-19T13:14:35.463419Z", "2024-03-19T13:14:36.465441Z", "2024-03-19T13:14:37.465956Z", "2024-03-19T13:14:38.468014Z", "2024-03-19T13:14:39.469998Z", "2024-03-19T13:14:40.471507Z", "2024-03-19T13:14:41.472992Z", "2024-03-19T13:14:42.473981Z", "2024-03-19T13:14:43.475771Z", "2024-03-19T13:14:44.477180Z", "2024-03-19T13:14:45.478472Z", "2024-03-19T13:14:46.480145Z", "2024-03-19T13:14:47.482254Z", "2024-03-19T13:14:48.483872Z", "2024-03-19T13:14:49.486022Z", "2024-03-19T13:14:50.487702Z", "2024-03-19T13:14:51.489271Z", "2024-03-19T13:14:52.490276Z", "2024-03-19T13:14:53.491760Z", "2024-03-19T13:14:54.493716Z", "2024-03-19T13:14:55.495072Z", "2024-03-19T13:14:56.496674Z", "2024-03-19T13:14:57.497990Z", "2024-03-19T13:14:58.499376Z", "2024-03-19T13:14:59.500840Z", "2024-03-19T13:15:00.502240Z", "2024-03-19T13:15:01.504019Z", "2024-03-19T13:15:02.505498Z", "2024-03-19T13:15:03.506302Z", "2024-03-19T13:15:04.508004Z", "2024-03-19T13:15:05.509561Z", "2024-03-19T13:15:06.511120Z", "2024-03-19T13:15:07.512746Z", "2024-03-19T13:15:08.514272Z", "2024-03-19T13:15:09.516365Z", "2024-03-19T13:15:10.517844Z", "2024-03-19T13:15:11.519442Z", "2024-03-19T13:15:12.521090Z", "2024-03-19T13:15:13.521981Z", "2024-03-19T13:15:14.523459Z", "2024-03-19T13:15:15.525385Z", "2024-03-19T13:15:16.526012Z", "2024-03-19T13:15:17.527591Z", "2024-03-19T13:15:18.529026Z", "2024-03-19T13:15:19.529952Z", "2024-03-19T13:15:20.531267Z", "2024-03-19T13:15:21.533395Z", "2024-03-19T13:15:22.534790Z", "2024-03-19T13:15:23.536782Z", "2024-03-19T13:15:24.538449Z", "2024-03-19T13:15:25.540236Z", "2024-03-19T13:15:26.542523Z", "2024-03-19T13:15:27.544251Z", "2024-03-19T13:15:28.545638Z", "2024-03-19T13:15:29.547245Z", "2024-03-19T13:15:30.548681Z", "2024-03-19T13:15:31.550011Z", "2024-03-19T13:15:32.551494Z", "2024-03-19T13:15:33.552928Z", "2024-03-19T13:15:34.554111Z", "2024-03-19T13:15:35.555628Z", "2024-03-19T13:15:36.557578Z", "2024-03-19T13:15:37.559739Z", "2024-03-19T13:15:38.561639Z", "2024-03-19T13:15:39.563110Z", "2024-03-19T13:15:40.564916Z", "2024-03-19T13:15:41.566670Z", "2024-03-19T13:15:42.568297Z", "2024-03-19T13:15:43.570209Z", "2024-03-19T13:15:44.571803Z", "2024-03-19T13:15:45.573539Z", "2024-03-19T13:15:46.574986Z", "2024-03-19T13:15:47.576445Z", "2024-03-19T13:15:48.578083Z", "2024-03-19T13:15:49.579788Z", "2024-03-19T13:15:50.581508Z", "2024-03-19T13:15:51.583128Z", "2024-03-19T13:15:52.587095Z", "2024-03-19T13:15:53.589218Z", "2024-03-19T13:15:54.590142Z", "2024-03-19T13:15:55.592109Z", "2024-03-19T13:15:56.593496Z", "2024-03-19T13:15:57.594751Z", "2024-03-19T13:15:58.596791Z", "2024-03-19T13:15:59.598061Z", "2024-03-19T13:16:00.599622Z", "2024-03-19T13:16:01.601386Z", "2024-03-19T13:16:02.603004Z", "2024-03-19T13:16:03.604447Z", "2024-03-19T13:16:04.606351Z", "2024-03-19T13:16:05.607971Z", "2024-03-19T13:16:06.609415Z", "2024-03-19T13:16:07.610149Z", "2024-03-19T13:16:08.611614Z", "2024-03-19T13:16:09.613556Z", "2024-03-19T13:16:10.614271Z", "2024-03-19T13:16:11.615913Z", "2024-03-19T13:16:12.617376Z", "2024-03-19T13:16:13.618140Z", "2024-03-19T13:16:14.619563Z", "2024-03-19T13:16:15.621012Z", "2024-03-19T13:16:16.621940Z", "2024-03-19T13:16:17.623880Z", "2024-03-19T13:16:18.625327Z", "2024-03-19T13:16:19.626044Z", "2024-03-19T13:16:20.627776Z", "2024-03-19T13:16:21.629170Z", "2024-03-19T13:16:22.630083Z", "2024-03-19T13:16:23.631518Z", "2024-03-19T13:16:24.632070Z", "2024-03-19T13:16:25.633972Z", "2024-03-19T13:16:26.635549Z", "2024-03-19T13:16:27.636551Z", "2024-03-19T13:16:28.638002Z", "2024-03-19T13:16:29.639978Z", "2024-03-19T13:16:30.642094Z", "2024-03-19T13:16:31.643432Z", "2024-03-19T13:16:32.644869Z", "2024-03-19T13:16:33.646533Z", "2024-03-19T13:16:34.647947Z", "2024-03-19T13:16:35.649668Z", "2024-03-19T13:16:36.651971Z", "2024-03-19T13:16:37.653410Z", "2024-03-19T13:16:38.654088Z", "2024-03-19T13:16:39.655677Z", "2024-03-19T13:16:40.657287Z", "2024-03-19T13:16:41.658048Z", "2024-03-19T13:16:42.659477Z", "2024-03-19T13:16:43.660985Z", "2024-03-19T13:16:44.662483Z", "2024-03-19T13:16:45.663919Z", "2024-03-19T13:16:46.664520Z", "2024-03-19T13:16:47.666422Z", "2024-03-19T13:16:48.667945Z", "2024-03-19T13:16:49.669398Z", "2024-03-19T13:16:50.670225Z", "2024-03-19T13:16:51.672075Z", "2024-03-19T13:16:52.674417Z", "2024-03-19T13:16:53.676062Z", "2024-03-19T13:16:54.677844Z", "2024-03-19T13:16:55.679316Z", "2024-03-19T13:16:56.680992Z", "2024-03-19T13:16:57.682256Z", "2024-03-19T13:16:58.682977Z", "2024-03-19T13:16:59.684470Z", "2024-03-19T13:17:00.686456Z", "2024-03-19T13:17:01.688129Z", "2024-03-19T13:17:02.689803Z", "2024-03-19T13:17:03.691496Z", "2024-03-19T13:17:04.693125Z", "2024-03-19T13:17:05.694577Z", "2024-03-19T13:17:06.697735Z", "2024-03-19T13:17:07.699356Z", "2024-03-19T13:17:08.700882Z", "2024-03-19T13:17:09.702140Z", "2024-03-19T13:17:10.704214Z", "2024-03-19T13:17:11.705772Z", "2024-03-19T13:17:12.707245Z", "2024-03-19T13:17:13.708735Z", "2024-03-19T13:17:14.709887Z", "2024-03-19T13:17:15.711734Z", "2024-03-19T13:17:16.713023Z", "2024-03-19T13:17:17.714378Z", "2024-03-19T13:17:18.715933Z", "2024-03-19T13:17:19.718432Z", "2024-03-19T13:17:20.720740Z", "2024-03-19T13:17:21.722939Z", "2024-03-19T13:17:22.724668Z", "2024-03-19T13:17:23.726309Z", "2024-03-19T13:17:24.727767Z", "2024-03-19T13:17:25.729394Z", "2024-03-19T13:17:26.730737Z", "2024-03-19T13:17:27.732415Z", "2024-03-19T13:17:28.733935Z", "2024-03-19T13:17:29.737868Z" ] ] } } ] } ], "title": "Reproduced with embedded data", "type": "logs" }, { "gridPos": { "h": 7, "w": 9, "x": 15, "y": 0 }, "id": 5, "options": { "content": "\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
Panellogs @ 10.0.12
QueriesA[grafana-mqtt-datasource]
DataStreaming 1 frames, 4 fields, 220 rows
Grafana10.4.0 (03f502a94d17f7dc4e6c34acdf8428aedd986e4c) // Open Source
", "mode": "html" }, "title": "Debug info", "type": "text" }, { "id": 6, "title": "Original Panel JSON", "type": "text", "gridPos": { "h": 13, "w": 9, "x": 15, "y": 7 }, "options": { "content": "{\n \"datasource\": {\n \"type\": \"grafana-mqtt-datasource\",\n \"uid\": \"e92d200a-67e3-4a4b-9f5d-f7612be011d8\"\n },\n \"gridPos\": {\n \"h\": 13,\n \"w\": 15,\n \"x\": 0,\n \"y\": 0\n },\n \"id\": 2,\n \"options\": {\n \"showTime\": true,\n \"showLabels\": false,\n \"showCommonLabels\": false,\n \"wrapLogMessage\": false,\n \"prettifyLogMessage\": false,\n \"enableLogDetails\": true,\n \"dedupStrategy\": \"none\",\n \"sortOrder\": \"Descending\"\n },\n \"pluginVersion\": \"10.0.12\",\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"grafana-mqtt-datasource\",\n \"uid\": \"e92d200a-67e3-4a4b-9f5d-f7612be011d8\"\n },\n \"refId\": \"A\",\n \"topic\": \"logs/ferret\"\n }\n ],\n \"title\": \"Panel Title\",\n \"type\": \"logs\"\n}", "mode": "code", "code": { "language": "json", "showLineNumbers": true, "showMiniMap": true } } }, { "id": 3, "title": "Data from panel above", "type": "table", "datasource": { "type": "datasource", "uid": "-- Dashboard --" }, "gridPos": { "h": 7, "w": 15, "x": 0, "y": 13 }, "options": { "showTypeIcons": true }, "targets": [ { "datasource": { "type": "datasource", "uid": "-- Dashboard --" }, "panelId": 2, "withTransforms": true, "refId": "A" } ] } ], "schemaVersion": 37, "title": "Debug: Panel Title // 2024-03-19 08:17:30", "tags": [ "debug", "debug-logs" ], "time": { "from": "2024-03-19T13:12:30.795Z", "to": "2024-03-19T13:17:30.795Z" } } ```

Environment (with versions)?

Grafana: 10.4.0

OS: Linux Mint

Browser: Firefox

Grafana platform?

Docker

Datasource(s)?

MQTT datasource

tonypowa commented 4 months ago

This issue is a test copy of an issue in another repo. Original issue: https://github.com/grafana/grafana/issues/#84737

moxious commented 4 months ago

Summary: User tonypowa reported an issue with Grafana 10.4.0 where a log panel begins outputting an infinite loop of empty logs after a few minutes, despite the actual MQTT data source sending log messages only once per second with content. Refreshing the page temporarily fixes the problem, suggesting the issue lies in the log panel rather than the MQTT data source.

moxious commented 4 months ago

Hello @tonypowa, thanks for reporting this issue. It seems like the problem revolves around the MQTT datasource in combination with the log panel. The Grafana plugin that appears to handle the functionality is a good candidate for further investigation. You might want to report this issue in the Plugins Platform project to ensure it receives attention from the appropriate team.

moxious commented 4 months ago

Elaboration:

Hello @tonypowa, thank you for providing a detailed description of the issue with Grafana's log panel. In order to assist you better and potentially identify the problem, we need a bit more information:

  1. Can you confirm if there are any error messages in the Grafana server logs or JavaScript console in the browser when the log panel starts spooling out empty logs?

  2. Could you provide the MQTT datasource configuration, ensuring any sensitive information such as passwords or access tokens are redacted?

  3. Is there a pattern to when the issue starts occurring, such as when the dashboard has been running for a specific amount of time, or after a certain number of log messages have been received?

  4. Are there any other network conditions or events that seem to coincide with the issue starting, such as peak usage times or network instability?

  5. You mentioned the issue is resolved temporarily by refreshing the page. Does this happen in other browsers as well, or have you experienced this only in Firefox?

  6. Could you please share more details about your Docker setup, including the Docker version and any relevant configuration details for the Grafana container?

  7. Are you able to reproduce this issue in a different setup/environment (e.g., local installation without Docker, different machine, etc.)?

Any additional information or observations that you think might be relevant to diagnose the issue would be greatly appreciated. Feel free to attach any logs or configurations that could help us investigate further.

Thank you!