teragrep / ajs_01

User interface for Teragrep
Apache License 2.0
0 stars 0 forks source link

Aggregations are behaving weirdly with graph display #260

Open BVVLD opened 2 months ago

BVVLD commented 2 months ago

Describe the bug Behavior of the graphical paragraph results becomes inconsistent, when aggregations are applied:

Expected behavior The data should display consistently, with graphs updating during paragraph running, if server actually updates the results. Results should not be displayed as text, when table is expected.

How to reproduce Create a notebook with several paragraph: paragraph with query including aggregation + groupby with just two fields in the table, paragraph with query including aggregation + groupby with more than two fields in the table, paragraph with pyspark query.

Software version AJS_7.0.0

Additional context Some problems might originate on the server side, including possible problems with configurations saving and applying during the paragraph run. In this case, the issue should be mirrored on zep_01 project and some changes should be applied to UI to prevent these scenarios before problem is fixed on the server.

BVVLD commented 2 months ago

the problem with pyspark seems to originate from server sending paragraph updates with this: { "code": "SUCCESS", "msg": [ { "type": "TABLE", "data": "count(_time)\n0\n" }, { "type": "TEXT", "data": "\n{'id': '72431276-0e55-4570-b4ba-44e50cf5cce9', 'runId': 'a255e464-2252-4113-9780-62b48aebd400', 'name': 'PySparkExample', 'timestamp': '2024-04-26T06:24:20.717Z', 'batchId': 1, 'numInputRows': 0, 'inputRowsPerSecond': 0.0, 'processedRowsPerSecond': 0.0, 'durationMs': {'getEndOffset': 0, 'setOffsetRange': 9, 'triggerExecution': 10}, 'stateOperators': [{'numRowsTotal': 1, 'numRowsUpdated': 0, 'memoryUsedBytes': 446, 'customMetrics': {'loadedMapCacheHitCount': 0, 'loadedMapCacheMissCount': 0, 'stateOnCurrentVersionSizeBytes': 302}}], 'sources': [{'description': 'source@5be392a3', 'startOffset': {'version': 1, 'archiveOffset': {'offset': -9223372036854775808, 'json': '-9223372036854775808'}}, 'endOffset': {'version': 1, 'archiveOffset': {'offset': -9223372036854775808, 'json': '-9223372036854775808'}}, 'numInputRows': 0, 'inputRowsPerSecond': 0.0, 'processedRowsPerSecond': 0.0}], 'sink': {'description': 'MemorySink'}}\n" } ] } i.e. two separate results where one is an incomplete table, and another is a text with stringified data