search

teragrep / pth_10

Data Processing Language (DPL) translator for Apache Spark
GNU Affero General Public License v3.0
0 stars 2 forks source link

accum doesn't work without the optional output field #240

Closed 51-code closed 4 months ago

51-code commented 5 months ago

Describe the bug

The query: index=abc earliest=-10y | accum offset results in a StreamingQueryException and a RuntimeException.

org.apache.spark.sql.streaming.StreamingQueryException: Job aborted due to stage failure: Task 66 in stage 211.0 failed 4 times ...

java.lang.RuntimeException: java.lang.String is not a valid external type for schema of bigint ...

However, providing the output field works: index=abc earliest=-10y | accum offset AS a_offset

Expected behavior

Should work without the output field, resulting in the overwrite of the field that was specified in the query.

How to reproduce

Run the queries above.

Screenshots

Software version

PTH-10: 4.18.0-8-ge7e4190c

Desktop (please complete the following information if relevant):

Additional context

eemhu commented 5 months ago

Note: It works if the source field is StringType.

eemhu commented 4 months ago

this was fixed in 4.20.1