teragrep / pth_10

Data Processing Language (DPL) translator for Apache Spark
GNU Affero General Public License v3.0
0 stars 2 forks source link

Aggregation command earliest and latest throw NullPointerException if no data #233

Closed eemhu closed 5 months ago

eemhu commented 5 months ago

Describe the bug

No data generates NullPointerException.

org.apache.spark.sql.streaming.StreamingQueryException: Job aborted due to stage failure: Task 0 in stage 9.0 failed 4 times, most recent failure: Lost task 0.3 in stage 9.0 (TID 13, xxx, executor 21): java.lang.NullPointerException
    at com.teragrep.pth10.ast.commands.aggregate.UDAFs.BufferClasses.TimestampMapBuffer.earliest(TimestampMapBuffer.java:131)
    at com.teragrep.pth10.ast.commands.aggregate.UDAFs.EarliestLatestAggregator_String.finish(EarliestLatestAggregator_String.java:92)
    at com.teragrep.pth10.ast.commands.aggregate.UDAFs.EarliestLatestAggregator_String.finish(EarliestLatestAggregator_String.java:58)

Expected behavior

Should return no data, but also no exception.

How to reproduce

Try running earliest or latest to no rows.

Screenshots

Software version

4.17.0

Desktop (please complete the following information if relevant):

Additional context

eemhu commented 5 months ago

internal pr created

51-code commented 5 months ago

Internal pr reviewed

eemhu commented 5 months ago

internal pr merged