teragrep / pth_10

Data Processing Language (DPL) translator for Apache Spark
GNU Affero General Public License v3.0
0 stars 2 forks source link

Aggregate 'sum' does not support multivalue fields #261

Closed eemhu closed 2 months ago

eemhu commented 3 months ago

Describe the bug

%dpl
index=join_json_one earliest="01/01/2021:00:00:00"
| spath
| eval monthYear=strftime(_time,"%Y-%m")
| eval attack = if((operation="attack"),1,0)
| eval defend = if((operation="defend"),1,0)
| eval spy = if((operation="spy"),1,0)
| stats sum(attack) AS attack, sum(defend) AS defend, sum(spy) AS spy by monthYear, user

returns mismatch data type error

Expected behavior

Should sum all of the MV field values together and then all rows together. e.g. [1, 2, 3] [1, 2, 3] -> 12

How to reproduce

Run command

Screenshots

Software version

5.3.0

Desktop (please complete the following information if relevant):

Additional context

Check other aggregate functions as well for the same issue!

If eval method returns MV fields to support them since Spark UDF only allow one return type

eemhu commented 2 months ago

merged