fix "Array cannot have a null element" dataform summary queries

GoogleCloudPlatform / security-analytics

Community Security Analytics provides a set of community-driven audit & threat queries for Google Cloud

Apache License 2.0

323 stars 68 forks source link

fix "Array cannot have a null element" dataform summary queries #46

Closed pfilourenco closed 12 months ago

pfilourenco commented 12 months ago

I had this error running this 2 querys (dataform/defenitions/summary): csa_6_01_summary_hourly.sqlx:

bigquery error: Query error: Array cannot have a null element; error in writing field VMs at [5:1]

csa_6_10_summary_daily.sqlx:

bigquery error: Query error: Array cannot have a null element; error in writing field vm_names at [5:1]

I applied the "IGNORE NULLS" as a fix, maybe this will be useful for the community.

rarsan commented 12 months ago

Thanks for reporting this and the PR fix @pfilourenco !

I suspect this occurs when the flow endpoint is a GKE cluster in which case InstanceDetails fields are not populated as they're not applicable.

In that case, we should probably add the GKE details (or GkeDetails fields) to account for the Pod details instead of VM instance details. I'll track that as a separate enhancement feature.