Fix streaming by applying streaming function after block is generated instead putting it together with other compute apply func.
The reason to only stream each row after blocks is generated is because if we put stream together as other computed function in []applyFunc slice, like this,
sinking to Bigquery might fail because the types of rows is not parsed since there is no operation of parsing row with its defined types based on schema. In particularly unix() timestamp which is type int64 before parsing the row with it's schema, by right it should be presto.timestamp type and this may cause some issues.
For the sake of simplicity, I just move the streaming operation after the blocks is created, so that i can just stream the row which i derived from presto column together with the respective data type.
Fix streaming by applying streaming function after block is generated instead putting it together with other compute apply func.
[]applyFunc
slice, like this, sinking to Bigquery might fail because the types of rows is not parsed since there is no operation of parsing row with its defined types based on schema. In particularlyunix()
timestamp which is typeint64
before parsing the row with it's schema, by right it should bepresto.timestamp
type and this may cause some issues.