apache / beam

Apache Beam is a unified programming model for Batch and Streaming data processing.
https://beam.apache.org/
Apache License 2.0
7.85k stars 4.25k forks source link

[Failing Test]: Replace missing values with Column Mean #32867

Closed ujjwalrajanand closed 1 week ago

ujjwalrajanand commented 1 week ago

What happened?

https://github.com/vaishnavipandey-vp/Apache-Beam-YAML/blob/c467c59f5f13a00514e90be1cfd505982577524b/Apache-Beam-YAML/Pipelines/calculateRowMean.yaml#L14C14-L17C81

Able to replace missing values with the row mean but we need to find a way to replace it with the column mean instead.

Issue Failure

Failure: Test is continually failing

Issue Priority

Priority: 1 (unhealthy code / failing or flaky postcommit so we cannot be sure the product is healthy)

Issue Components

robertwb commented 1 week ago

MapToFields applies to a single row at a time. I would suggest starting a discussion on the users list about how best to do this.