search

kaiko-ai / typedspark

Column-wise type annotations for pyspark DataFrames
Apache License 2.0
65 stars 4 forks source link

fix DataSetImplements such that we can use functools.reduce again #221

Closed nanne-aben closed 11 months ago

nanne-aben commented 11 months ago

In the current master, the following code results in linting errors:

functools.reduce(
    DataSet.unionByName,
    [create_empty_dataset(spark, A), create_empty_dataset(spark, A)],
)

This is at odds with earlier versions of typedspark. This PR fixes the behaviour so that it's consistent with earlier versions.