Whether the `apply` function is implemented using the pandas_udf function? - Githubissues

databricks / koalas

Koalas: pandas API on Apache Spark

Apache License 2.0

3.32k stars 356 forks source link

Whether the `apply` function is implemented using the pandas_udf function? #2217

Open RainFung opened 2 years ago

RainFung commented 2 years ago

Whether the apply function is implemented using the pandas_udf function?

itholic commented 2 years ago

Yeah, for some case we use mapInPandas which leveraging the pandas_udf internally.

https://github.com/databricks/koalas/blob/07d846225d27492a9b85e58900018fda576650e3/databricks/koalas/frame.py#L2557-L2563