search

apache / incubator-wayang

Apache Wayang(incubating) is the first cross-platform data processing system.
https://wayang.incubator.apache.org/
Apache License 2.0
174 stars 70 forks source link

Add wayang-api-json and wayang-api-python to create WayangPlans from python #436

Closed juripetersen closed 2 months ago

juripetersen commented 2 months ago

This PRs description can be found in one of our blog posts:

https://wayang.apache.org/blog/wayang-python-api

juripetersen commented 2 months ago

This PR will also entail an issue arising with typing. Currently the interchange format between python and wayang is mainly handled by serializing strings and parsing them to the desired type in the udf. It is of course desirable to add type hints to operators that allow the parsing to happen implicitly. The issue will be added after merging the PR.

2pk03 commented 2 months ago

@zkaoudi @kbeedkar - pls review too before we merge.

kbeedkar commented 2 months ago

looks good to me +1