Migrate schema parsing from Python to Scala - Githubissues

komprenilo / liga

Liga: Let Data Dance with ML Models

Apache License 2.0

10 stars 5 forks source link

Migrate schema parsing from Python to Scala #82

Closed da-tubi closed 1 year ago

da-tubi commented 1 year ago

Having trouble on: https://github.com/liga-ai/liga/blob/v0.2.0/python/liga/spark/sql/schema.py#L45-L63

da-tubi commented 1 year ago

For UDT:

https://github.com/apache/spark/blob/v3.3.1/sql/catalyst/src/main/scala/org/apache/spark/sql/types/UDTRegistration.scala

da-tubi commented 1 year ago

See https://github.com/liga-ai/ligavision/pull/28

da-tubi commented 1 year ago

closed by #94