eakmanrq / sqlframe

Turning PySpark Into a Universal DataFrame API
https://sqlframe.readthedocs.io/en/stable/
MIT License
290 stars 9 forks source link

feat: add openai enrichment support #35

Closed eakmanrq closed 4 months ago

eakmanrq commented 4 months ago

Adds the ability to use ChatGPT to enrich generated SQL in order to make it more like human SQL. The SQL used by SQLFrame when executing queries does not use optimization or ChatGPT so this just affects when a user calls df.sql() directly in order to see the generated SQL.

Video: https://www.loom.com/share/3fab8493818b46f7bff03faf101d2bb6?sid=ba027a0c-4ab7-4531-a4f8-f1a95b70d7e6