vanna-ai / vanna

๐Ÿค– Chat with your SQL database ๐Ÿ“Š. Accurate Text-to-SQL Generation via LLMs using RAG ๐Ÿ”„.
https://vanna.ai/docs/
MIT License
11.01k stars 858 forks source link

ใ€featใ€‘ hive/kyuubi and presto/trino jdbc support #364

Open zyclove opened 5 months ago

zyclove commented 5 months ago

Is your feature request related to a problem? Please describe.

Visual display and analysis of data in big data scenarios requires support for automatic SQL generation.

Describe the solution you'd like

Support hive/spark-sql and trino/presto SQL automatic generation

Describe alternatives you've considered Read hive-metastore metadata collection, automatically generate SQL according to the supported engine (hive/spark/kyuubi/presto/trino) and data table structure, and display the data visually

zainhoda commented 5 months ago

This should already be possible if you select "Other Database" in the documentation: https://vanna.ai/docs/other-database-openai-vanna-vannadb/

Basically Vanna just needs you to provide a method called run_sql that takes in a SQL statement as a string and returns a pandas dataframe. As long as you provide Vanna that function, everything else should work similarly.

zyclove commented 5 months ago

@zainhoda

Can the JayDeBeApi framework be introduced to support connecting to the database through jdbc jar? It is also very convenient to use, more versatile and flexible.

import jaydebeapi

# Set JDBC driver path and connection URL
driver = "org.apache.kyuubi.jdbc.KyuubiHiveDriver"
url = "jdbc:kyuubi://host:port/default"
jdbc_driver_path = ["/path/to/kyuubi-hive-jdbc-*.jar"]

# Connect to the database using JayDeBeApi
conn = jaydebeapi.connect(driver, url, ["user", "password"], jdbc_driver_path)

# Create a cursor object
cursor = conn.cursor()

# Execute the SQL query
cursor.execute("SELECT * FROM example_table LIMIT 10")

# Retrieve query results
result_set = cursor.fetchall()

# Process the results
for row in result_set:
    print(row)

# Close the cursor and the connection
cursor.close()
conn.close()
image

https://kyuubi.readthedocs.io/en/v1.9.0/client/python/jaydebeapi.html

image