Sinaptik-AI / pandas-ai

Chat with your database (SQL, CSV, pandas, polars, mongodb, noSQL, etc). PandasAI makes data analysis conversational using LLMs (GPT 3.5 / 4, Anthropic, VertexAI) and RAG.
https://pandas-ai.com
Other
12.75k stars 1.23k forks source link

Enable referencing columns from different frames in a smartdatalake or agent for data merging & curation #1051

Closed davidbioinf closed 3 months ago

davidbioinf commented 6 months ago

🚀 The feature

Currently, when I ask smartdatalake or agent with multiple dataframes to do a lookup from a column in one dataframe to the values in another column from another dataframe, it's not able to figure out which column belongs to which dataframe and keeps erroring out. Setting up name= parameter in the smartdataframe and then adding the smartdf to smartdl still does not let me reference the dataframes by name.

Motivation, pitch

This feature would be exceptionally powerful as beginning of most data engineering use cases result from joining data from multiple files that are formatted quite differently and need complex joining.

Alternatives

No response

Additional context

No response