abhijithneilabraham / tableQA

AI Tool for querying natural language on tabular data.
GNU General Public License v3.0
295 stars 47 forks source link

* Support more dataframes #23

Open abhijithneilabraham opened 3 years ago

abhijithneilabraham commented 3 years ago

-Currently the feature supports csv files only. However, integrating more dataframes is easy. Go through the get_dataframe() method in data_utils.py and include support to detect the incoming file and parse the dataframe from it.

rhamnett commented 3 years ago

It'd be good to just be able to do something like:

agent = Agent(df)
abhijithneilabraham commented 3 years ago

It'd be good to just be able to do something like:

agent = Agent(df)

Sure. We will update it asap.

rhamnett commented 3 years ago

@abhijithneilabraham keep up the great work, looking forward to contributing

abhijithneilabraham commented 3 years ago

Thanks! @rhamnett The repo is updated with the changes you suggested, kindly go through the readme and examples and report any bugs.

rhamnett commented 3 years ago

Hello @abhijithneilabraham! That's great! I gave it a quick test on a sample dataset i created from my own columns and the basic functionality seems to work!

Some support for date types etc - datetime64 etc would be really good and then getting the model to understand what today/yesterday/this week/last week means from the current date.

agent.query_db("how many website visits were there in the last week?") agent.query_db("how many orders have I had today?") agent.query_db("how many orders have I had for the week beginning 31st August 2020?")

This kind of thing 👍

abhijithneilabraham commented 3 years ago

Makes sense. In fact, I was already building something like that, but the issue is to map the natural language to datetime without approaching it in a rule based way. Let me think about this and will update asap.

rhamnett commented 3 years ago

Makes sense. In fact, I was already building something like that, but the issue is to map the natural language to datetime without approaching it in a rule based way. Let me think about this and will update asap.

Yes, that's whats going to make it interesting :)

TheurgicDuke771 commented 3 years ago

Hi, What will be the df when use MySQL db (have multiple table)?

abhijithneilabraham commented 3 years ago

Haven't implemeted multi-table support yet. We need to join tables in that case, it seems. Will plan that for future. Thanks 😊 .

TheurgicDuke771 commented 3 years ago

thanks for the quick reply. Then df should be select * from table, right?

abhijithneilabraham commented 3 years ago

That's correct. Download a table as a dataframe and pass it to this.

You could also contribute a feature if you can enable something like agent=Agent(db_url, table_name). Refer agent.py,data_utils.py

TheurgicDuke771 commented 3 years ago

Sure, will check that. And send you a pr if get anything solid.

abhijithneilabraham commented 3 years ago

Thanks!

abhijithneilabraham commented 3 years ago

Hello @abhijithneilabraham! That's great! I gave it a quick test on a sample dataset i created from my own columns and the basic functionality seems to work!

Some support for date types etc - datetime64 etc would be really good and then getting the model to understand what today/yesterday/this week/last week means from the current date.

agent.query_db("how many website visits were there in the last week?") agent.query_db("how many orders have I had today?") agent.query_db("how many orders have I had for the week beginning 31st August 2020?")

This kind of thing 👍

This has been done too! We have a Date class in column_types.py

dharmesh2002 commented 2 years ago

Hi Abhijit Have used this tool, it is an excellent tool. Have tested some of my scenarios, they are working fine. Have one query, do joins works in this tool like left join or right join

abhijithneilabraham commented 2 years ago

Hi @dharmesh2002 , Can you be more specific where you have tested a scenario with SQL joins?

dharmesh2002 commented 2 years ago

Hi @abhijithneilabraham , i haven't tested sql queries, what i need is, to connect sql and than to join with tables. So that's why i was checking with you whether that is feasible?

dharmesh2002 commented 2 years ago

@abhijithneilabraham - is it possible to add new functions like group by or having clause?

abhijithneilabraham commented 2 years ago

Yes. Can you add support for them?

On Thu, Apr 21, 2022, 7:48 AM dharmesh2002 @.***> wrote:

@abhijithneilabraham https://github.com/abhijithneilabraham - is it possible to add new functions like group by or having clause?

— Reply to this email directly, view it on GitHub https://github.com/abhijithneilabraham/tableQA/issues/23#issuecomment-1104637670, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIOHO46G666EDB72L7LWKMDVGC3FTANCNFSM4RZTADAQ . You are receiving this because you were mentioned.Message ID: @.***>

dharmesh2002 commented 2 years ago

@abhijithneilabraham sorry I didn’t understood, can u pls help me where to add group by function, I tried adding but it didn’t work

abhijithneilabraham commented 2 years ago

@dharmesh2002 Can you make a pull request with your code and tag it here?

snehithakrishna2 commented 2 years ago

@dharmesh2002 Can you make a pull request with your code and tag it here? hi, @abhijithneilabraham https://github.com/abhijithneilabraham/tableQA/pull/61

dharmesh2002 commented 2 years ago

@abhijithneilabraham please if you can have a look on pool request we shared above

abhijithneilabraham commented 2 years ago

Hi, I reviewed the code. Have you tried testing it on an actual database and see if the results suit your needs in a natural language query?

On Fri, Apr 22, 2022, 9:27 PM dharmesh2002 @.***> wrote:

@abhijithneilabraham https://github.com/abhijithneilabraham please if you can have a look on pool request we shared above

— Reply to this email directly, view it on GitHub https://github.com/abhijithneilabraham/tableQA/issues/23#issuecomment-1106649895, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIOHO44IC3HVSCJZKISVCWTVGLD6XANCNFSM4RZTADAQ . You are receiving this because you were mentioned.Message ID: @.***>

shubhamrangate2413 commented 4 months ago

Sure, will check that. And send you a pr if get anything solid. @TheurgicDuke771 , I am working on a similar use case of connecting the Agent with Postgresql db. Are you able to crack anything for making a connection with db?