sfu-db / dataprep

Open-source low code data preparation library in python. Collect, clean and visualization your data in python with a few lines of code.
http://dataprep.ai
MIT License
2.01k stars 204 forks source link

Add multiple query parameter support to connector.query() #786

Open Fuyuzora opened 2 years ago

Fuyuzora commented 2 years ago

Is your feature request related to a problem? Please describe. Currently the q parameter only support one query parameter, but the target API may support many query parameters. For example, GitHub allows searching for multiple topics within a single query:

q=topic:python+topic:distributed+topic:machine-learning

However it will return empty df from connector.query() function

Describe the solution you'd like I wish in future releases that support for chaining multiple search parameters can be added to the connector.query() function

Describe alternatives you've considered Currently, I am calling multiple requests with one parameter each and joining the dataframes retrieved from each call to obtain the final result. The solution is far less efficient than directly utilizing GitHub's API

Additional context A query that will search for repositories containing multiple topics, works if directly copied and pasted into search bar:

https://api.github.com/search/repositories?q=topic:python+topic:distributed+topic:machine-learning

A function call that will return a dataframe from GitHub:

await api_conn.query('CONFIG_HERE', q='topic:python')

A function call that will not return the desired dataframe

await api_conn.query('CONFIG_HERE', q='topic:python+topic:distributed+topic:machine-learning')