The connector seems can't work for API with "?" as seperator

[qiushuo222]

Describe the bug The connector seems can't work for API with "?" as seperator. For example, like below link came from previous version of PokéAPI. https://pokeapi.co/api/v2/item-category?stat-boosts

Note that the current version of PokéAPI has been updated recently with "/" as seperator, but when I was working on the assignment, they were turly using "?" as sperator and connector cannot work on it as well as pagination.

To Reproduce To get what is in https://pokeapi.co/api/v2/item-category?stat-boosts:

from dataprep.connector import Connector
conn_poke = Connector("./pokemon")
df = await conn_poke.query('itemcat', type = "itemcat", name = "stat-boosts")

meta.json

{
    "tables": [
        "itemcat"
    ]
}

itemcat.json

{
    "version": 1,
    "request": {
        "url": "https://pokeapi.co/api/v2/item-category/",
        "method": "GET",
        "params": {
            "name": true
        },
        "pagination": {
            "type": "offset",
            "offsetKey": "offset",
            "limitKey": "limit",
            "maxCount": 20
        }
    },

    "response": {
        "ctype": "application/json",
        "tablePath": "$.results[*]",
        "schema": {
            "name": {
                "target": "$.name",
                "type": "string"
            },
            "url": {
                "target": "$.url",
                "type": "string"
            }
        },
        "orient": "records"
    }
}

Expected behavior The connector is supposed to reture a dataframe that the content is results array of json object.

Desktop:

• OS: Windows WSL2 Ubuntu 20.04.3 LTS
• Platform: ipynb in VSCode
• Platform Version:1.63.2
• Python Version 3.8.8
• Dataprep Version: 0.4.1

Additional context Note that the current version of PokéAPI has been updated recently with "/" as seperator, but when I was working on the assignment, they were turly using "?" as sperator and connector cannot work on it as well as pagination. The endpoint link https://pokeapi.co/api/v2/item-category?stat-boosts is still working and still can see the problem.

[wangxiaoying]

Hi @qiushuo222 , I cannot reproduce the error locally. The only thing I modified was the , type = "itemcat" since type is not defined in the config. I also tried pagination and it also works. Can you give the config that could reproduce the error?

I'm not sure whether I understand your question correctly, but if you want to define a non-url parameter variable, you can take a look at name_or_id_or_keyword this example: https://github.com/sfu-db/APIConnectors/blob/develop/api-connectors/ourairport/country.json Basically in your json file you can change the url field as "url": "https://pokeapi.co/api/v2/item-category?{name}".

[qiushuo222]

Hi @wangxiaoying , thanks for your quick feedback. I do submit correct config but wrong code, it should not having "type" in parameter.

I just tried and seems all working fine now, which is so weird for me as when we were working on the assignment, I used same code and config but always got empty dataframe as return.

Sorry for the inconvinient caused here, I shall close this issue.