Query with list of values according to application/x-www-form-urlencoded not possible

[elupus]

Long story short

Making a request according to application/x-www-form-urlencoded for a list of values is not supported using normal params parameter. For example with a parameter string of: {'select' : [1, 2, 3]}

Can't find any good reference to a standard (closest i got was https://stackoverflow.com/questions/2602043/rest-api-best-practice-how-to-accept-list-of-parameter-values-as-input).

Expected behaviour

Resulting query like "?select[]=1&select[]=2&select[]=3" or without the [] "?select=1&select=2&select=3"

Actual behaviour

Traceback (most recent call last):
  File "C:\Users\Joakim\Source\hass\nibeuplink\tests\issue.py", line 13, in <module>
    loop.run_until_complete(go(loop))
  File "C:\Program Files\Python36\lib\asyncio\base_events.py", line 466, in run_until_complete
    return future.result()
  File "C:\Users\Joakim\Source\hass\nibeuplink\tests\issue.py", line 8, in go
    params = {'select[]' : [1, 2, 3]}) as resp:
  File "C:\Users\Joakim\Source\hass\env\lib\site-packages\aiohttp\client.py", line 603, in __aenter__
    self._resp = yield from self._coro
  File "C:\Users\Joakim\Source\hass\env\lib\site-packages\aiohttp\client.py", line 226, in _request
    session=self)
  File "C:\Users\Joakim\Source\hass\env\lib\site-packages\aiohttp\client_reqrep.py", line 79, in __init__
    url2 = url.with_query(params)
  File "C:\Users\Joakim\Source\hass\env\lib\site-packages\yarl\__init__.py", line 768, in with_query
    "should be str or int, got {!r}".format(v))
TypeError: Invalid variable type: mapping value should be str or int, got [1, 2, 3]

Steps to reproduce

import asyncio
import aiohttp

async def go(loop):
    async with aiohttp.ClientSession(loop=loop) as session:
        async with session.get(
                'http://httpbin.org/get',
                params = {'select' : [1, 2, 3]}) as resp:
            data = await resp.json()

loop = asyncio.get_event_loop()
loop.run_until_complete(go(loop))
loop.close()

Your environment

Windows 10 aiohttp=2.2.5 yarl=0.13.0

[asvetlov]

There is no such standard, libraries use different encoding schema. That's why I doubt if yarl/aiohttp should support it: if we choose one convention people will blame us that another convention is not used etc.

[elupus]

I will close this as partly invalid. I had missed the support for MultiDict parameters. They work perfectly fine for passing this data.

[Forever-Young]

My code snippet to overcome this issue (in class inherited from ClientSession)

    async def _request(self, *args, **kwargs):
        params = kwargs.pop('params', None)
        if params:
            new_params = MultiDict()
            if isinstance(params, Mapping):
                for k, v in params.items():
                    if isinstance(v, (list, tuple)):
                        for value in v:
                            new_params.add(k, value)
                    else:
                        new_params.extend({k: v})
            else:
                new_params = params
            kwargs['params'] = new_params
...

(moving from requests from aiohttp, a lot of places uses this feature http://docs.python-requests.org/en/master/user/quickstart/#passing-parameters-in-urls bottom part) Feature itself I think goes from usage of urllib.urlencode with doseq=True

[asvetlov]

Inheritance from ClientSession is deprecated and will be strictly forbidden in future aiohttp releases.

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. If you feel like there's important points made in this discussion, please include those exceprts into that new issue.