Got more than 8190 bytes when reading Header value is too long.

[salmosri]

Hi Team,

When attempting to carry out a move of Spotify -> Google the following error appears which causes the app to hang (but not crash):

Listening on http://0.0.0.0:3132 Please open your browser window to http://localhost:3132 Gathering tracks for playlist Saved Tracks (784) Error handling request Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/aiohttp/client_reqrep.py", line 559, in start (message, payload) = yield from self._protocol.read() File "/usr/local/lib/python3.6/site-packages/aiohttp/streams.py", line 509, in read yield from self._waiter File "/usr/local/lib/python3.6/site-packages/aiohttp/client_proto.py", line 165, in data_received messages, upgraded, tail = self._parser.feed_data(data) File "aiohttp/_http_parser.pyx", line 274, in aiohttp._http_parser.HttpParser.feed_data (aiohttp/_http_parser.c:4364) File "aiohttp/_http_parser.pyx", line 382, in aiohttp._http_parser.cb_on_header_value (aiohttp/_http_parser.c:6391) aiohttp.http_exceptions.LineTooLong: 400, message='Got more than 8190 bytes when reading Header value is too long.'

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/aiohttp/web_protocol.py", line 422, in start resp = yield from self._request_handler(request) File "/usr/local/lib/python3.6/site-packages/aiohttp/web.py", line 306, in _handle resp = yield from handler(request) File "/usr/local/lib/python3.6/site-packages/pyportify/middlewares.py", line 18, in index_handler ret = yield from handler(request) File "/usr/local/lib/python3.6/site-packages/pyportify/app.py", line 117, in transfer_start yield from transfer_playlists(request, s, g, lists) File "/usr/local/lib/python3.6/site-packages/pyportify/app.py", line 169, in transfer_playlists done = yield from asyncio.gather(*tasks) File "/usr/local/lib/python3.6/site-packages/pyportify/app.py", line 242, in search_gm_track track = yield from g.find_best_track(search_query) File "/usr/local/lib/python3.6/site-packages/pyportify/google.py", line 56, in find_best_track data = yield from self.search_all_access(search_query) File "/usr/local/lib/python3.6/site-packages/pyportify/google.py", line 48, in search_all_access 'ct': '1,2,3,4,6,7,8,9', File "/usr/local/lib/python3.6/site-packages/pyportify/google.py", line 113, in _http_get params=merged_params, File "/usr/local/lib/python3.6/site-packages/aiohttp/helpers.py", line 97, in iter ret = yield from self._coro File "/usr/local/lib/python3.6/site-packages/aiohttp/client.py", line 241, in _request yield from resp.start(conn, read_until_eof) File "/usr/local/lib/python3.6/site-packages/aiohttp/client_reqrep.py", line 564, in start message=exc.message, headers=exc.headers) from exc aiohttp.client_exceptions.ClientResponseError: 400, message='Got more than 8190 bytes when reading Header value is too long.'

==============

This issue exists while using the install method of brew or downloading the dmg and running the pyportify-copyall script.

I am using MacOS Sierra 10.12.6

Let me know if there is anything i can do to resolve this or provide more information!

Cheers, Shadi

[rckclmbr]

It looks like aiohttp limits header length to 8190 in their response parser. I don't see any easy way in their API to change this length, since this is hidden behind a few layers of abstraction. I'll dig into this and see what I can do.

There's a C and python parser, but they both have this param.

https://github.com/aio-libs/aiohttp/blob/e7c9390111932dd6dbb642170d7a0da1876271ec/aiohttp/http_parser.py#L59

[robhazan]

I'm encountering the same error on Ubuntu. Any debugging information I can provide to help get this resolved? Anything I can change to avoid this error?

[robhazan]

@rckclmbr - friendly ping to see if there's anything that can be done about this. Any workarounds?

[rckclmbr]

@robhazan there's not much I can do about this short of moving away from aiohttp, since aiohttp doesn't support what we need it to. We need to wait until https://github.com/aio-libs/aiohttp/issues/2304 is resolved (they set the milestone for 3.1) then we can work around this.

The other approach is to figure out why the header is exceeding 8190 -- it's not all users, and I can't troubleshoot it since it's not happening to me. Maybe there's something we can do to make google not have such a large header, if we could find what it actually is.

The hackish solution is to manually edit aiohttp to allow a longer header. I don't think it's a good idea to make a release with that though, for a variety of reasons...

[dIgor93]

Hello! I found some solution for same problem. Pass the handler_args parameter when creating app. It will pass to parser.

web.Application(handler_args={'max_field_size': 16380})

[magnus919]

Running into this years later with Home Assistant.

[danelec-hpm]

Even 2024, with all the tokens and suff we have in headers today, who was crazy enough to set an 8 kb limit. Did we not learn from Bill Gates "640kb should be enough for everyone" resulting in the Gate A20 in the Bios Keyboard to be used to access High Memory above 640kb.