400 Login failed on [toloka] provider

[olegmiercoles]

When try to execute any search on toloka provider it returns no links. From log debug I can see errors of 400 bad requests and information that Request Header or Cookie Too Large. Not sure is it something about client, configuration or the provider itself.

I've verified that the login/password are correct. I've also get try to lgin from postman using this form-data from config

"{'username': USERNAME, 'password': PASSWORD, 'autologin': '1', 'login': 'Enter'}"

and it seems to be working, at least it returns me redirect instead of bad request.

Here is the part of log debug:

2024-09-22 12:49:48.502 T:4449    debug <general>: [script.elementum.burst] Searching with payload (general): {'proxy_url': '', 'internal_proxy_url': 'http://10.10.10.108:65222', 'elementum_url': 'http://10.10.10.108:65220', 'silent': False, 'skip_auth': False, 'query': 'FOO'}
2024-09-22 12:49:48.505 T:4449  warning <general>: [script.elementum.burst] Burstin' with Гуртом
2024-09-22 12:49:48.506 T:4449  warning <general>: [script.elementum.burst] No 'en' translation available...
2024-09-22 12:49:48.511 T:3816    debug <general>: ------ Window Init (DialogExtendedProgressBar.xml) ------
2024-09-22 12:49:48.512 T:4449    debug <general>: [script.elementum.burst] Translated titles from Elementum: {'source': 'FOO', 'original': 'FOO'}
2024-09-22 12:49:48.513 T:4468    debug <general>: [script.elementum.burst] [toloka] Processing toloka with general method
2024-09-22 12:49:48.514 T:4468    debug <CSettingsManager>: requested setting (filter_music) was not found.
2024-09-22 12:49:48.515 T:4468    debug <general>: [script.elementum.burst] [toloka] General URL: https://toloka.to/tracker.php?nm=QUERYEXTRA&o=10
2024-09-22 12:49:48.517 T:4468    debug <general>: [script.elementum.burst] [toloka] execute_process for toloka with <function extract_torrents at 0x7e114938>
2024-09-22 12:49:48.518 T:4449    debug <general>: [script.elementum.burst] Timer: 0s / 27s
2024-09-22 12:49:48.527 T:4468    debug <CAddonSettings[0@plugin.video.elementum]>: trying to load setting definitions from old format...
2024-09-22 12:49:48.549 T:4468    debug <general>: [script.elementum.burst] [toloka] Queries: ['{title}']
2024-09-22 12:49:48.550 T:4468    debug <general>: [script.elementum.burst] [toloka] Extras:  ['']
2024-09-22 12:49:48.551 T:4468    debug <general>: [script.elementum.burst] [toloka] Before keywords - Query: '{title}' - Extra: ''
2024-09-22 12:49:48.552 T:4468  warning <general>: [script.elementum.burst] [toloka] Falling back to original title in absence of None language title
2024-09-22 12:49:48.557 T:4468  warning <general>: [script.elementum.burst] [toloka] Using translated 'original' title 'FOO'
2024-09-22 12:49:48.559 T:4468    debug <general>: [script.elementum.burst] [toloka] After keywords  - Query: '%D1%80%D0%B0%D1%82%D0%B0%D1%82%D1%83%D0%B9' - Extra: ''
2024-09-22 12:49:48.560 T:4468    debug <general>: [script.elementum.burst] -   toloka query: '%D1%80%D0%B0%D1%82%D0%B0%D1%82%D1%83%D0%B9'
2024-09-22 12:49:48.561 T:4468    debug <general>: [script.elementum.burst] --  toloka url_search before token: 'https://toloka.to/tracker.php?nm=FOO&o=10'
2024-09-22 12:49:48.561 T:4468    debug <general>: [script.elementum.burst] --- toloka using POST payload: {}
2024-09-22 12:49:48.562 T:4468    debug <general>: [script.elementum.burst] ----toloka filtering with post_data: {}
2024-09-22 12:49:48.562 T:4468    debug <CSettingsManager>: requested setting (toloka_passkey) was not found.
2024-09-22 12:49:48.577 T:4468    debug <general>: [script.elementum.burst] Opening URL: b'https://toloka.to/login.php'
2024-09-22 12:49:48.770 T:4449    debug <general>: [script.elementum.burst] Timer: 0s / 27s
2024-09-22 12:49:49.029 T:4468     info <general>: Skipped 1 duplicate messages..
2024-09-22 12:49:49.029 T:4468    debug <general>: [script.elementum.burst] Status for b'https://toloka.to/login.php' : 400
2024-09-22 12:49:49.031 T:4468  critical <general>: [script.elementum.burst] [toloka] Login failed: 400
2024-09-22 12:49:49.034 T:4468    debug <general>: [script.elementum.burst] [toloka] Failed login content: '<html>\r\n<head><title>400 Request Header Or Cookie Too Large</title></head>\r\n<body bgcolor="white">\r\n<center><h1>400 Bad Request</h1></center>\r\n<center>Request Header Or Cookie Too Large</center>\r\n<hr><center>nginx</center>\r\n</body>\r\n</html>\r\n<!-- a padding to disable MSIE and Chrome friendly error page -->\r\n<!-- a padding to disable MSIE and Chrome friendly error page -->\r\n<!-- a padding to disable MSIE and Chrome friendly error page -->\r\n<!-- a padding to disable MSIE and Chrome friendly error page -->\r\n<!-- a padding to disable MSIE and Chrome friendly error page -->\r\n<!-- a padding to disable MSIE and Chrome friendly error page -->\r\n'
2024-09-22 12:49:49.047 T:4468  warning <general>: [script.elementum.burst] [toloka] >>              Гуртом returned  0 results in 0.5 seconds

System: Android TV Kodi version: 21.1 Elementum version: 0.1.103 Elementum burst version: 0.0.89

Any suggestions will be appreciate.

[antonsoroko]

@olegmiercoles

1. Maybe it is because of long cookie.

There is a file with cookies in .kodi/temp/burst/common_cookies.jar (see https://kodi.wiki/view/Kodi_data_folder for location of .kodi in your OS) It is a plain text file, you can open it and search for toloka and see if cookie is "bad" (maybe it has some garbage and it is indeed too long). Anyway - remove the line for toloka and try again. also, you can backup that file and remove the file completely.

Also, in burst settings in "maintenance" tab you can remove all cookies - in case if you unable to get access to file system of your device.

2. There is no "login_headers": field for toloka so it should not be "Request Header" issue. https://github.com/elgatito/script.elementum.burst/blob/97dcfb60aa43fa646712c6ec60fb9fc7ee80ecc0/burst/providers/providers.json#L2381

[olegmiercoles]

Thank you, @antonsoroko I did as you suggested: I've renamed common_cookies.jar to common_cookies_bkp.jar and it helped. Probably I will keep backup-file for some time, but later will remove it.

Thank you again for assistance!

[antonsoroko]

@olegmiercoles I am just curious - have you tried to take a look into that file? Interesting how a line for toloka looks like. Probably it has some garbage for some reason. If you remove toloka_sid from that line - you can share it here (so it will not have your login cookie). Up to you of course.

[olegmiercoles]

Sure. I did my the best to cleanup all the hashes and IDs. Hope, I didn't miss anything :)

First line for toloka_data was pretty huge combo of hash+autologinid+hash+userid+hash And toloka_302_u also contained around 8.000 characters hash inside.

#LWP-Cookies-2.0
Set-Cookie3: toloka_data="something-was-here-like-b%432%3b%9Bs%3d%autologinid%here-as-well-something-likeb%432%3b%9Bs%3d%userid-and-here-the-same"; path="/"; domain="toloka.to"; path_spec; secure; expires="2025-05-29 18:56:57Z"; httponly=None; version=0
Set-Cookie3: toloka_302_tt=there-was-some-numbers; path="/"; domain="toloka.to"; path_spec; secure; expires="2025-05-29 18:56:57Z"; version=0
Set-Cookie3: toloka_302_f="there-was-something-like-in-the-first-line-a%54b%33%"; path="/"; domain="toloka.to"; path_spec; secure; expires="2025-05-29 18:56:57Z"; version=0
Set-Cookie3: toloka_302_uf=there-was-some-numbers; path="/"; domain="toloka.to"; path_spec; secure; expires="2025-05-29 18:56:57Z"; version=0
Set-Cookie3: toloka_302_u="about-8.000-characters-was-here"; path="/"; domain="toloka.to"; path_spec; secure; expires="2025-05-29 18:56:57Z"; version=0
Set-Cookie3: toloka___tt=there-was-some-numbers; path="/"; domain="toloka.to"; path_spec; secure; expires="2025-03-04 19:58:37Z"; version=0
Set-Cookie3: toloka___f="there-was-something-like-in-the-first-line-a%54b%33%"; path="/"; domain="toloka.to"; path_spec; secure; expires="2025-03-04 19:58:37Z"; version=0
Set-Cookie3: toloka___uf=0; path="/"; domain="toloka.to"; path_spec; secure; expires="2025-03-04 19:58:37Z"; version=0
Set-Cookie3: toloka___u="there-was-something-like-in-the-first-line-a%54b%33%"; path="/"; domain="toloka.to"; path_spec; secure; expires="2025-03-04 19:58:37Z"; version=0

[antonsoroko]

@olegmiercoles thanks! so looks like by default many popular HTTP servers have total limit for HTTP Header (and cookie is a part of header) as 8KB. so looks like in total your cookies were >8KB, thus i think we found the root cause. somehow toloka website generated such long cookie.

i guess we could create a "blacklist" for cookies (e.g. ignore and do not save some cookies), although i am not sure if we really need it, since i see such issue for the first time. maybe @elgatito can add more ideas.

but anyway, thanks for info. if there will be more issues like this then we can circle back to this.

[elgatito]

@antonsoroko Quick googling says there is nothing "ready" in Python's requests to control size of a request to avoid such errors.

I also have problems with Toloka, even with sync, it does something (I was not debugging those), that invalidates a session everywhere, not only on Elementum/Burst side.

Not sure if we should/can do something with it.