Closed mcepl closed 2 years ago
@nsapa - Do you have time (and interest) in looking into this?
@JimmXinu I am taking this.
@mcepl Did chrome_content.py log any exception? The requested fanfic have 122 chapters so testing is going to be slow.
A ValueError here mean that the header is missing the content size.
The proxy have is a simple header : type_of_content||size_in_bytes$END_OF_HEADER$
.
I currently do not see where the proxy could fail sending a size...
Well, it is true, that the proxy run was rather wild as well. Multiple crashes, sometimes even multiple ones on one page.
2022-07-15 22:54:17.837 CEST INFO selenium_recovery Look like we are operational again. Retrying https://www.fanfiction.net/s/10339852/5/The-Many-Harry-Potters-of-Little-Hangleton
2022-07-15 22:54:19.607 CEST CRITICAL mainloop Unrecoverable error from Selenium: unknown error: unexpected command response
(Session info: chrome=103.0.5060.114). Killing this instance...
2022-07-15 22:54:19.607 CEST ERROR selenium_recovery Selenium error while getting https://www.fanfiction.net/s/10339852/5/The-Many-Harry-Potters-of-Little-Hangleton, recovery started.
2022-07-15 22:54:19.608 CEST INFO ProxiedBrowser(suicide) Killing Chrome with pid 24994 as last ressort cleanup.
2022-07-15 22:54:19.608 CEST INFO ProxiedBrowser(suicide) Killing chrome driver with pid 24995 as last ressort cleanup.
...
2022-07-15 22:54:22.020 CEST INFO ProxiedBrowser(init) chromedriver version 103.0.5060.53 (a1711811edd74ff1cf2150f36ffa3b0dae40b17f-refs/branch-heads/5060@{#853}) running as pid 25503 driving Chrome version 103.0.5060.114 running as pid 25502
2022-07-15 22:54:22.199 CEST INFO ProxiedBrowser(init) Trying to load existing cookie...
2022-07-15 22:54:26.827 CEST WARNING main Exception WebDriverException in the main loop (Message: unknown error: unexpected command response
(Session info: chrome=103.0.5060.114)
At this point, Selenium is totally broken and I don't have a recovery path in the main Exception handler (because I didn't think this could happen). Look like it is a known issue in Chromium 103 and the fix is merged for 104.
Anyway, I will send a PR to make nsapa_proxy.py handle a broken proxy response (some sanity check that raise exceptions.FailedToDownload("nsapa_proxy: invalid header")).
@nsapa Just that I have collected a lovely log when trying to update https://www.fanfiction.net/s/14001550/1/Harry-Potter-and-the-Expectations-of-the-Future
@nsapa - I don't think I got a PR for this?
This should make fanficfare detect proxy issue. I am also updating the proxy to alert the user if the Chrome version is incompatible.
Great, thanks!