search

Iftakharpy / algoexpert-download

Downloads all questions and solutions using python from https://www.algoexpert.io
Apache License 2.0
24 stars 7 forks source link

Probably needed an update of QUESTION_STATEMENT_XPATH #5

Open UomoLepre opened 2 years ago

UomoLepre commented 2 years ago

While running Download PDF, it stucks with the first one.

Downloading pdf: 1 Palindrome Check

Traceback (most recent call last): File "C:\Users\capar\Desktop\Nuova cartella (2)\algoexpert-download-main\main.py", line 19, in main() File "C:\Users\capar\Desktop\Nuova cartella (2)\algoexpert-download-main\main.py", line 12, in main download_question_pdf() File "C:\Users\capar\Desktop\Nuova cartella (2)\algoexpert-download-main\downloaders\question_pdf\question_pdf_downloader.py", line 52, in main DRIVER.get(f'{QUESTION_URL_PREFIX}{question_name}') File "C:\pint\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 430, in get self.execute(Command.GET, {'url': url}) File "C:\pint\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 418, in execute self.error_handler.check_response(response) File "C:\pint\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 243, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.WebDriverException: Message: unknown error: unexpected command response (Session info: chrome=103.0.5060.114) Stacktrace: Backtrace: Ordinal0 [0x00C96463+2188387] Ordinal0 [0x00C2E461+1762401] Ordinal0 [0x00B43D78+802168] Ordinal0 [0x00B37210+750096] Ordinal0 [0x00B3675A+747354] Ordinal0 [0x00B35D3F+744767] Ordinal0 [0x00B3557C+742780] Ordinal0 [0x00B49BF3+826355] Ordinal0 [0x00B9CF6D+1167213] Ordinal0 [0x00B8C5F6+1099254] Ordinal0 [0x00B66BE0+945120] Ordinal0 [0x00B67AD6+948950] GetHandleVerifier [0x00F371F2+2712546] GetHandleVerifier [0x00F2886D+2652765] GetHandleVerifier [0x00D2002A+520730] GetHandleVerifier [0x00D1EE06+516086] Ordinal0 [0x00C3468B+1787531] Ordinal0 [0x00C38E88+1805960] Ordinal0 [0x00C38F75+1806197] Ordinal0 [0x00C41DF1+1842673] BaseThreadInitThunk [0x76D4FA29+25] RtlGetAppContainerNamedObjectPath [0x772C7A9E+286] RtlGetAppContainerNamedObjectPath [0x772C7A6E+238]

I've also got a different stack trace changing LOAD_QUESTION_MAX_WAIT_TIME = 10:

Traceback (most recent call last): File "C:\Users\capar\AppData\Local\JetBrains\PyCharm Community Edition 2021.1.1\plugins\python-ce\helpers\pydev\pydevd.py", line 1483, in _exec pydev_imports.execfile(file, globals, locals) # execute the script File "C:\Users\capar\AppData\Local\JetBrains\PyCharm Community Edition 2021.1.1\plugins\python-ce\helpers\pydev_pydev_imps_pydev_execfile.py", line 18, in execfile exec(compile(contents+"\n", file, 'exec'), glob, loc) File "C:/Users/capar/Desktop/Nuova cartella (2)/algoexpert-download-main/main.py", line 19, in main() File "C:/Users/capar/Desktop/Nuova cartella (2)/algoexpert-download-main/main.py", line 12, in main download_question_pdf() File "C:\Users\capar\Desktop\Nuova cartella (2)\algoexpert-download-main\downloaders\question_pdf\question_pdf_downloader.py", line 53, in main WebDriverWait(DRIVER, LOAD_QUESTION_MAX_WAIT_TIME).until(EC.visibility_of_element_located((By.XPATH, QUESTION_STATEMENT_XPATH))) File "C:\pint\lib\site-packages\selenium\webdriver\support\wait.py", line 89, in until raise TimeoutException(message, screen, stacktrace) selenium.common.exceptions.TimeoutException: Message: Stacktrace: Backtrace: Ordinal0 [0x00C96463+2188387] Ordinal0 [0x00C2E461+1762401] Ordinal0 [0x00B43D78+802168] Ordinal0 [0x00B71880+989312] Ordinal0 [0x00B71B1B+989979] Ordinal0 [0x00B9E912+1173778] Ordinal0 [0x00B8C824+1099812] Ordinal0 [0x00B9CC22+1166370] Ordinal0 [0x00B8C5F6+1099254] Ordinal0 [0x00B66BE0+945120] Ordinal0 [0x00B67AD6+948950] GetHandleVerifier [0x00F371F2+2712546] GetHandleVerifier [0x00F2886D+2652765] GetHandleVerifier [0x00D2002A+520730] GetHandleVerifier [0x00D1EE06+516086] Ordinal0 [0x00C3468B+1787531] Ordinal0 [0x00C38E88+1805960] Ordinal0 [0x00C38F75+1806197] Ordinal0 [0x00C41DF1+1842673] BaseThreadInitThunk [0x76D4FA29+25] RtlGetAppContainerNamedObjectPath [0x772C7A9E+286] RtlGetAppContainerNamedObjectPath [0x772C7A6E+238]

Iftakharpy commented 2 years ago

For the first issue, are you using the same version of the chrome browser and the chrome driver? If not try using the same version of those. If it still doesn't work check if the QUESTION_URL_PREFIX variable is correct? This variable value should end with /.

For the second one, LOAD_QUESTION_MAX_WAIT_TIME try to time the loading time of a question manually then add at least 8-15 seconds to that just to be safe this is necessary to let the browser load the question properly.

UomoLepre commented 2 years ago

I've found this recent issue here: https://stackoverflow.com/questions/72758996/selenium-seleniumwire-unknown-error-cannot-determine-loading-status-from-unkn

I've tried in headless mode and beta versione, but doesn't work. Any suggest?

Iftakharpy commented 2 years ago

My suggestion is to use the official google chrome browser and the same version of the chrome driver. For example, if the chrome browser version is 103.0.5060.134 then you should download the chrome driver with the same major version(103.0.5060.134).

Iftakharpy commented 2 years ago

@UomoLepre have you tried using the same version of chrome browser and chrome driver? Did that fix your issue?

Iftakharpy commented 2 years ago

@UomoLepre If you do not reply within this week I will assume you have resolved your issue and will close the issue.

UomoLepre commented 2 years ago

I've tried same version of chromedriver and browser (production release) but it's still not working

Iftakharpy commented 2 years ago

Can you try checking if they changed URL paths?

UomoLepre commented 2 years ago

The URL didn't changed, this is a screen from a free excercise XPATH: //*[@id="root"]/div[1]/div[4]/div[4]/div/div[1]/div/div[1]/div/div[2]/div[2] image

Iftakharpy commented 2 years ago

Yeah I see they are using uid instead of name now

Iftakharpy commented 2 years ago

I updated the question pdf downloader can you try running it now?

UomoLepre commented 2 years ago

Still not working, i've checked ulr and config, but still not working

Iftakharpy commented 2 years ago

Do you get a new error?

UomoLepre commented 2 years ago

Same error, i've tried changing XPATH but i get or the same or different errors

Iftakharpy commented 2 years ago

What is the value of LOAD_QUESTION_MAX_WAIT_TIME in your config file? Can you try with a value like 120?

UomoLepre commented 2 years ago

I've got this error almost immediately: Chromedriver 104.0.5112.79

selenium.common.exceptions.WebDriverException: Message: unknown error: cannot determine loading status from unknown error: unexpected command response (Session info: chrome=104.0.5112.81)