Cleanup and Login Button - 419 page expired

[Angel0ffDeath]

@neothematrix I saw you made a cleanup and added search for login button, so:

1. You removed sending Enter key. In this case you don't need this row as well - from selenium.webdriver.common.keys import Keys
2. Login button is inside section "content" which we already extracted so it will be more elegant if you use the following to find login button - elem.find_element(By.ID, "clogs-captcha-button").click(). You can find the button also with self.browser.find_element(By.ID, "clogs-captcha-button").click(), but this will search the whole document. ID's are unique. I really don't know why you prefer to write complex XPATH expressions instead simple one...

[neothematrix]

thanks @Angel0ffDeath ,good point on the not needed import and different find_element, I'll test them! The reason I'm reverting is because I kept having random failures with the "keys Enter" approach, often times after that step, I'd get a "409 page expired" error, while with this method it seems I don't have that issue. I haven't been able to properly understand the reason though, but reverting to the find_element click didn't have this issue. Have you ever encountered it?

[Angel0ffDeath]

@neothematrix The point when I made the implementation with sending Enter key was 1 element less to find. By me this Error 409 never happen. This is a public repository, which means the code should work properly on as much as possible more machines, so I don't mind you reverted back to find login button. I'm just saying there are much simple and faster ways to do that. Yeah I know the speed is not important in this case, but the question is principle :). And about writing complex XPATH expressions or simple one - it is up to you :)

[neothematrix]

I keep getting this random 419 page expired error after the login page, even after reverting to the previous login click button, I can't understand why :|

[Angel0ffDeath]

@neothematrix Unfortunately I can t reproduce this error by me, so I cannot give good advise. Usually 'Page expired' means session is expired, i.e. to long idle time. Check this row: browser.set_page_load_timeout(90) # Extended timeout for Raspberry Pi.

You can try to increase it. I cannot help you without some more feedback from the log file and images... Obviously it is login page (since you reverted to buton.click... You said you run the script in docker container. Does this occcure if you not run it via docker?

Edit: One more thing to concern - if you run too often the script from one and the same IP, it is possible the site to block you. There is a hidden CAPTCHA script - if you don't recognize the pictures... By me this never triggered, but probably if you do it often - it triggers. You can add additional save image rows after you send username, after you send pass and after you send click. Use debug level 2. Probably this will help to understand where is the problem

[Angel0ffDeath]

This is the script which runs after you press login button (or send Enter, it is the same)

This one is interesting: if (window.grecaptcha) { // check if captcha disabled or user is blocking google

Probably you should investigate it....

[neothematrix]

it turns out, I was double sending the login form because my encoded base64 password wrongly contained a "\n" :-(