It works just not in a docker container

[zachowj]

The current implementation works 99% of the time in the dev container or on a bare metal setup. But when built in a docker container it doesn't work at all. At this point, I've run out of ideas to find a solution.

Some observations:

• The browser loads the pages but when the username is submitted it returns to the login page asking for the username again. It would also do this for the password field at some point in the testing. My guess here is some javascript is not loading the correct value in the form being submitted, but I could be totally off base.
• I've tried messing with the shm_size and other things, but my knowledge of docker is limited.

[XerxesDGreat]

Hey, I've been tooling with another script filling a similar purpose which was having a similar issue; the user name would get put into the input, the submit button clicked, the page would submit, but then it would come back to the initial login page. While the approach is different (that script makes use of Selenium rather than using playwright), the root issue may be the same: Xfinity is blocking automated access to their login page. There are likely some things which can be done with the headers or something which can work around these blockers; indeed I've found a python selenium replacement for chromium which does exactly this and has made jantman's script functional again.

The python lib is called undetected_chromedriver in case you're interested in poking around at it to see what may be special about it. I know that Selenium adds some fingerprints which websites can pick up on, but maybe there are other things there which may be helpful.

Any rate, I'll poke around at the script as well, see if there's any learnings I got from the Selenium/python side of things which may be relevant. I don't have high hopes tbh, but I want this to work too :-)

[zachowj]

@XerxesDGreat I've been running the current script for over 3 weeks on a standalone VM with no docker and it completes the data retrieval 99% of the time. If I dockerize the same script with the same version of Chromium it fails after entering a valid username and password and then returning to the login screen.

So it's not the script or Chromium but something when you add them all to docker.

[XerxesDGreat]

Yeah, I got that; I'm just trying to eliminate variables, one of which is Xfinity being aggressive with their automation blocking

[TheRealFalseReality]

How can we run this standalone to work? Trying Docker version, not working yet. But really interested in this integration!

[zachowj]

@XerxesDGreat

Finally, found some time and motivation to look into undetected_chromedriver which then lead me to playwright-extra. Got it working with my setup yesterday and seems pretty reliable. Been running for a little over 24 hours on a 20-minute interval and has only missed one login.

Thanks.

[xiaobeilou2000]

thanks for the hard working. It finally worked in my docker.

[TheRealFalseReality]

Making progress but the sensor is unknown. Dunno if I got the username or password wrong but seems correct. It's a username and not an email, right?

[XerxesDGreat]

Awesome, thank you! You succeeded where I failed 😅