seleniumbase / SeleniumBase

📊 Python's all-in-one framework for web crawling, scraping, testing, and reporting. Supports pytest. UC Mode provides stealth. Includes many tools.
https://seleniumbase.io
MIT License
4.46k stars 910 forks source link

Freezes on ChromeOS when loading Driver #2697

Closed twv123 closed 2 months ago

twv123 commented 2 months ago

Saw similar issue 323, but the fix did not work. Understand it is not supported on ChromeOS, but would really like to get working as it seems(wishful thinking) like should be minor tweaks of Linux --

Sample never gets to 'Driver loaded.':

Code:

from seleniumbase import Driver

print('starting...')

driver = Driver(uc=True)
print('Driver loaded.')

Path:

(venv) twv123@penguin:~/my_code_projects/python/webscrape$ echo $PATH
/home/twv123/my_code_projects/python/webscrape/venv/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games

And related:

(venv) twv123@penguin:~/my_code_projects/python/webscrape$ which chromium
/usr/bin/chromium
(venv) twv123@penguin:~/my_code_projects/python/webscrape$ which google-chrome
/usr/bin/google-chrome
(venv) twv123@penguin:~/my_code_projects/python/webscrape/venv/bin$ ls -l
total 15876
-rw-r--r-- 1 twv123 twv123     2018 Apr 12 17:07 activate
-rw-r--r-- 1 twv123 twv123      944 Apr 12 17:07 activate.csh
-rw-r--r-- 1 twv123 twv123     2224 Apr 12 17:07 activate.fish
-rw-r--r-- 1 twv123 twv123     9033 Apr 12 17:07 Activate.ps1
-rwxr-xr-x 1 twv123 twv123      261 Apr 12 17:08 behave
-rwxr-xr-x 1 twv123 twv123      268 Apr 12 17:08 chardetect
-rwxr-xr-x 1 twv123 twv123 16157160 Apr 15 09:32 chromedriver
-rwxr-xr-x 1 twv123 twv123      267 Apr 12 17:08 markdown-it
-rwxr-xr-x 1 twv123 twv123      280 Apr 12 17:08 normalizer
-rwxr-xr-x 1 twv123 twv123      263 Apr 12 17:08 nosetests
-rwxr-xr-x 1 twv123 twv123      268 Apr 12 17:08 pip
-rwxr-xr-x 1 twv123 twv123      268 Apr 12 17:08 pip3
-rwxr-xr-x 1 twv123 twv123      268 Apr 12 17:08 pip3.11
-rwxr-xr-x 1 twv123 twv123      262 Apr 12 17:08 pygmentize
-rwxr-xr-x 1 twv123 twv123      263 Apr 12 17:08 pynose
-rwxr-xr-x 1 twv123 twv123      268 Apr 12 17:08 py.test
-rwxr-xr-x 1 twv123 twv123      268 Apr 12 17:08 pytest
lrwxrwxrwx 1 twv123 twv123        7 Apr 12 17:07 python -> python3
lrwxrwxrwx 1 twv123 twv123       16 Apr 12 17:07 python3 -> /usr/bin/python3
lrwxrwxrwx 1 twv123 twv123        7 Apr 12 17:07 python3.11 -> python3
-rwxr-xr-x 1 twv123 twv123      278 Apr 12 17:08 sbase
-rwxr-xr-x 1 twv123 twv123      278 Apr 12 17:08 seleniumbase
-rwxr-xr-x 1 twv123 twv123      255 Apr 12 17:08 wheel
(venv) twv123@penguin:~/my_code_projects/python/webscrape/venv/bin$ 

chromedriver appears to be in the $PATH as well....?

Any thoughts appreciated - would love to use SeleniumBase to simplify things.

Thanks.

mdmintz commented 2 months ago

Similar to https://github.com/seleniumbase/SeleniumBase/issues/323#issuecomment-488065275 ? That user was running in a special crostini shell, and so chromium had to be installed separately.

If that's not your issue, you really need to provide a stack trace so that this is actually debuggable. Also, I would need to know if regular mode works for you without UC Mode. It's highly unlikely that UC Mode will work on ChromeOS unless it's already supported.

twv123 commented 2 months ago

Thank you for quick response. Yes, works in Driver(headed=True), not with uc=True. It checks if I am human automatically, then fails and gives checkbox, which still fails after manually checking box... meh.

Not sure if this trace helps, but sounds like I may need to try another solution to get around the Cloudflare "Veryfing you are human" - your solution was the most elegant....

(venv) twv123@penguin:~/my_code_projects/python/webscrape$ gdb -ex r -ex bt -ex q --args python test.py
GNU gdb (Debian 13.1-3) 13.1
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from python...
(No debugging symbols found in python)
Starting program: /home/twv123/my_code_projects/python/webscrape/venv/bin/python test.py
warning: opening /proc/self/mem file failed: Permission denied (13)
warning: opening /proc/PID/mem file for lwp 28024.28024 failed: Permission denied (13)
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Starting...
warning: opening /proc/PID/mem file for lwp 28027.28027 failed: Permission denied (13)
[Detaching after vfork from child process 28027]
warning: opening /proc/PID/mem file for lwp 28042.28042 failed: Permission denied (13)
[Detaching after vfork from child process 28042]
warning: opening /proc/PID/mem file for lwp 28044.28044 failed: Permission denied (13)
[Detaching after vfork from child process 28044]
^Z
Program received signal SIGTSTP, Stopped (user).
__GI___clock_nanosleep (clock_id=1, flags=1, req=0x7fffffffd260, rem=0x0) at ../sysdeps/unix/sysv/linux/clock_nanosleep.c:71
71      ../sysdeps/unix/sysv/linux/clock_nanosleep.c: No such file or directory.
#0  __GI___clock_nanosleep (clock_id=1, flags=1, req=0x7fffffffd260, rem=0x0) at ../sysdeps/unix/sysv/linux/clock_nanosleep.c:71
#1  0x00000000005d6af4 in ?? ()
#2  0x000000000052c177 in _PyEval_EvalFrameDefault ()
#3  0x000000000051f067 in _PyObject_FastCallDictTstate ()
#4  0x000000000056622d in _PyObject_Call_Prepend ()
#5  0x00000000005e4a72 in ?? ()
#6  0x0000000000517eb3 in _PyObject_MakeTpCall ()
#7  0x000000000052b940 in _PyEval_EvalFrameDefault ()
#8  0x00000000005855a4 in ?? ()
#9  0x000000000052fd39 in _PyEval_EvalFrameDefault ()
#10 0x000000000055c931 in _PyFunction_Vectorcall ()
#11 0x000000000052f8a2 in _PyEval_EvalFrameDefault ()
#12 0x000000000055c931 in _PyFunction_Vectorcall ()
#13 0x000000000056a381 in PyObject_Call ()
#14 0x000000000052f8a2 in _PyEval_EvalFrameDefault ()
#15 0x000000000052360b in PyEval_EvalCode ()
#16 0x0000000000647497 in ?? ()
#17 0x0000000000644d4f in ?? ()
#18 0x0000000000651010 in ?? ()
#19 0x0000000000650d5b in _PyRun_SimpleFileObject ()
#20 0x0000000000650b84 in _PyRun_AnyFileObject ()
#21 0x000000000064f90f in Py_RunMain ()
#22 0x00000000006275c7 in Py_BytesMain ()
#23 0x00007ffff7cd724a in __libc_start_call_main (main=main@entry=0x627530, argc=argc@entry=2, argv=argv@entry=0x7fffffffe048) at ../sysdeps/nptl/libc_start_call_main.h:58
#24 0x00007ffff7cd7305 in __libc_start_main_impl (main=0x627530, argc=2, argv=0x7fffffffe048, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe038) at ../csu/libc-start.c:360
#25 0x0000000000627461 in _start ()
A debugging session is active.

        Inferior 1 [process 28024] will be killed.

Quit anyway? (y or n) y
(venv) twv123@penguin:~/my_code_projects/python/webscrape$