search

ylink-lfs / upwork-scraper-selenium

An upwork website scraper powered by selenium and bs4 library written in Python. IT WORKS!
GNU General Public License v3.0
4 stars 1 forks source link

FileNotFoundError: [Errno 2] No such file or directory: 'Xvfb' #2

Open liuxiaojiao opened 6 months ago

liuxiaojiao commented 6 months ago

Followed your instructions, got the correct version of chromedriver but got an error when processing python3 main.py --main_category 'Web, Mobile & Software Dev' --output out.json

Process Process-41: Traceback (most recent call last): File "/Users/XXX/Library/Python/3.8/lib/python/site-packages/multiprocess/process.py", line 315, in _bootstrap self.run() File "/Users/XXX/Library/Python/3.8/lib/python/site-packages/multiprocess/process.py", line 108, in run self._target(*self._args, self._kwargs) File "/Users/XXX/Documents/upwork-scraper-selenium-master/scraper/init.py", line 149, in _handle_job_list_page disp = Display(visible=0, size=(1920, 1080)) File "/Users/XXX/Library/Python/3.8/lib/python/site-packages/pyvirtualdisplay/display.py", line 54, in init self._obj = cls( File "/Users/XXX/Library/Python/3.8/lib/python/site-packages/pyvirtualdisplay/xvfb.py", line 44, in init AbstractDisplay.init( File "/Users/XXX/Library/Python/3.8/lib/python/site-packages/pyvirtualdisplay/abstractdisplay.py", line 85, in init helptext = get_helptext(program) File "/Users/XXX/Library/Python/3.8/lib/python/site-packages/pyvirtualdisplay/util.py", line 13, in get_helptext p = subprocess.Popen( File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/subprocess.py", line 854, in init self._execute_child(args, executable, preexec_fn, close_fds, File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/subprocess.py", line 1702, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) FileNotFoundError: [Errno 2] No such file or directory: 'Xvfb'**

ylink-lfs commented 6 months ago

try using linux_depedencies_all_in_one.sh in the repository to install necessary dependencies under Linux including Xvfb, or you can use headless mode so that no display-related dependency is required.