seleniumbase / SeleniumBase

📊 Python's all-in-one framework for web crawling, scraping, testing, and reporting. Supports pytest. UC Mode provides stealth. Includes many tools.
https://seleniumbase.io
MIT License
5.34k stars 980 forks source link

Too many cases result in insufficient ports #695

Closed lmy690043305 closed 4 years ago

lmy690043305 commented 4 years ago

The reason is that selenium webdriver creates an HTTP connection at each step,Is there any solution to this problem

log: ----------------------------Captured stdout teardown---------------------------- WARNING: Unable to get screenshot for failure logs! Traceback (most recent call last): File "c:\python36\lib\site-packages\urllib3\connection.py", line 157, in _new_conn (self._dns_host, self.port), self.timeout, **extra_kw File "c:\python36\lib\site-packages\urllib3\util\connection.py", line 84, in create_connection raise err File "c:\python36\lib\site-packages\urllib3\util\connection.py", line 74, in create_connection sock.connect(sa) ConnectionRefusedError: [WinError 10061] 由于目标计算机积极拒绝,无法连接。

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "c:\python36\lib\site-packages\urllib3\connectionpool.py", line 672, in urlopen chunked=chunked, File "c:\python36\lib\site-packages\urllib3\connectionpool.py", line 387, in _make_request conn.request(method, url, **httplib_request_kw) File "c:\python36\lib\http\client.py", line 1239, in request self._send_request(method, url, body, headers, encode_chunked) File "c:\python36\lib\http\client.py", line 1285, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "c:\python36\lib\http\client.py", line 1234, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "c:\python36\lib\http\client.py", line 1026, in _send_output self.send(msg) File "c:\python36\lib\http\client.py", line 964, in send self.connect() File "c:\python36\lib\site-packages\urllib3\connection.py", line 184, in connect conn = self._new_conn() File "c:\python36\lib\site-packages\urllib3\connection.py", line 169, in _new_conn self, "Failed to establish a new connection: %s" % e urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x0000003620AC97F0>: Failed to establish a new connection: [WinError 10061] 由于目标计算机积极拒绝,无法连接。

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "c:\python36\lib\site-packages\seleniumbase\core\log_helper.py", line 15, in log_screenshot element = driver.find_element_by_tag_name('body') File "c:\python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 530, in find_element_by_tag_name return self.find_element(by=By.TAG_NAME, value=name) File "c:\python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 978, in find_element 'value': value})['value'] File "c:\python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 319, in execute response = self.command_executor.execute(driver_command, params) File "c:\python36\lib\site-packages\selenium\webdriver\remote\remote_connection.py", line 374, in execute return self._request(command_info[0], url, body=data) File "c:\python36\lib\site-packages\selenium\webdriver\remote\remote_connection.py", line 397, in _request resp = self._conn.request(method, url, body=body, headers=headers) File "c:\python36\lib\site-packages\urllib3\request.py", line 80, in request method, url, fields=fields, headers=headers, urlopen_kw File "c:\python36\lib\site-packages\urllib3\request.py", line 171, in request_encode_body return self.urlopen(method, url, extra_kw) File "c:\python36\lib\site-packages\urllib3\poolmanager.py", line 330, in urlopen response = conn.urlopen(method, u.request_uri, kw) File "c:\python36\lib\site-packages\urllib3\connectionpool.py", line 760, in urlopen response_kw File "c:\python36\lib\site-packages\urllib3\connectionpool.py", line 760, in urlopen response_kw File "c:\python36\lib\site-packages\urllib3\connectionpool.py", line 760, in urlopen response_kw File "c:\python36\lib\site-packages\urllib3\connectionpool.py", line 720, in urlopen method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] File "c:\python36\lib\site-packages\urllib3\util\retry.py", line 436, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='127.0.0.1', port=25357): Max retries exceeded with url: /session/89a4a58ff2fc50fc28acca5a4ce0a8ce/element (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x0000003620AC97F0>: Failed to establish a new connection: [WinError 10061] 由于目标计算机积极拒绝,无法连接。',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "c:\python36\lib\site-packages\urllib3\connection.py", line 157, in _new_conn (self._dns_host, self.port), self.timeout, **extra_kw File "c:\python36\lib\site-packages\urllib3\util\connection.py", line 84, in create_connection raise err File "c:\python36\lib\site-packages\urllib3\util\connection.py", line 74, in create_connection sock.connect(sa) ConnectionRefusedError: [WinError 10061] 由于目标计算机积极拒绝,无法连接。

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "c:\python36\lib\site-packages\urllib3\connectionpool.py", line 672, in urlopen chunked=chunked, File "c:\python36\lib\site-packages\urllib3\connectionpool.py", line 387, in _make_request conn.request(method, url, **httplib_request_kw) File "c:\python36\lib\http\client.py", line 1239, in request self._send_request(method, url, body, headers, encode_chunked) File "c:\python36\lib\http\client.py", line 1285, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "c:\python36\lib\http\client.py", line 1234, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "c:\python36\lib\http\client.py", line 1026, in _send_output self.send(msg) File "c:\python36\lib\http\client.py", line 964, in send self.connect() File "c:\python36\lib\site-packages\urllib3\connection.py", line 184, in connect conn = self._new_conn() File "c:\python36\lib\site-packages\urllib3\connection.py", line 169, in _new_conn self, "Failed to establish a new connection: %s" % e urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x0000003620AC94E0>: Failed to establish a new connection: [WinError 10061] 由于目标计算机积极拒绝,无法连接。

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "c:\python36\lib\site-packages\seleniumbase\core\log_helper.py", line 23, in log_screenshot driver.get_screenshot_as_file(screenshot_path) File "c:\python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 1032, in get_screenshot_as_file png = self.get_screenshot_as_png() File "c:\python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 1064, in get_screenshot_as_png return base64.b64decode(self.get_screenshot_as_base64().encode('ascii')) File "c:\python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 1074, in get_screenshot_as_base64 return self.execute(Command.SCREENSHOT)['value'] File "c:\python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 319, in execute response = self.command_executor.execute(driver_command, params) File "c:\python36\lib\site-packages\selenium\webdriver\remote\remote_connection.py", line 374, in execute return self._request(command_info[0], url, body=data) File "c:\python36\lib\site-packages\selenium\webdriver\remote\remote_connection.py", line 397, in _request resp = self._conn.request(method, url, body=body, headers=headers) File "c:\python36\lib\site-packages\urllib3\request.py", line 76, in request method, url, fields=fields, headers=headers, urlopen_kw File "c:\python36\lib\site-packages\urllib3\request.py", line 97, in request_encode_url return self.urlopen(method, url, extra_kw) File "c:\python36\lib\site-packages\urllib3\poolmanager.py", line 330, in urlopen response = conn.urlopen(method, u.request_uri, kw) File "c:\python36\lib\site-packages\urllib3\connectionpool.py", line 760, in urlopen response_kw File "c:\python36\lib\site-packages\urllib3\connectionpool.py", line 760, in urlopen response_kw File "c:\python36\lib\site-packages\urllib3\connectionpool.py", line 760, in urlopen response_kw File "c:\python36\lib\site-packages\urllib3\connectionpool.py", line 720, in urlopen method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] File "c:\python36\lib\site-packages\urllib3\util\retry.py", line 436, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='127.0.0.1', port=25357): Max retries exceeded with url: /session/89a4a58ff2fc50fc28acca5a4ce0a8ce/screenshot (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x0000003620AC94E0>: Failed to establish a new connection: [WinError 10061] 由于目标计算机积极拒绝,无法连接。',))

mdmintz commented 4 years ago

@lmy690043305 Which version of SeleniumBase are you using? Which browser were you using? If you are multi-threading tests, how many threads at once are you using? Are you running on a Selenium Grid? Based on your error logs, the browser window wasn't open when trying to take a screenshot. If you are manually closing browser windows, keep in mind that SeleniumBase will do this automatically at the end of each test so that it can get the screenshot for the logs. Also, you can tell your tests to reuse the browser session by adding --reuse-session to your pytest run command (which will have all tests use the same browser window). I'll need more information to determine what happened. It's also possible that your web driver is out-of-date. Use seleniumbase install chromedriver latest to get the latest chromedriver.

lmy690043305 commented 4 years ago

@lmy690043305 Which version of SeleniumBase are you using? Which browser were you using? If you are multi-threading tests, how many threads at once are you using? Are you running on a Selenium Grid? Based on your error logs, the browser window wasn't open when trying to take a screenshot. If you are manually closing browser windows, keep in mind that SeleniumBase will do this automatically at the end of each test so that it can get the screenshot for the logs. Also, you can tell your tests to reuse the browser session by adding --reuse-session to your pytest run command (which will have all tests use the same browser window). I'll need more information to determine what happened. It's also possible that your web driver is out-of-date. Use seleniumbase install chromedriver latest to get the latest chromedriver.

SeleniumBase 1.35.8,chrome browser,but now only one thread also has this problem. It is true that there was an error in the screenshot, but there was no error when the script was successful. Only if the script failed, an error was reported, the script has no manually close, updated chromedriver to lastest, error also occured

lmy690043305 commented 4 years ago

add “--reuse-session” error also occured

lmy690043305 commented 4 years ago

@lmy690043305 Which version of SeleniumBase are you using? Which browser were you using? If you are multi-threading tests, how many threads at once are you using? Are you running on a Selenium Grid? Based on your error logs, the browser window wasn't open when trying to take a screenshot. If you are manually closing browser windows, keep in mind that SeleniumBase will do this automatically at the end of each test so that it can get the screenshot for the logs. Also, you can tell your tests to reuse the browser session by adding --reuse-session to your pytest run command (which will have all tests use the same browser window). I'll need more information to determine what happened. It's also possible that your web driver is out-of-date. Use seleniumbase install chromedriver latest to get the latest chromedriver.

I found that after the script failed, the server port of the driver was released. However, I have two machines. One is normal and the other reports this problem. One is Windows Server 2012 R2 and the other is Windows Server 2008 R2. Other configurations are the same

mdmintz commented 4 years ago

Hi @lmy690043305 Possibly the framework won't work on something as old as "Windows Server 2008 R2", but I'm not sure. You can also try upgrading SeleniumBase to a newer version and see if that resolves your issue.

mdmintz commented 4 years ago

I'm going to close this because you mentioned that you're using a very old version of SeleniumBase. It is likely working on the latest version, as I can't reproduce. If you encounter this issue again with the latest version of SeleniumBase, feel free to reopen this.