"pip install" installs browsermob-proxy 0.8.0, not 2.0 or 2.1.5

[tslever]

I'm trying to load webpages, wait until they've finished loading, scroll to their bottoms, and wait for until they've finished loading. Because the load times are indeterminate, I am using "browsermob-proxy.Client.wait_for_traffic_to_stop" to wait for traffic to stop before scrolling. I am using Python via Anaconda and Jupyter-Notebook. Given a quiet period of 0.1 seconds and a timeout of 20 seconds, my code consistently prints below my Jupyter-Notebook cell a duration of about 19.9 seconds, which is the difference between the wait time limit and the length of the quiet period, which I think is basically the amount of time wait_for_traffic_to_stop is allowed to wait.

That all being said, in Anaconda Prompt, I executed the command "pip install git+https://github.com/AutomatedTester/browsermob-proxy-py.git" to try to install browsermob-proxy 2.0. Despite browsermob-proxy-py's documentation seeming to imply that my command would install browsermob-proxy 2.0, my command installed browsermob-proxy 0.8.0. Given a post in the thread at "https://github.com/lightbody/browsermob-proxy/issues/212", it seems that the above time-out issue was fixed in browsermob-proxy 2. How do I install browsermob-proxy 2 or later via Anaconda Prompt?

[mctwynne]

@tslever I think you might be mixing up browsermob-proxy-py, the Python package, and browsermob-proxy, the proxy itself. If you're looking for version 2.0+ of browsermob-proxy you can find and download it here.

[jayden-xie]

The last release of Python package is 2 years ago, https://pypi.org/project/browsermob-proxy/#history @AutomatedTester When will you push the latest code to pip? Including fix for killing Java process? Thanks.

[tslever]

Hi Jayden.

Thanks for getting back to me.

I would like to "pip install" what I believe needs to be a browsermob-proxy Python-3 package via the most up-to-date Anaconda Prompt. Please see below Python-3 module which today outputted a "duration", equal to the allowed time to wait of 19.916 seconds, when waiting for a quiet period from Google.com.

# Find_Time_to_Load_Web_Page.py # # Created: 01/09/19 by Tom Lever # Updated: 04/16/19 by Tom Lever # # This program attempts to find the time it takes to load a web page. # Unfortunately, this program relies on browsermob-proxy which, # in waiting for data traffic associated with web-page loading to stop, # times out for, likely, "https://" web pages. # # Inputs: None # Dependencies: browsermobproxy.Server, selenium.webdriver, time # Outputs: At this time, to screen, the time-out value. # Enable creating browsermobproxy.Server class instances. from browsermobproxy import Server `# Enable creating selenium.webdriver class instances.` `from selenium import webdriver` # Enable creating time class instances. import time # Initialize the path to a proxy server batch file for creating a Server class instance. path_to_server_batch = "C:/Users/thoma/Python/Web_Scraping/browsermob-proxy-2.1.4/bin/browsermob-proxy.bat" `# Initialize the path to the chromedriver executable.` `path_to_chromedriver = "C:/Users/thoma/Python/Web_Scraping/chromedriver.exe"` # Create the server class instance. server = Server(path_to_server_batch) `# "Start" the server.` `server.start()` # Create a proxy. proxy = server.create_proxy() `# Have the proxy server load a Chrome browser.` `chrome_options = webdriver.ChromeOptions()` `chrome_options.add_argument("--proxy-server={0}".format(proxy.proxy))` `driver = webdriver.Chrome(executable_path=path_to_chromedriver, options=chrome_options)` # Have the proxy server have Chrome load a specific webpage. driver.get("https://google.com/") `# Record the present time.` `start_time = time.time()` # Wait for traffic to stop. status_code = proxy.wait_for_traffic_to_stop(100, 20000) `# Print the time between load beginning and traffic end, or wait time limit.` `duration = time.time() - start_time` `print(duration)` # Shut down use of the server. Erase the server class instance. server.stop() ` # Close the Google-loaded Chrome browser. driver.quit() # driver.close()

[jayden-xie]

Killing the Java process of proxy is quite a problem now, on both Windows and Linux, because we have tests running in parallel. If the Java process got ability to self-destruct after 5 minutes idle time, it would be nice! Thank you.