gingeleski / headbro

Headless browser rendering service for HTTP responses.
GNU General Public License v3.0
4 stars 0 forks source link

Experiment with BrowserMob implementation #18

Closed gingeleski closed 6 years ago

gingeleski commented 6 years ago

Trying to do this as a means to an end for #5 + #6

gingeleski commented 6 years ago

About to push... when I say "buggy" in the commit message, I mean the following is going on

(venv) PS C:\Users\gingeleski\workspace\headbro> python headbro.py
Traceback (most recent call last):
  File "headbro.py", line 51, in <module>
    browsermob_server.start()
  File "C:\Users\gingeleski\workspace\headbro\venv\lib\site-packages\browsermobproxy\server.py", line 122, in start
    raise ProxyServerError(message)
browsermobproxy.exceptions.ProxyServerError: The Browsermob-Proxy server process failed to start. Check <_io.TextIOWrapper name='C:\\Users\\gingelesk
i\\workspace\\headbro\\server.log' mode='w' encoding='cp1252'>for a helpful error message.
(venv) PS C:\Users\gingeleski\workspace\headbro>
gingeleski commented 6 years ago

After this bug is overcome, will need to refer to the following example (credit here) for Chrome hooks...

from browsermobproxy import Server
server = Server("path/to/browsermob-proxy")
server.start()
proxy = server.create_proxy()

from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--proxy-server={0}".format(proxy.proxy))
browser = webdriver.Chrome(chrome_options = chrome_options)

proxy.new_har("google")
driver.get("http://www.google.co.uk")
proxy.har # returns a HAR JSON blob

server.stop()
driver.quit()
gingeleski commented 6 years ago

One problem seemed to be that Java wasn't on the path.

P.S. error output from Browsermob sucks

gingeleski commented 6 years ago

And the whole folder inside the initial zip from Github's releases section must be preserved.

Otherwise saw -

Error: Could not find or load main class net.lightbody.bmp.proxy.Main
gingeleski commented 6 years ago

Apparently the whole time this was Browsermob being finnicky about path - like the path argument I was giving it from Python.

That's working now, but need to test further to see how well this is working with the proxy.