kiyoon / jupynium.nvim

Selenium-automated Jupyter Notebook that is synchronised with NeoVim in real-time.
MIT License
482 stars 15 forks source link

Specifying `firefox_profiles_ini_path` causes unexpected behaviour #130

Open Dzuchun opened 2 months ago

Dzuchun commented 2 months ago

Specifying firefox_profiles_ini_path as /home/dzu/.mozilla/firefox/profiles.ini (location of Firefox profiles for my user) causes server starting for a full minute, then suddenly allocating the entire system's RAM, before eventually exiting with 137.

During my testing, server managed to actually start and successfully open Firefox with proper profile a couple of times, but I'm not sure what that's correlated to.

To Reproduce Steps to reproduce the behavior:

  1. Specify firefox_profiles_ini_path option during jupynium plugin setup
  2. Open *.ju.py file
  3. Run :JupyniumStartAndAttachToServerInTerminal
  4. About a minute in, see exit code 137

Expected behavior For server to start instantly (as it usually does!), opening Firefox with my Default=1 profile.

Logs in /tmp/jupynium/logs/ from the terminal

There are three sets of logs produced in kinda-random order.

First one corresponds to jupynium asking for too much RAM and killed by the system:

jupynium.pynvim_helpers: 15 - INFO - nvim addr: /run/user/1000/nvim.2 2442.0 jupynium.pynvim_helpers: 34 - INFO - nvim attached jupynium.pynvim_helpers: 38 - INFO - Initialising.. jupynium.pynvim_helpers: 39 - INFO - Communicating with channel_id 9 jupynium.cmds.jupynium: 100 - INFO - Using firefox profile: /home/dzu/.mozilla/firefox/7l5t4nem.default-release-1721230317039 [Process exited 137]

Second produced while jupynium allocated all the RAM, but seemingly failed for different reason:

jupynium.pynvim_helpers: 15 - INFO - nvim addr: /run/user/1000/nvim.5436.0 jupynium.pynvim_helpers: 34 - INFO - nvim attached jupynium.pynvim_helpers: 38 - INFO - Initialising.. jupynium.pynvim_helpers: 39 - INFO - Communicating with channel_id 9 jupynium.cmds.jupynium: 100 - INFO - Using firefox profile: /home/dzu/.mozilla/firefox/7l5t4nem.default-release-1721230317039 jupynium.cmds.jupynium: 618 - ERROR - Exception occurred Traceback (most recent call last): File "/usr/lib/python3.12/site-packages/urllib3/connectionpool.py", line 715, in urlopen httplib_response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/site-packages/urllib3/connectionpool.py", line 467, in _make_request six.raise_from(e, None) File "", line 3, in raise_from File "/usr/lib/python3.12/site-packages/urllib3/connectionpool.py", line 462, in _make_request httplib_response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/dzu/.local/lib/python3.12/site-packages/jupynium/cmds/jupynium.py", line 521, in main with webdriver_firefox( ^^^^^^^^^^^^^^^^^^ File "/home/dzu/.local/lib/python3.12/site-packages/jupynium/cmds/jupynium.py", line 109, in webdriver_firefox return webdriver.Firefox(options=options, service=service) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/dzu/.local/lib/python3.12/site-packages/selenium/webdriver/firefox/webdriver.py", line 71, in __init__ super().__init__(command_executor=executor, options=options) File "/home/dzu/.local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 212, in __init__ self.start_session(capabilities) File "/home/dzu/.local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 299, in start_session response = self.execute(Command.NEW_SESSION, caps)["value"] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/dzu/.local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 352, in execute response = self.command_executor.execute(driver_command, params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/dzu/.local/lib/python3.12/site-packages/selenium/webdriver/remote/remote_connection.py", line 302, in execute return self._request(command_info[0], url, body=data) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/dzu/.local/lib/python3.12/site-packages/selenium/webdriver/remote/remote_connection.py", line 322, in _request response = self._conn.request(method, url, body=body, headers=headers) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/site-packages/urllib3/request.py", line 81, in request return self.request_encode_body( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/site-packages/urllib3/request.py", line 173, in request_encode_body return self.urlopen(method, url, **extra_kw) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/site-packages/urllib3/poolmanager.py", line 376, in urlopen response = conn.urlopen(method, u.request_uri, **kw) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/site-packages/urllib3/connectionpool.py", line 799, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/site-packages/urllib3/util/retry.py", line 550, in increment raise six.reraise(type(error), error, _stacktrace) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/site-packages/urllib3/packages/six.py", line 769, in reraise raise value.with_traceback(tb) File "/usr/lib/python3.12/site-packages/urllib3/connectionpool.py", line 715, in urlopen httplib_response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/site-packages/urllib3/connectionpool.py", line 467, in _make_request six.raise_from(e, None) File "", line 3, in raise_from File "/usr/lib/python3.12/site-packages/urllib3/connectionpool.py", line 462, in _make_request httplib_response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) [Process exited 0]

Third one would correspond to successfully launched Firefox, but they are very rare to occur and nvim cut them for some reason. These are basically the same as the first set, but with message about writing log to some random file in /tmp in place of exit code.

Output of jupynium --version

Jupynium v0.2.4

Output of nvim --version

NVIM v0.10.0
Build type: Release
LuaJIT 2.1.1720049189
Run "nvim -V1 -v" for more info

Additional context Arch Linux user here. Since pip won't allow installing any packages globally by default, I allowed it to do so with --break-system-packages flag for jupynium specifically (since it has no pacman-managed alternative). Rest of the dependencies were installed systemwide via pacman (the intended way).

Every time server is let to run with firefox_profiles_ini_path specified, system seems to "lose" RAM: less free RAM is reported by htop. Eventually, most of the RAM gets marked as shared, and I was not able to find process responsible for that. That persists between user sessions.

My firewall rules are relatively strict, so if there are any ports need to be open (apart from 8888 in the default url), it would be helpful to know.

Rest of plugin's functions seem to work just fine, and I really enjoy it!

kiyoon commented 2 months ago

Hi, does it happen with all kinds of profile, let's say even a clean one? Or is it that some of your plugins don't really like Jupynium?

Dzuchun commented 2 months ago

Now I feel sorry for not testing that.

Empty default profile indeed causes no problems. But what really surprised me - none of the plugins seem to be related!

It appears that the root cause was Firefox's offline translation dictionaries (sounds like a good feature, so I casually asked it to "Download all" and moved on). Apparently, the more are downloaded, the longer it would take jupynium-launched browser to start (manual launch is always instant).

"Remove all" seems to solve the problem, so you may close the issue.

Also I've observed behavior mentioned in the original profile.ini feature request (#17) - jupynium-launched Firefox always reports it's profile as some randomly-created one in the /tmp folder. I noticed that, just because of it always using light variant of the default theme. Using custom theme seem to override that.

kiyoon commented 2 months ago

Don't worry, it's good that you left the reference so it helps others when they face similar problems.

Maybe the translation stuff doesn't like selenium in general, or it could really be Jupynium's problem.

So should you or should you not use the custom theme? I don't really have crazy settings on my firefox so I'm not understanding it well..

Dzuchun commented 2 months ago

So should you or should you not use the custom theme?

Custom theme caused no problems.

I suggested using one, if you're not comfortable with the default light theme (as jupynium-launched Firefox always used it, even if it was configured to use default dark in the settings).

kiyoon commented 2 months ago

Okay, thanks for sharing this!