brotab doesn't work on ubuntu 22.04

[ofirgall]

Hey, I have upgrade my PC to Ubuntu 22.04 lately and I can't figure out why brotab isn't working.

bt and the mediator are communicating, I can see logs at /tmp/brotab_mediator.log but it seems that it can't grab info from my firefox. First I thought its something about firefox is installed by snap in 22.04 so I installed as a .deb instead but it didn't work. I have installed the extensions I have ran bt install and I ensured that the mediator json is actually there.

ofirg@ofirg:/tmp $ bt clients
ofirg@ofirg:/tmp $ bt windows
ofirg@ofirg:/tmp $ bt list

ofirg@ofirg:/tmp $

Firefox log

14:34:49.378 Detecting browser background.js:200:9
14:34:49.379 Connecting to native app background.js:208:11
14:34:49.382 It's Firefox: [object Object] background.js:211:13
14:34:49.382 Connected to native app brotab_mediator background.js:645:9

brotab_mediator.log

2022-07-03 14:04:58,125 19400 INFO     main.py:143:list_tabs Listing tabs
2022-07-03 14:04:58,126 19400 INFO     main.py:115:create_clients Created clients: []

Versions

ofirg@ofirg:/tmp $ firefox -V
Mozilla Firefox 102.0
ofirg@ofirg:/tmp $ python3 -V
Python 3.10.4

Mediator

ofirg@ofirg:/tmp $ cat /home/ofirg/.mozilla/native-messaging-hosts/brotab_mediator.json
{
  "name": "brotab_mediator",
  "description": "This mediator exposes interface over TCP to control browser's tabs",
  "path": "/home/ofirg/.local/bin/bt_mediator",
  "type": "stdio",
  "allowed_extensions": [ "brotab_mediator@example.org" ]
}

How can I troubleshoot this any further?

[balta2ar]

Looks like bt can't find open ports that mediator is listening to. It tries to scan the range from 4625 to 4635 and it seems it can't find any. Try checking which port mediator is actually listening to. e.g. in my setup:

$ sudo netstat -nplt | grep python | grep 46
tcp        0      0 127.0.0.1:4626          0.0.0.0:*               LISTEN      1586875/python      
tcp        0      0 127.0.0.1:4628          0.0.0.0:*               LISTEN      1589848/python
$ curl 'http://localhost:4626/get_browser'
firefox
$ curl 'http://localhost:4628/get_browser'
chrome/chromium

[ofirgall]

Thanks for the quick reply. You are right there are no sockets listening from python.

ofirg@ofirg:~/dotfiles (master) $ sudo netstat -nplt | grep python
ofirg@ofirg:~/dotfiles (master) $

There are no mediator logs, it seems that the firefox didn't run the mediator, when I run the mediator manually I see the startup logs.

This console.log(port) should sender be undefined

13:23:27.313
Object { name: "brotab_mediator", sender: undefined, error: null, onMessage: {…}, onDisconnect: {…}, postMessage: (), disconnect: () }

[rsyring]

Possibly due to Firefox being a snap by default in 22.04?

If so, probably related to #33.

[ofirgall]

As i said I installed firefox as .deb

[rsyring]

Ah, obviously missed that. Sorry. :)

[ofirgall]

Any news about that @balta2ar ?

[balta2ar]

Sorry, I must have missed your reply.

When you do bt install, it prints a bunch of paths, e.g.:

$ bt install
Installing mediator manifest /home/ybochkarev/.mozilla/native-messaging-hosts/brotab_mediator.json
Installing mediator manifest /home/ybochkarev/.config/chromium/NativeMessagingHosts/brotab_mediator.json
Installing mediator manifest /home/ybochkarev/.config/google-chrome/NativeMessagingHosts/brotab_mediator.json
Installing mediator manifest /home/ybochkarev/.config/BraveSoftware/Brave-Browser/NativeMessagingHosts/brotab_mediator.json

Could you check what's in the file that corresponds Firefox?

$ cat /home/ybochkarev/.mozilla/native-messaging-hosts/brotab_mediator.json
{
  "name": "brotab_mediator",
  "description": "This mediator exposes interface over TCP to control browser's tabs",
  "path": "/home/ybochkarev/.local/bin/bt_mediator",
  "type": "stdio",
  "allowed_extensions": [ "brotab_mediator@example.org" ]
}

and then try running bt_mediator exactly as you see its path in path field, e.g.

$ /home/ybochkarev/.local/bin/bt_mediator

That how Firefox would try to run it.

If that doesn't work, it could indicate that your env when running from a terminal and when running by Firefox are different, e.g. PATH could be different. In that that case we can dig further by examining the differences. You'll need to create a wrapper script, e.g. /home/ybochkarev/.local/bin/bt_mediator_wrapper with something like this:

#!/bin/bash

env > /tmp/wrapper.log
/home/ybochkarev/.local/bin/bt_mediator

And then we can compare the output of env when run from a terminal vs when run by Firefox.

[ofirgall]

Hey,

I have the same results as you:

ofirg@ofirg:~ $ cat /home/ofirg/.mozilla/native-messaging-hosts/brotab_mediator.json
{
  "name": "brotab_mediator",
  "description": "This mediator exposes interface over TCP to control browser's tabs",
  "path": "/home/ofirg/.local/bin/bt_mediator",
  "type": "stdio",
  "allowed_extensions": [ "brotab_mediator@example.org" ]
}

and ~/.local/bin/bt_mediator runs (no output to stdout/err)

[SpriteN]

Can't get brotab to work. I have tried to run ~/.local/bin/bt_mediator and get ImportError: cannot import name 'url_quote' from 'werkzeug.urls'

[rolflobker]

Can't get brotab to work. I have tried to run ~/.local/bin/bt_mediator and get ImportError: cannot import name 'url_quote' from 'werkzeug.urls'

You probaby have some dependency issue. I presume you install using pip instead of pipx ? If you run pip install flask -U you might see

brotab 1.4.2 requires Flask==2.0.2, but you have flask 3.0.2 which is incompatible.

brotab requires some outdated packages, so try to get the pipx install working (or create your own venv and install the required packages (with specific versions) in there.

Looks like among others you need:

• urllib3-1.25.11
• werkzeug-3.0.1
• Flask-2.0.2

(found in ~/.local/pipx/venvs/brotab/lib/python3.12/site-packages after installing using pipx method)

Edit... well, that doesn't work...

After installing using pipx , I ran into this

ModuleNotFoundError: No module named 'pkg_resources'

Injecting setuptools in the brotab venv using pipx inject brotab pkg_resources didn't help.

So I checked the PR's and found this: https://github.com/balta2ar/brotab/pull/109

Created local requirements.txt with contents:

And installed using

pipx runpip brotab install -r requirements.txt

Installing collected packages: Werkzeug
  Attempting uninstall: Werkzeug
    Found existing installation: Werkzeug 3.0.1
    Uninstalling Werkzeug-3.0.1:
      Successfully uninstalled Werkzeug-3.0.1
Successfully installed Werkzeug-2.3.8

Now bt_mediator runs without issues for me.

/tmp/brotab.log

2024-03-03 09:32:11,899 1697354 INFO     runner.py:31:here Started mediator process, pid=1697354
2024-03-03 09:32:11,900 1697354 INFO     http_server.py:35:serve Serving mediator on 127.0.0.1:4625

(close bt_mediator before using the addon in firefox, which launches its own bt_mediator process)

[AndydeCleyre]

Another Ubuntu 22.04 user reporting in.

I did need to inject setuptools in order to stop the pkg_resources error, but then none of the bt commands reported any output or errors. Running bt_mediator reported:

Traceback (most recent call last):
  File "/home/andy/.local/bin/bt_mediator", line 5, in <module>
    from brotab.mediator.brotab_mediator import main
  File "/home/andy/.local/share/venvs/3a0e2b54c68fb330bd06cd7d789cd280/venv/lib/python3.12/site-packages/brotab/mediator/brotab_mediator.py", line 15, in <module>
    from brotab.mediator.http_server import MediatorHttpServer
  File "/home/andy/.local/share/venvs/3a0e2b54c68fb330bd06cd7d789cd280/venv/lib/python3.12/site-packages/brotab/mediator/http_server.py", line 7, in <module>
    from flask import Flask
  File "/home/andy/.local/share/venvs/3a0e2b54c68fb330bd06cd7d789cd280/venv/lib/python3.12/site-packages/flask/__init__.py", line 7, in <module>
    from .app import Flask as Flask
  File "/home/andy/.local/share/venvs/3a0e2b54c68fb330bd06cd7d789cd280/venv/lib/python3.12/site-packages/flask/app.py", line 28, in <module>
    from . import cli
  File "/home/andy/.local/share/venvs/3a0e2b54c68fb330bd06cd7d789cd280/venv/lib/python3.12/site-packages/flask/cli.py", line 18, in <module>
    from .helpers import get_debug_flag
  File "/home/andy/.local/share/venvs/3a0e2b54c68fb330bd06cd7d789cd280/venv/lib/python3.12/site-packages/flask/helpers.py", line 16, in <module>
    from werkzeug.urls import url_quote
ImportError: cannot import name 'url_quote' from 'werkzeug.urls' (/home/andy/.local/share/venvs/3a0e2b54c68fb330bd06cd7d789cd280/venv/lib/python3.12/site-packages/werkzeug/urls.py). Did you mean: 'unquote'?

So I injected werkzeug<3 into brotab's venv as in that PR, and now it's working.