[Bug] WS problems with docker container & lack of WSS

[gymnae]

Please have a look at the F.A.Q. for solutions to common problems before posting an issue!

Describe the bug It appears that the container cannot map to the port of the websocket via python. Also, I use a reverse-proxy to the container via caddy2 / nginx. Those reverse proxies provide TLS certificates. Thus as websocket from the container right now is offered as insecure web socket. But this only becomes a problem once the websocket can be reached, which right now is not possible:

Both problems together, appear, to keep me from getting the web interface to display and info polled from the mumble server.

To Reproduce docker-compose run --use-aliases bot -ip 'mumble-server' -port '64738' -password '' -username '' -webip '<PUBLICIP/DOMAIN>' The Port 7001 is not in use at the time of starting the container. Adding '-e 7001' does not change this behavior.

Error message

[JJMumbleBot(4.3.3).WebInterface]:######### Initializing Web Interface #########
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/app/__main__.py", line 286, in <module>
    service.BotService(args.server_ip, int(args.server_port), args.server_password)
  File "/app/JJMumbleBot/core/bot_service.py", line 120, in __init__
    web_helper.initialize_web()
  File "/app/JJMumbleBot/web/web_helper.py", line 186, in initialize_web
    asyncio.get_event_loop().run_until_complete(ws)
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.9/asyncio/tasks.py", line 685, in _wrap_awaitable
    return (yield from awaitable.__await__())
  File "/usr/local/lib/python3.9/site-packages/websockets/server.py", line 965, in __await_impl__
    server = await self._create_server()
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1494, in create_server
    raise OSError(err.errno, 'error while attempting '
OSError: [Errno 99] error while attempting to bind on address ('<MYPUBLICIP>', 7001): cannot assign requested address

Expected behavior It might be a python problem, maybe using REST would be advantageous for the docker scenario.

Additional context Maybe the directive where the python app / websocket is listening to is somewhere setup as 127.0.0.1 or localhost, but should be changed to 0.0.0.0

[DuckBoss]

Hello!

I'm not sure what's causing the port binding error, as pulling the latest docker build and running it worked for me. I tried it on my end by pulling the latest docker build from docker hub, and running it like so:

> sudo docker pull jasonjerome/jjmumblebot:latest 
> sudo docker run --rm --name jjmumblebot --network host -it jasonjerome/jjmumblebot python /app -ip <server_ip> -port <server_port> -password <server_password> -username <bot_displayname> -superuser <superuser_displayname>

In my case I was using the default 7000/7001 Ports for the REST web page + socket connection.

It might be a python problem, maybe using REST would be advantageous for the docker scenario.

The web interface already uses REST to display information, and in addition it uses sockets to display live updated information such as any audio tracks that the bot may be playing.

Maybe the directive where the python app / websocket is listening to is somewhere setup as 127.0.0.1 or localhost, but should be changed to 0.0.0.0

You can change the web listening IP to 0.0.0.0 using the '-webip ' argument and see if that works. You could also try changing the default REST port and Sockets port using the -webpageport <port> and '-websocketport ' arguments to something other than 7000/7001 to see if there's still a port binding issue.

[gymnae]

Tried your command with a freshly pulled image.

I tried both with host and with a custom network. I would like to limit reachability to a sub-domain and avoid using host mode if at all possible.

sudo docker run --rm --name jjmumblebot --network host -p 7000:7000 -p 7001:7001 --name grogu -it jasonjerome/jjmumblebot python /app -ip <MUMBLESERVER> -port 64738 -password <SERVERPASS> -username grogu -superuser SuperUser -webip <MYDOMAIN>
WARNING: Published ports are discarded when using host network mode
[JJMumbleBot(4.3.4).StartUp]:######### Initializing JJMumbleBot #########
[JJMumbleBot(4.3.4).Database]:######### Initializing Internal Database #########
[JJMumbleBot(4.3.4).Database]:######### Initialized Internal Database #########
[JJMumbleBot(4.3.4).StartUp]:######### Initialized Temporary Directories #########
[JJMumbleBot(4.3.4).StartUp]:######### Initialized PGUI #########
[JJMumbleBot(4.3.4).General]:######### Initializing Core Plugins #########
[JJMumbleBot(4.3.4).General]:Auto-Updater v4.0.0 Plugin Initialized.
[JJMumbleBot(4.3.4).General]:Audio Commands v4.3.0 Plugin Initialized.
[JJMumbleBot(4.3.4).General]:Whisper v4.0.0 Plugin Initialized.
[JJMumbleBot(4.3.4).General]:Bot Commands v4.3.0 Plugin Initialized.
[JJMumbleBot(4.3.4).General]:Core Commands v4.0.0 Plugin Initialized.
[JJMumbleBot(4.3.4).General]:######### Core Plugins Initialized #########
[JJMumbleBot(4.3.4).General]:######### Initializing Extension Plugins #########
[JJMumbleBot(4.3.4).General]:Sound Board v4.3.0 Plugin Initialized.
[JJMumbleBot(4.3.4).General]:Randomizer v4.0.0 Plugin Initialized.
[JJMumbleBot(4.3.4).General]:Media v4.3.0 Plugin Initialized.
[JJMumbleBot(4.3.4).General]:Text To Speech v4.3.0 Plugin Initialized.
[JJMumbleBot(4.3.4).General]:Images v4.2.0 Plugin Initialized.
[JJMumbleBot(4.3.4).General]:Server Tools v4.3.0 Plugin Initialized.
[JJMumbleBot(4.3.4).General]:######### Extension Plugins Initialized #########
[JJMumbleBot(4.3.4).StartUp]:######### Initializing Mumble Client #########
[JJMumbleBot(4.3.4).StartUp]:######### Initialized Mumble Client #########
[JJMumbleBot(4.3.4).WebInterface]:######### Initializing Web Interface #########
[JJMumbleBot(4.3.4).WebInterface]:Initialized Socket Server.
[JJMumbleBot(4.3.4).WebInterface]:Initialized Flask Server.
[JJMumbleBot(4.3.4).WebInterface]:######### Initialized Web Interface #########
Error in connection handler
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/websockets/server.py", line 191, in handler
    await self.ws_handler(self, path)
  File "/app/JJMumbleBot/web/web_helper.py", line 34, in send_message
    packed_data = json.dumps(web_data)
  File "/usr/local/lib/python3.9/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/usr/local/lib/python3.9/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/local/lib/python3.9/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/usr/local/lib/python3.9/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type bytes is not JSON serializable
[2021-01-08 11:23:46,243] ERROR in app: Exception on /channels [GET]
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1953, in full_dispatch_request
    return self.finalize_request(rv)
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1968, in finalize_request
    response = self.make_response(rv)
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 2112, in make_response
    rv = jsonify(rv)
  File "/usr/local/lib/python3.9/site-packages/flask/json/__init__.py", line 370, in jsonify
    dumps(data, indent=indent, separators=separators) + "\n",
  File "/usr/local/lib/python3.9/site-packages/flask/json/__init__.py", line 211, in dumps
    rv = _json.dumps(obj, **kwargs)
  File "/usr/local/lib/python3.9/json/__init__.py", line 234, in dumps
    return cls(
  File "/usr/local/lib/python3.9/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/local/lib/python3.9/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/usr/local/lib/python3.9/site-packages/flask/json/__init__.py", line 100, in default
    return _json.JSONEncoder.default(self, o)
  File "/usr/local/lib/python3.9/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type bytes is not JSON serializable
Error in connection handler
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/websockets/server.py", line 191, in handler
    await self.ws_handler(self, path)
  File "/app/JJMumbleBot/web/web_helper.py", line 34, in send_message
    packed_data = json.dumps(web_data)
  File "/usr/local/lib/python3.9/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/usr/local/lib/python3.9/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/local/lib/python3.9/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/usr/local/lib/python3.9/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type bytes is not JSON serializable

I tried to assign different ports to the websocket, non worked. Above and below number 1000

sudo docker run --rm --name jjmumblebot --network host -p 7000:7000 -p 7001:7001 --name grogu -it jasonjerome/jjmumblebot python /app -ip <MUMBLESERVER> -port 64738 -password <SERVERPASS> -username grogu -superuser SuperUser -webip <MYEXTERNALIP>

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/app/__main__.py", line 286, in <module>
    service.BotService(args.server_ip, int(args.server_port), args.server_password)
  File "/app/JJMumbleBot/core/bot_service.py", line 120, in __init__
    web_helper.initialize_web()
  File "/app/JJMumbleBot/web/web_helper.py", line 186, in initialize_web
    asyncio.get_event_loop().run_until_complete(ws)
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.9/asyncio/tasks.py", line 687, in _wrap_awaitable
    return (yield from awaitable.__await__())
  File "/usr/local/lib/python3.9/site-packages/websockets/server.py", line 965, in __await_impl__
    server = await self._create_server()
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1494, in create_server
    raise OSError(err.errno, 'error while attempting '
OSError: [Errno 99] error while attempting to bind on address (<SERVERIP>', 7001): cannot assign requested address

[DuckBoss]

I have limited experience with Docker, and I'm not sure how to help you with this problem. I'll attach the "help-wanted" tag to this issue and I hope someone who is more experienced with your situation can help you. @kaminascripts helped me setup JJMumbleBot for Docker support, so hopefully if he's still active he might have an idea or two.

• Have you tried binding to a different host port such as: -p 9998:7000 and -p 9999:7001 instead of also using 7001 on the host side? I'm not sure if this would help you with your issue, but reading some documentation lead me to this.
• Alternatively, you could try building the docker container yourself from the source files in the repository. You will have access to the docker files to modify it as per your network needs.

As far as the web interface goes, I am working on remaking it since the current web interface lacks a lot of features and wasn't initially built to be used as anything more than a way to check on the server status and online user list.

I'll continue to look into this issue and see what I can do.

[gymnae]

I have limited experience with Docker, and I'm not sure how to help you with this problem. I'll attach the "help-wanted" tag to this issue and I hope someone who is more experienced with your situation can help you. @kaminascripts helped me setup JJMumbleBot for Docker support, so hopefully if he's still active he might have an idea or two.

I fiddle around a bit more myself, then I might approach him

• Have you tried binding to a different host port such as: -p 9998:7000 and -p 9999:7001 instead of also using 7001 on the host side? I'm not sure if this would help you with your issue, but reading some documentation lead me to this.
• Alternatively, you could try building the docker container yourself from the source files in the repository. You will have access to the docker files to modify it as per your network needs.

I tried different combinations, I think the problem really lies with python trying to bing a websocket to anyport, it didn't work. If i set the server IP to 0.0.0.0 then it does bind, but the browser also tries to connect to 0.0.0.0:7001 for the websocket, which can't work. If I give it 127.0.0.1 it also binds, but same problem for the browser. The public IP just won't work, even when run privileged mode.

As far as the web interface goes, I am working on remaking it since the current web interface lacks a lot of features and wasn't initially built to be used as anything more than a way to check on the server status and online user list.

I'll continue to look into this issue and see what I can do.

I really appreciate your drive for this project :)

This post might be related: https://stackoverflow.com/questions/54101508/how-do-you-dockerize-a-websocket-server You are also not exposing 7001 or any other port in your docker container, which means for those who do not want to run it as network_mode=host since they might be using a reverse proxy in front, it would be good to expose the ports required.

[DuckBoss]

This post might be related: https://stackoverflow.com/questions/54101508/how-do-you-dockerize-a-websocket-server You are also not exposing 7001 or any other port in your docker container, which means for those who do not want to run it as network_mode=host since they might be using a reverse proxy in front, it would be good to expose the ports required.

Thank you so much for bringing this to my attention. Since my development setup doesn't use anything other than the host network, I haven't run into any of these issues. I will keep this in mind as I continue to rework the web interface. I hope you can wait just a little bit longer while I work on this!

I am currently working on the web interface and have opted to use FastAPI which will provide both a REST endpoint + websocket on the same port. I have listed below some of the main features I hope to get out of this:

• One port access for both the interface + socket: For example localhost:7000/ provides the web interface, and localhost:7000/ws will provide the socket connection for the interface.
• Modern front-end visuals: It will also be visually updated as I will be using ReactJS for the front-end work.
• Transition to a plugin-based web server: I am making the web interface into it's own plugin so that it is more of a drag-and-drop optional feature to the bot. It will behave similarly to every other plugin in JJMumbleBot.
• Better API implementation: Utilizing the FastAPI library has also given me the opportunity to make a proper API system so that any custom third-party web interface implementation will be easy to use for those that do not wish to use the one I make. It also allows the API to be used for whatever integrations people may want with their bot into other software.
• Dark/Light Mode themes: It wasn't hard to add this feature, and some people prefer dark or light web page themes, so I have added a toggle that allows the user to use whatever he/she prefers.

[DuckBoss]

Could you try out the v5.0.0+ releases and let me know if it fixed your connectivity issue? I'm still working on WSS.

[DuckBoss]

Please refer to v.5.1.0 Release for updates to the web server security and web interface. This update includes many API updates and user/channel administration features.

I'm going to close this issue, if your problem persists or you have any new information please feel free to re-open this!

[gymnae]

I need to verify something. It appears that I worked with 4.3.3 all this time, even though I pulled from github and pulled the image from docker hub. I need to verify this. Until then my comment below might be in err.

_Hi there, at this time I am not sure if it's just my server or what is wrong, but the web interface is still not connecting via the websocket, so no interaction is possible.

Clicking on bot report I receive the following error: URL: http://domain.tld/command

Bad Request

The browser (or proxy) sent a request that this server could not understand.

The error console on every page shows me:

Uncaught DOMException: An invalid or illegal string was specified domain.tld:219
    <anonym> http://domain.tld/:219

Replace domain.tld with the real subdomain the bot lives on :) My reverse proxy is caddy2, but it also didn't work with nginx as revers proxy before.

Just for due diligence, this is my docker-compose:

version: '3.9'
services:
  bot:
    build: .
    privileged: true
    volumes:
     - ./JJMumbleBot/cfg:/app/JJMumbleBot/cfg
    environment:
      - name=grogu
      - network-alias=grogu
    container_name: grogu
    entrypoint: python /app
    expose:
      - "7000"
      - "7001"
    ports:
      - 7000:7000
      - 7001:7001
     # This parameters can be overrided in docker-compose run command
    command: -ip mumble-server -port 64738 -username grogu -password <mumblerserverpass> -superuser <secret>
    networks:
     default:
        aliases:
          - grogu

networks:
  default:
    external:
       name: <networkname>

and this is how I sping up the container:

/opt/bin/docker-compose run -d -p 7001:7001 -p 7000:7000 --use-aliases --name grogu bot -ip 'mumble-server' -port '64738' -password 'pommes' -username 'grogu' -superuser '<secret>' -webip ''

mumble-server translates to the container of the mumble-server, they reside on the same host and in the same network.

Also, if I may do so here: I'd like to request the bot to be able to be active in several channels at the same time, for example have it in the master channel and also sound in sub-channels. but that is a request for later :)_

[DuckBoss]

I need to verify something. It appears that I worked with 4.3.3 all this time, even though I pulled from github and pulled the image from docker hub. I need to verify this. Until then my comment below might be in err.

I hope that's the only issue and the new version works fine! I'll reopen this issue since this hasn't been solved yet.

Also, if I may do so here: I'd like to request the bot to be able to be active in several channels at the same time, for example have it in the master channel and also sound in sub-channels. but that is a request for later :)_

Yes I can add this feature using the whisper support that mumble has. Currently I think the bot can send audio to multiple users using whisper (even if they are in different channels), but not multiple channels specifically. I'll try to get this working for the next update.

[gymnae]

We can close this issue, the problem has been fixed since 5.x I am spinning up the container now via docker run instead of docker-compose, I was more used to that anyhow.