search

uhh-lt / bbb-live-subtitles

BBB plugin for automatic subtitles in conference calls
Apache License 2.0
26 stars 8 forks source link

Install instructions for BBB 2.4.x - everything works but mongodbconnector.py doesn't (for now) #4

Open motsiw opened 2 years ago

motsiw commented 2 years ago

I tried my luck to set this one up on a seperate server in combination with BBB 2.4. Here's what I found out up to now (lines that are missing in the orignal docs start with NEW):

Set up a BBB 2.4 server. Set up a second server (let's call it STTL for now) in the same subnet with Ubuntu 20.04.

=== On the STTL server ===

Enable root login via SSH: sudo passwd sudo nano /etc/ssh/sshd_config change: from => #PermitRootLogin prohibit-password to => PermitRootLogin yes sudo systemctl restart ssh Connect via PuTTY as root.

Update your STTL server

apt-get update apt-get upgrade apt-get dist-upgrade

Install Python 3.8

apt-get install python3.8 python3.8-dev portaudio19-dev

Now clone the bbb-live-subtitles package somewhere:

mkdir ~/projects cd ~/projects git clone https://github.com/uhh-lt/bbb-live-subtitles cd bbb-live-subtitles/

create the virtual environment and install the dependencies

NEW: apt install python3-virtualenv virtualenv -p /usr/bin/python3.8 bbbsub_env source bbbsub_env/bin/activate NEW: apt install build-essential pip install redis pymongo jaspion pyyaml pyaudio samplerate scipy pyetherpadlite websockets

Clone Kaldi-Model-Server

git clone https://github.com/uhh-lt/kaldi-model-server.git

Install PyKaldi

Download the PyKaldi wheel and install it

PyKaldi 0.2.1:

wget http://ltdata1.informatik.uni-hamburg.de/pykaldi_tmp/pykaldi-0.2.1-cp38-cp38-linux_x86_64.whl pip install pykaldi-0.2.1-cp38-cp38-linux_x86_64.whl

Install Kaldi and Intel MKL (see note below if you have a different CPU than Intel)

./install_mkl.sh ./install_kaldi_intel.sh ~/projects/bbb-live-subtitles/bbbsub_env/bin/python3.8

OR if you have a non-Intel CPU:

./install_kaldi.sh ~/projects/bbb-live-subtitles/bbbsub_env/bin/python3.8

Download the english and german model

NEW: cd kaldi-model-server ./download_example_models.sh NEW: cd ..

NEW: Install Redis-Tools

NEW: apt install redis-tools

==== Now connect to BBB-SERVER via PuTTY ====

Configure Redis to Accept Remote Connections

If you like to host the speech recognition on another server in your local network, you need to allow connections to redis in your local network. When all scripts run on the same machine you can skip this step

Open the redis configuration file in an editor nano /etc/redis/redis.conf

Comment the line bind 127.0.0.1 with a #:

bind 127.0.0.1

Save the file and restart the redis server

/etc/init.d/redis-server restart

Open firewall: ufw allow from

This would bind redis to any IP Adress. You can now test the access from another machine within your network with redis-cli for example:

redis-cli -h -p 6379 (from STTL machine)

Note that you shouldn't let Redis listen on a public IP, as you would otherwise expose raw speech data packages and other data to the public. If in doubt consult your admin about network and firewall settings and make sure that Redis can only be accessed from trusted hosts.

Install Audio-Fork

NEW: apt-get install libwebsockets-dev NEW: wget -q https://ltdata1.informatik.uni-hamburg.de/bbb-live-subtitles/audio_fork.zip NEW: unzip -q audio_fork.zip -d audio_fork NEW: cd audio_fork NEW: cp mod_audio_fork. /opt/freeswitch/lib/freeswitch/mod NEW: cp libwebsockets /usr/local/lib NEW: ldconfig NEW: cd ..

Open the modules file with your favorite editor: nano /opt/freeswitch/etc/freeswitch/autoload_configs/modules.conf.xml

Add the following at the end of the file but above :

NEW: Enable Freeswitch loopback

NEW: nano /opt/freeswitch/conf/autoload_configs/event_socket.conf.xml NEW: Uncomment this line:

NEW nano /opt/freeswitch/conf/autoload_configs/acl.conf.xml NEW: Add your local server subnet like this:

Save the changes and restart freeswitch: service freeswitch restart

NEW: Allow MongoDB access: NEW: nano /etc/mongod.conf NEW: Add / change line: bindIp: 127.0.0.1,

Find out Freeswitch password and Etherpad key: Freeswitch password: nano /opt/freeswitch/etc/freeswitch/autoload_configs/event_socket.conf.xml => note Mongo-DB: nano /usr/share/etherpad-lite/APIKEY.txt => note

==== Back to your STTL server ====

cd ~/projects/bbb-live-subtitles/ nano esl_to_redis.py

Change: server = '' app = Jaspion(host=server, port=8021, password='')

nano ws_receiver.py Replace localhost with BBB server name/ip: red = redis.Redis(host='', port=6379, password='')

Setup this way, I get esl_to_redis.py: runs fine ws_receiver.py: runs fine kaldi_starter.py: runs fine mongodbconnector.py: fails while including EtherPadLite.py

Dear developers, it would be a shame if this gets lost. Any hints / new instructions how to get it really working? TIA!

Alienmaster commented 2 years ago

Hi,

thanks for this awesome extension to the Readme! I will definitely add them to the existing Readme.

Getting this project tested and updated for BBB2.4 is at the top of my ToDo list! Its great to hear that you get most of the services run with 2.4 . When my VM is running my plan is to have a look on the problems next week.

motsiw commented 2 years ago

Thanks for looking into this. In the end, the BBB folks added a lot of security configs since v2.2.

Freeswitch now needs activation of loopback auto: nano /opt/freeswitch/conf/autoload_configs/event_socket.conf.xml Uncomment this line: <param name="apply-inbound-acl" value="loopback.auto"/> Then: nano /opt/freeswitch/conf/autoload_configs/acl.conf.xml Add your local subnet:

  <list name="loopback.auto" default="allow">
      <node type="allow" cidr="<yourSubnetBaseIP/24"/>
  </list>

On a BBB 2.4 server, the UFW firewall is active (and bbb-conf will activate it everytime BBB starts), so we have to care about this when both servers communicate, for example like this: ufw allow from <STTLserverIP>

MongoDB doesn't allow access from other IPs, we have allow our STTL server: nano /etc/mongod.conf Add /change line: bindIp: 127.0.0.1,<yourSTTLserverIP>

And there's some packets you forgot to install in the original docs, and some wrong paths. What's really missing in your PY-files is a reconnect mechanism (e. g. if the BBB server get's rebooted overnight), and a mechanism to run these py scripts in the background / as daemon. This way, your development would be production ready and could be performance tested.

best regards Motsi

bmilde commented 2 years ago

Hi Motsi!

Thanks for trying this out on 2.4. Just to clarify, is mongodbconnector.py still broken on 2.4 or did you manage to get it to work? What's the exact error message, is it just the import failing?

motsiw commented 2 years ago

mongodbconnector.py: fails while including EtherPadLite.py

Alienmaster commented 2 years ago

Can you install the newest version of pyetherpadlite and try it again please? There was an error in pyetherpadlite. https://pypi.org/project/pyetherpadlite/

If it does not work, please copy the error message.

My development environment is unfortunately broken so i can't test it here now.

tibroc commented 2 years ago

The mongodbconnector.py works for me on 2.4 and it also succeeds to write in the etherpad. But I can't get the connection in ws_reciever.py to work. I checked the freeswitch loopback and acl settings, but without success (it seems the socket_to_redis() function is never entered).

fishfree commented 2 years ago

I test against BBB 2.5. The same error as above with socket_to_redis():

2022-08-18 10:19:28 INFO {'Event': 'LOADER_START', 'Caller-Destination-Number': '91407', 'Caller-Orig-C│', 'callerName': 'Administrator', 'callState': 'CALL_STARTED'}}
aller-ID-Name': 'w_ipnfx4jao0us_2-bbbID-Administrator', 'Caller-Username': 'Administrator', 'Language':│2022-08-18 10:19:26 INFO {'header': {'name': 'VoiceCallStateEvtMsg', 'meetingId': '6a304786df17c7aa604f
 'German', 'Audio-Channel': '91407~w_ipnfx4jao0us_2-bbbID-Administrator~audio', 'Control-Channel': '914│f341293730427b213cc1-1660788778966', 'userId': '91407'}, 'body': {'meetingId': '6a304786df17c7aa604ff34
07~w_ipnfx4jao0us_2-bbbID-Administrator~control', 'Text-Channel': '91407~w_ipnfx4jao0us_2-bbbID-Adminis│1293730427b213cc1-1660788778966', 'voiceConf': '91407', 'clientSession': '2', 'userId': 'w_ipnfx4jao0us
trator~text'}                                                                                          │', 'callerName': 'Administrator', 'callState': 'CALL_STARTED'}}
2022-08-18 10:19:29 ERROR connection handler failed                                                    │2022-08-18 10:19:26 INFO {'header': {'name': 'VoiceCallStateEvtMsg', 'meetingId': '6a304786df17c7aa604f
Traceback (most recent call last):                                                                     │f341293730427b213cc1-1660788778966', 'userId': '91407'}, 'body': {'meetingId': '6a304786df17c7aa604ff34
  File "/home/ubuntu/projects/bbb-live-subtitles/bbbsub_env/lib/python3.8/site-packages/websockets/lega│1293730427b213cc1-1660788778966', 'voiceConf': '91407', 'clientSession': '2', 'userId': 'w_ipnfx4jao0us
cy/server.py", line 232, in handler                                                                    │', 'callerName': 'Administrator', 'callState': 'CALL_STARTED'}}
    await self.ws_handler(self)                                                                        │2022-08-18 10:19:27 INFO {'header': {'name': 'VoiceCallStateEvtMsg', 'meetingId': '6a304786df17c7aa604f
  File "/home/ubuntu/projects/bbb-live-subtitles/bbbsub_env/lib/python3.8/site-packages/websockets/lega│f341293730427b213cc1-1660788778966', 'userId': '91407'}, 'body': {'meetingId': '6a304786df17c7aa604ff34
cy/server.py", line 1146, in _ws_handler                                                               │1293730427b213cc1-1660788778966', 'voiceConf': '91407', 'clientSession': '2', 'userId': 'w_ipnfx4jao0us
    return await cast(                                                                                 │', 'callerName': 'Administrator', 'callState': 'IN_ECHO_TEST'}}
  File "ws_receiver.py", line 53, in socket_to_redis                                                   │2022-08-18 10:19:29 INFO {'header': {'name': 'VoiceCallStateEvtMsg', 'meetingId': '6a304786df17c7aa604f
    if wasTalking[voiceUserId] == 0:                                                                   │f341293730427b213cc1-1660788778966', 'userId': '91407'}, 'body': {'meetingId': '6a304786df17c7aa604ff34
KeyError: 'w_ipnfx4jao0us'                                                                             │1293730427b213cc1-1660788778966', 'voiceConf': '91407', 'clientSession': '2', 'userId': 'w_ipnfx4jao0us
2022-08-18 10:19:29 INFO connection closed

I install all stuff of BBB and this plugin in the same server, so the server name of this plugin are alll localhost.

jaywebmyne commented 1 year ago

I test against BBB 2.5. The same error as above with socket_to_redis():

2022-08-18 10:19:28 INFO {'Event': 'LOADER_START', 'Caller-Destination-Number': '91407', 'Caller-Orig-C│', 'callerName': 'Administrator', 'callState': 'CALL_STARTED'}}
aller-ID-Name': 'w_ipnfx4jao0us_2-bbbID-Administrator', 'Caller-Username': 'Administrator', 'Language':│2022-08-18 10:19:26 INFO {'header': {'name': 'VoiceCallStateEvtMsg', 'meetingId': '6a304786df17c7aa604f
 'German', 'Audio-Channel': '91407~w_ipnfx4jao0us_2-bbbID-Administrator~audio', 'Control-Channel': '914│f341293730427b213cc1-1660788778966', 'userId': '91407'}, 'body': {'meetingId': '6a304786df17c7aa604ff34
07~w_ipnfx4jao0us_2-bbbID-Administrator~control', 'Text-Channel': '91407~w_ipnfx4jao0us_2-bbbID-Adminis│1293730427b213cc1-1660788778966', 'voiceConf': '91407', 'clientSession': '2', 'userId': 'w_ipnfx4jao0us
trator~text'}                                                                                          │', 'callerName': 'Administrator', 'callState': 'CALL_STARTED'}}
2022-08-18 10:19:29 ERROR connection handler failed                                                    │2022-08-18 10:19:26 INFO {'header': {'name': 'VoiceCallStateEvtMsg', 'meetingId': '6a304786df17c7aa604f
Traceback (most recent call last):                                                                     │f341293730427b213cc1-1660788778966', 'userId': '91407'}, 'body': {'meetingId': '6a304786df17c7aa604ff34
  File "/home/ubuntu/projects/bbb-live-subtitles/bbbsub_env/lib/python3.8/site-packages/websockets/lega│1293730427b213cc1-1660788778966', 'voiceConf': '91407', 'clientSession': '2', 'userId': 'w_ipnfx4jao0us
cy/server.py", line 232, in handler                                                                    │', 'callerName': 'Administrator', 'callState': 'CALL_STARTED'}}
    await self.ws_handler(self)                                                                        │2022-08-18 10:19:27 INFO {'header': {'name': 'VoiceCallStateEvtMsg', 'meetingId': '6a304786df17c7aa604f
  File "/home/ubuntu/projects/bbb-live-subtitles/bbbsub_env/lib/python3.8/site-packages/websockets/lega│f341293730427b213cc1-1660788778966', 'userId': '91407'}, 'body': {'meetingId': '6a304786df17c7aa604ff34
cy/server.py", line 1146, in _ws_handler                                                               │1293730427b213cc1-1660788778966', 'voiceConf': '91407', 'clientSession': '2', 'userId': 'w_ipnfx4jao0us
    return await cast(                                                                                 │', 'callerName': 'Administrator', 'callState': 'IN_ECHO_TEST'}}
  File "ws_receiver.py", line 53, in socket_to_redis                                                   │2022-08-18 10:19:29 INFO {'header': {'name': 'VoiceCallStateEvtMsg', 'meetingId': '6a304786df17c7aa604f
    if wasTalking[voiceUserId] == 0:                                                                   │f341293730427b213cc1-1660788778966', 'userId': '91407'}, 'body': {'meetingId': '6a304786df17c7aa604ff34
KeyError: 'w_ipnfx4jao0us'                                                                             │1293730427b213cc1-1660788778966', 'voiceConf': '91407', 'clientSession': '2', 'userId': 'w_ipnfx4jao0us
2022-08-18 10:19:29 INFO connection closed

I install all stuff of BBB and this plugin in the same server, so the server name of this plugin are alll localhost.

any luck ? i am stuck with this error from long time.

rasos commented 1 year ago

PoC has been presented at the last day of the 16th BigBlueButton Developer summit (https://conference.bigbluebutton.org/) at Humboldt-Universität, Berlin. Text is shown in an extra window and also sent to the Etherpad. Is there a new fork / branch?