Forked project with the standard Mod software updated, mod plugins, file manager and a docker prebuilt image

[raidolo]

Hi Alexis,

I've forked this project and I made some improvements to run the latest software from MOD, all their plugins using the plugin builder and I've also put the File Manager.

I've also built a precompiled Docker image.

The forked project is here: https://github.com/raidolo/mod-docker

The prebuilt image is here: https://hub.docker.com/r/raidolo/mod-ui

Thanks for your initial work!

Hope you like this new version.

Cheers, Alessio.

[ajboni]

Hello Alessio, great job!!!
I'm on a new system (arch with pipewire) trying the fork (thanks!!), so far without luck, something related to the user/usergroup.

mod-ui  | useradd: UID 1000 is not unique
mod-ui  | error: failed switching to "baj": unable to find user baj: no matching entries in passwd file
mod-ui exited with code 1

cat /etc/passwd | grep baj               
baj:x:1000:1000:baj:/home/baj:/bin/fish

[raidolo]

Hello Alessio, great job!!!

I'm on a new system (arch with pipewire) trying the fork (thanks!!), so far without luck, something related to the user/usergroup.

mod-ui  | useradd: UID 1000 is not unique

mod-ui  | error: failed switching to "baj": unable to find user baj: no matching entries in passwd file

mod-ui exited with code 1

cat /etc/passwd | grep baj               

baj:x:1000:1000:baj:/home/baj:/bin/fish

Hey thanks, are you building or running the container?

[ajboni]

Running from dockerhub, the run_docker script

[raidolo]

Running from dockerhub, the run_docker script

Well I think I know what's going on, to share the prebuilt container I can't put the useradd during the build in the CMD call.

Basically in the image there is already a user id 1000 in the passwd of the docker image... my user ;)

It should be done at runtime passing the user in the docker env variables.

If you want to test it I believe you should add another local user and start the docker container from him...

I'll fix it soon. Sorry

[raidolo]

Running from dockerhub, the run_docker script

I've fixed the prebuilt image, now you can use your own user, chekout the git repo again and pull the new image, I've changed the run script also... I've tested it on another user on my system and it works fine.

Let me know!

[ajboni]

Running from dockerhub, the run_docker script

I've fixed the prebuilt image, now you can use your own user, chekout the git repo again and pull the new image, I've changed the run script also... I've tested it on another user on my system and it works fine.

Let me know!

Thanks! Unfortunately I'm getting a similar error:

Using default tag: latest
latest: Pulling from raidolo/mod-ui
Digest: sha256:8332a3a738591b1aa20e29c3c2a2176dae6212617d6dbc01bb6ea6fac32ea271
Status: Image is up to date for raidolo/mod-ui:latest
docker.io/raidolo/mod-ui:latest
[+] Running 1/0
 ✔ Container mod-ui  Created                                                                                                                                                               0.0s 
Attaching to mod-ui
mod-ui  | PUID
mod-ui  | 
mod-ui  | 
mod-ui  | useradd: user 'abc' already exists
mod-ui  | usermod: invalid user ID 'abc'
mod-ui exited with code 3

[raidolo]

Running from dockerhub, the run_docker script

I've fixed the prebuilt image, now you can use your own user, chekout the git repo again and pull the new image, I've changed the run script also... I've tested it on another user on my system and it works fine. Let me know!

Thanks! Unfortunately I'm getting a similar error:

Using default tag: latest
latest: Pulling from raidolo/mod-ui
Digest: sha256:8332a3a738591b1aa20e29c3c2a2176dae6212617d6dbc01bb6ea6fac32ea271
Status: Image is up to date for raidolo/mod-ui:latest
docker.io/raidolo/mod-ui:latest
[+] Running 1/0
 ✔ Container mod-ui  Created                                                                                                                                                               0.0s 
Attaching to mod-ui
mod-ui  | PUID
mod-ui  | 
mod-ui  | 
mod-ui  | useradd: user 'abc' already exists
mod-ui  | usermod: invalid user ID 'abc'
mod-ui exited with code 3

Did you change the docker-compose file? You need to clone again the repo probably... As you can see the "echo" of the PUID is empty... so probably you have old files, I've added/changed two vars inside the run_docker.sh and the docker-compose file.

[ajboni]

Forgot to pull (sorry 🤦🏼)!! I can start the container now, for a bit at least. There are some warnings though about ttl missing. Also I get an error about jack not found, and it's probably due to pipewire. I will try to do some troubleshooting, I don't know if it will work at all...

Using default tag: latest
latest: Pulling from raidolo/mod-ui
Digest: sha256:8332a3a738591b1aa20e29c3c2a2176dae6212617d6dbc01bb6ea6fac32ea271
Status: Image is up to date for raidolo/mod-ui:latest
docker.io/raidolo/mod-ui:latest
[+] Running 1/0
 ✔ Container mod-ui  Created                                                                                                                                                               0.0s 
Attaching to mod-ui
mod-ui  | PUID 1000
mod-ui  | 1000
mod-ui  | useradd: user 'abc' already exists
mod-ui  | usermod: no changes
mod-ui  | Forking... child PID: 22
mod-ui  | Cannot connect to server socket err = No such file or directory
mod-ui  | Cannot connect to server request channel
mod-ui  | jack server is not running or cannot be started
mod-ui  | JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
mod-ui  | JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
mod-ui  |  * Serving Flask app 'browsepy'
mod-ui  |  * Debug mode: off
mod-ui  | WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
mod-ui  |  * Running on all addresses (0.0.0.0)
mod-ui  |  * Running on http://127.0.0.1:8081
mod-ui  |  * Running on http://172.20.0.2:8081
mod-ui  | Press CTRL+C to quit
mod-ui  | lilv_world_load_bundle(): warning: Replacing version 0.0 of <http://code.google.com/p/amsynth/amsynth> from <file:///usr/lib/x86_64-linux-gnu/lv2/amsynth.lv2/>
mod-ui  | lilv_world_load_bundle(): note: New version 1.5 found in <file:///usr/lib/lv2/amsynth.lv2/>
mod-ui  | lilv_world_load_bundle(): warning: Ignoring bundle <file:///usr/lib/lv2/Vex.lv2/>
mod-ui  | lilv_world_load_bundle(): note: Newer version of <urn:juce:Vex> loaded from <file:///usr/lib/lv2/vex.lv2/>
mod-ui  | error: /usr/lib/lv2/Fluida.lv2/Fluida.ttl:48:4: missing ';' or '.'
mod-ui  | lilv_world_load_file(): error: Error loading file `file:///usr/lib/lv2/Fluida.lv2/Fluida.ttl'
mod-ui  | ALSA lib confmisc.c:165:(snd_config_get_card) Cannot get card index for MODDUO
mod-ui  | Cannot connect to server socket err = No such file or directory
mod-ui  | Cannot connect to server request channel
mod-ui  | jack server is not running or cannot be started
mod-ui  | JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
mod-ui  | JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
mod-ui  | WARNING:tornado.general:Connect error on fd 11: ECONNREFUSED
mod-ui  | WARNING:tornado.general:Connect error on fd 10: ECONNREFUSED
mod-ui  | WARN: Missing file /mod/data/mod_api_key.pub with the public API KEY
mod-ui  | ERROR: Failed to open HMI serial port, error was:
mod-ui  | SerialBase.__init__() got an unexpected keyword argument 'writeTimeout'
mod-ui  | Using FakeHMI => <mod.development.FakeHMI object at 0x7feef2833cd0>
mod-ui  | cc start socket missing
mod-ui  | last state file does not exist or is corrupt
mod-ui  | last state file does not exist or is corrupt
mod-ui  | HMI initialized right away
mod-ui  | Readsock none
mod-ui  | Writesock none
mod-ui  | Host failed to initialize, is the backend running?
mod-ui  | jack client deactivated NOT
mod-ui  | jack client deactivated NOT
mod-ui exited with code 1

[raidolo]

Forgot to pull (sorry 🤦🏼)!!

I can start the container now, for a bit at least. There are some warnings though about ttl missing.

Also I get an error about jack not found, and it's probably due to pipewire. I will try to do some troubleshooting, I don't know if it will work at all...

Using default tag: latest

latest: Pulling from raidolo/mod-ui

Digest: sha256:8332a3a738591b1aa20e29c3c2a2176dae6212617d6dbc01bb6ea6fac32ea271

Status: Image is up to date for raidolo/mod-ui:latest

docker.io/raidolo/mod-ui:latest

[+] Running 1/0

 ✔ Container mod-ui  Created                                                                                                                                                               0.0s 

Attaching to mod-ui

mod-ui  | PUID 1000

mod-ui  | 1000

mod-ui  | useradd: user 'abc' already exists

mod-ui  | usermod: no changes

mod-ui  | Forking... child PID: 22

mod-ui  | Cannot connect to server socket err = No such file or directory

mod-ui  | Cannot connect to server request channel

mod-ui  | jack server is not running or cannot be started

mod-ui  | JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock

mod-ui  | JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock

mod-ui  |  * Serving Flask app 'browsepy'

mod-ui  |  * Debug mode: off

mod-ui  | WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.

mod-ui  |  * Running on all addresses (0.0.0.0)

mod-ui  |  * Running on http://127.0.0.1:8081

mod-ui  |  * Running on http://172.20.0.2:8081

mod-ui  | Press CTRL+C to quit

mod-ui  | lilv_world_load_bundle(): warning: Replacing version 0.0 of <http://code.google.com/p/amsynth/amsynth> from <file:///usr/lib/x86_64-linux-gnu/lv2/amsynth.lv2/>

mod-ui  | lilv_world_load_bundle(): note: New version 1.5 found in <file:///usr/lib/lv2/amsynth.lv2/>

mod-ui  | lilv_world_load_bundle(): warning: Ignoring bundle <file:///usr/lib/lv2/Vex.lv2/>

mod-ui  | lilv_world_load_bundle(): note: Newer version of <urn:juce:Vex> loaded from <file:///usr/lib/lv2/vex.lv2/>

mod-ui  | error: /usr/lib/lv2/Fluida.lv2/Fluida.ttl:48:4: missing ';' or '.'

mod-ui  | lilv_world_load_file(): error: Error loading file `file:///usr/lib/lv2/Fluida.lv2/Fluida.ttl'

mod-ui  | ALSA lib confmisc.c:165:(snd_config_get_card) Cannot get card index for MODDUO

mod-ui  | Cannot connect to server socket err = No such file or directory

mod-ui  | Cannot connect to server request channel

mod-ui  | jack server is not running or cannot be started

mod-ui  | JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock

mod-ui  | JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock

mod-ui  | WARNING:tornado.general:Connect error on fd 11: ECONNREFUSED

mod-ui  | WARNING:tornado.general:Connect error on fd 10: ECONNREFUSED

mod-ui  | WARN: Missing file /mod/data/mod_api_key.pub with the public API KEY

mod-ui  | ERROR: Failed to open HMI serial port, error was:

mod-ui  | SerialBase.__init__() got an unexpected keyword argument 'writeTimeout'

mod-ui  | Using FakeHMI => <mod.development.FakeHMI object at 0x7feef2833cd0>

mod-ui  | cc start socket missing

mod-ui  | last state file does not exist or is corrupt

mod-ui  | last state file does not exist or is corrupt

mod-ui  | HMI initialized right away

mod-ui  | Readsock none

mod-ui  | Writesock none

mod-ui  | Host failed to initialize, is the backend running?

mod-ui  | jack client deactivated NOT

mod-ui  | jack client deactivated NOT

mod-ui exited with code 1

I've tried pipewire-jack, just for the sake of it, but it didn't work with the mod-host. I had to start jack manually. I've ended up using pulse-jack and having jack started at login through qjackctl started in the X Session...

[ajboni]

Some success on initial tests: Managed to play an audio file with mpv using jack in a archlinux container with pipewire and the hosts pipewire sockets mounted.

docker run -it -v /run/user/1000/pipewire-0:/tmp/pipewire-0 -e XDG_RUNTIME_DIR=/tmp --rm archlinux /bin/bash

***  Inside container ***
pacman -Sy
pacman -S pipewire
pacman -S pipewire-jack

*** Try ALSA
pacman -S alsa-utils
speaker-test -Dpipewire -c2

*** pw-play
pw-play /usr/share/sounds/alsa/Rear_Center.wav 
pacman -S mpv

*** JACK
mpv --ao=jack /usr/share/sounds/alsa/Rear_Center.wav 

Using this great answer as starting point: https://stackoverflow.com/questions/28985714/run-apps-using-audio-in-a-docker-container/75775875#75775875

Its debatable if we would want to run the mod-host container on pipewire, but I'm hoping we can "pass" the pw jack server from the host to the client, somehow...

[raidolo]

Some success on initial tests: Managed to play an audio file with mpv using jack in a archlinux container with pipewire and the hosts pipewire sockets mounted.

docker run -it -v /run/user/1000/pipewire-0:/tmp/pipewire-0 -e XDG_RUNTIME_DIR=/tmp --rm archlinux /bin/bash

***  Inside container ***
pacman -Sy
pacman -S pipewire
pacman -S pipewire-jack

*** Try ALSA
pacman -S alsa-utils
speaker-test -Dpipewire -c2

*** pw-play
pw-play /usr/share/sounds/alsa/Rear_Center.wav 
pacman -S mpv

*** JACK
mpv --ao=jack /usr/share/sounds/alsa/Rear_Center.wav 

Using this great answer as starting point: https://stackoverflow.com/questions/28985714/run-apps-using-audio-in-a-docker-container/75775875#75775875

Its debatable if we would want to run the mod-host container on pipewire, but I'm hoping we can "pass" the pw jack server from the host to the client, somehow...

Great job! Yeah I'm not sure it worth the effort... also because that means it should be parametric somehow inside the container and if the pw dev is passed along then a script should do the rest... or we can have two containers one for pure jack and one for pipewire... but I'm not sure that mod-host is able to connect to pipewire even without using docker. Did you try to run mod-host over pipewire on your system without docker?

Anyway... Were you able to start the web interface and do some play with it?

I solved another issue regarding the samples, I was not able to go lower than 128 samples on my audio card, the audio was robotic and damaged, I got a flood of xhci_usb messages in the dmesg when trying 64 samples. Then I found out that the issue was the USB controller I was using, so I've connected the PreSonus 26c interface on the USB-C port on the back of my motherboard and suddenly it was ok at 64 samples. Very weird.

[ajboni]

Great job! Yeah I'm not sure it worth the effort... also because that means it should be parametric somehow inside the container and if the pw dev is passed along then a script should do the rest... or we can have two containers one for pure jack and one for pipewire... but I'm not sure that mod-host is able to connect to pipewire even without using docker. Did you try to run mod-host over pipewire on your system without docker?

Thanks! No, that's the next step hopefully tonight! The issue I see is that more and more distros are coming with pipewire as default and in general the experience is quite smooth, I can see supporting both jack and jack through pipewire could be beneficial (mostly for the casual users like myself). I'm pretty sure pw "just" run jack on the background, and a JACK aware app does not need to know anything about it. So I guess it will be doing the same that I did with pw but sharing the JACK backend instead... I guess...

Anyway... Were you able to start the web interface and do some play with it? Not yet! I don;t know how to bypass this pw thing without altering my system too much !

I solved another issue regarding the samples, I was not able to go lower than 128 samples on my audio card, the audio was robotic and damaged, I got a flood of xhci_usb messages in the dmesg when trying 64 samples. Then I found out that the issue was the USB controller I was using, so I've connected the PreSonus 26c interface on the USB-C port on the back of my motherboard and suddenly it was ok at 64 samples. Very weird.

That's weird, maybe your previous usb port was 2.0 or connected to a hub (the front usb of the case it likely a hub?)

Anyways, thanks for you efforts. Will update here as soon as I have it working under my system.

[raidolo]

Thanks! No, that's the next step hopefully tonight! The issue I see is that more and more distros are coming with pipewire as default and in general the experience is quite smooth, I can see supporting both jack and jack through pipewire could be beneficial (mostly for the casual users like myself). I'm pretty sure pw "just" run jack on the background, and a JACK aware app does not need to know anything about it. So I guess it will be doing the same that I did with pw but sharing the JACK backend instead... I guess...

Mhhh I don't remember if jack was running in background, from what I've read I've completely substituted jack doing this: Copy the pipewire-jack-x86_64.conf to /etc/ld.so.conf.d/ and then running ldconfig, following the "JACK Emulation" documentation found here: https://gitlab.freedesktop.org/pipewire/pipewire/-/blob/master/INSTALL.md I dind't try using pw-jack script... maybe that was my problem. For convenience you could first try to build the orginal mod-host source on the host instead of docker, if it works in the host, then we'll see how to make it work in the docker container, it take little time to compile.... Let me know any development,

That's weird, maybe your previous usb port was 2.0 or connected to a hub (the front usb of the case it likely a hub?)

Anyways, thanks for you efforts. Will update here as soon as I have it working under my system.

Nope the front USB-C port is connected on the motherboard internal header pins, with the cable you know, the uber giga fast usb 3.2 headers... but in linux it's recognized as the same bus of all other device connected (perhaps?), maybe the drivers are not that good. The USB-C port in the back instead is recognized as a diferent bus in lsusb.... don't ask, I dunno :)

[raidolo]

I did a quick test.... the problem is not the mod-host, it's the mod-ui, that in the tornado web server has a jack client embedded inside...

jack client activated
WARNING:tornado.general:Connect error on fd 23: ECONNREFUSED
WARNING:tornado.general:Connect error on fd 22: ECONNREFUSED

mod-host is running fine with pipewire...

raid@debian:~/mod-host$ ./mod-host -p 5555 -f 5556  
Forking... child PID: 5715
raid@debian:~/mod-host$ error: failed to open file /usr/lib/lv2/remaincalm-plugins/manifest.ttl (No such file or directory)
lilv_world_load_file(): error: Error loading file `file:///usr/lib/lv2/remaincalm-plugins/manifest.ttl'
lilv_world_load_bundle(): error: Error reading file:///usr/lib/lv2/remaincalm-plugins/manifest.ttl
mod-host ready!

raid@debian:~/mod-host$ ps -ef |egrep 'wire|mod-host'
raid        5031    1428  0 20:08 ?        00:00:01 /usr/bin/pipewire
raid        5033    1428  0 20:08 ?        00:00:00 /usr/bin/wireplumber
raid        5715       1  0 20:16 pts/1    00:00:00 ./mod-host -p 5555 -f 5556
raid        5729    3372  0 20:16 pts/1    00:00:00 grep -E wire|mod-host
raid@debian:~/mod-host$ 

[ajboni]

Great find, can confirm! trying to set up everything from scratch inside an arch linux container compiling everything from original sources (mod repos). Same behaviour. BUT, I can see the JACK/PW client in helvum created once I launch mod-ui which I suppose is great news!

EDIT: And mod-ui is able to succesfully connect mod-monitor to the capture port

WARNING:tornado.general:Connect error on fd 24: ECONNREFUSED
DEBUG:root:[host] popped from queue: feature_enable processing 0
DEBUG:root:[host] sending -> feature_enable processing 0
DEBUG:root:[host] received as response <- 'resp 0\x00'
DEBUG:root:[host] popped from queue: transport 0 4.000000 120.000000
DEBUG:root:[host] sending -> transport 0 4.000000 120.000000
last state file does not exist or is corrupt
HMI initialized right away
DEBUG:root:[host] popped from queue: remove -1
DEBUG:root:[host] sending -> remove -1
DEBUG:root:[host] popped from queue: state_tmpdir /mod/mod-ui/data/pedalboard-tmp-data
DEBUG:root:[host] sending -> state_tmpdir /mod/mod-ui/data/pedalboard-tmp-data
DEBUG:root:[host] popped from queue: feature_enable aggregated-midi 1
DEBUG:root:[host] sending -> feature_enable aggregated-midi 1
DEBUG:root:[host] popped from queue: transport 1 4.000000 120.000000
DEBUG:root:[host] sending -> transport 1 4.000000 120.000000
**DEBUG:root:[host] popped from queue: connect system:capture_1 mod-monitor:in_1
DEBUG:root:[host] sending -> connect system:capture_1 mod-monitor:in_1
DEBUG:root:[host] popped from queue: connect system:capture_2 mod-monitor:in_2
DEBUG:root:[host] sending -> connect system:capture_2 mod-monitor:in_2**
DEBUG:root:[host] popped from queue: state_load /root/.pedalboards/default.pedalboard
DEBUG:root:[host] sending -> state_load /root/.pedalboards/default.pedalboard
DEBUG:root:[host] popped from queue: feature_enable processing 2
DEBUG:root:[host] sending -> feature_enable processing 2
DEBUG:root:[host] received as response <- 'resp 0\x00'
DEBUG:root:[host] popped from queue: remove 9993
DEBUG:root:[host] sending -> remove 9993
DEBUG:root:[host] received as response <- 'resp 0\x00'
DEBUG:root:[host] popped from queue: transport_sync none
DEBUG:root:[host] sending -> transport_sync none
DEBUG:root:[host] received as response <- 'resp 0\x00'
Readsock none

EDIT2: This is the line that is failing to connect and failing to set up the readsock object: https://github.com/moddevices/mod-ui/blob/a637efa97c631ef41bc33012a220c627049b5615/mod/host.py#L1245

mod host works ok.

EDIT3: mod-host is not ok, I think its failing to launch whatever it does on port 5556, that's why we get a connection reffused from mod-ui

(modui-env) [root@209a85def9d2 mod-ui]# netstat -tulpn | grep LISTEN
tcp        0      0 0.0.0.0:5555            0.0.0.0:*               LISTEN      7011/mod-host   

Getting closer!!

[ajboni]

OK you can ignore everything from last message. I can confirm that creating a blank arch container, and installing everything from source and just replacing jack2 with pipewire jack works both for mod-host and mod-ui . I had to mount the pipewire dev as before:

docker run -it -p 8888 -v /run/user/1000/pipewire-0:/tmp/pipewire-0 -v /dev/shm:/dev/shm:rw -e XDG_RUNTIME_DIR=/tmp archlinux /bin/bash

  pacman -Sy
  pacman -S pipewire-jack git lilv make base-devel readline fftw pkg-config python3 python-pip python-virtualenv
  git clone https://github.com/moddevices/mod-host.git /mod/mod-host/source 
  cd /mod/mod-host/source 
  make
  git clone https://github.com/moddevices/mod-ui.git /mod/mod-ui
  cd /mod/mod-ui/
  virtualenv modui-env
  source modui-env/bin/activate
  pip3 install -r requirements.txt
  make -C utils
  export PYTHON_VERSION=3.11
  sed -i -e 's/collections.MutableMapping/collections.abc.MutableMapping/' /mod/mod-ui/modui-env/lib/python$PYTHON_VERSION/site-packages/tornado/httputil.py && sed -i -e 's/Image.ANTIALIAS/Image.Resampling.LANCZOS/' /mod/mod-ui/modtools/pedalboard.py && sed -i -e 's/var\ DEBUG\ \=\ true/var\ DEBUG\ \=\ false/' /mod/mod-ui/html/index.html
  /mod/mod-host/source/mod-host -p 5555 -f 5556
  python3 ./server.py
  export MOD_DEV_HOST=0
  export MOD_DEV_ENVIRONMENT=0
  python3 server.py 

launch the web, modify jack transport and bpm, check on host that is in sync 

I think the only approach would be to mantaining another image which has pipewire installed as I don't think they both can be installed at the same time in the system.

[raidolo]

modify jack transport

Cool! Yes I believe maintaining a single container would be too much effort.

What do you mean with this quoted? How and where?

[ajboni]

modify jack transport

Cool! Yes I believe maintaining a single container would be too much effort.

What do you mean with this quoted? How and where?

Sorry, I missed this, I mean changing the BPM / hit play in the web and verify that it changes the OS settings as well.