Server + STT server services autostart on Arch Linux

[giostark]

Hi All , This project make me think that maybe some hope still could exist. I would need some support to automate the start of the server on Arch Linux. I read the wiki about systemd: https://wiki.archlinux.org/title/Systemd I also tried to adapt other service's file but I encounter always the same issue.

Jan 18 15:34:20 virtual-arch run-sepia.sh[25946]: Elasticsearch is NOT yet setup (or not running with default settings)! Run setup.sh Jan 18 15:34:25 virtual-arch run-sepia.sh[25954]: /home/ios/Downloads/SEPIA-Home/run-sepia.sh: line 12: cd: elasticsearch: No such fi Jan 18 15:34:25 virtual-arch run-sepia.sh[25955]: /home/ios/Downloads/SEPIA-Home/run-sepia.sh: line 13: ./run.sh: No such file or dir Jan 18 15:34:25 virtual-arch run-sepia.sh[25956]: /home/ios/Downloads/SEPIA-Home/run-sepia.sh: line 15: ./wait.sh: No such file or di Jan 18 15:34:25 virtual-arch run-sepia.sh[25954]: Checking Elasticsearch setup ...

I tried this sort of service's content:

[Unit] Description=SEPIA SERVER After=network.target

[Service] Type=oneshot RemainAfterExit=yes User=ios Group=users LimitNOFILE=65536 Restart=on-failure RestartSec=5 WatchdogSec=30 RootDirectory=/home/ios/Downloads/SEPIA-Home/ ExecStart=run-sepia.sh ExecStop=shutdown-sepia.sh ExecReload=restart-sepia.sh NotifyAccess=all PrivateDevices=true AmbientCapabilities=CAP_NET_BIND_SERVICE TimeoutSec=300

[Install] WantedBy=multi-user.target

I tried also absolute path (in that case the service just stuck doing nothing). I tried to apply the "root" folder path but is ineffective. Also trying simpler service produce the same result. Is like the server need lots of stuff be launched from its folder. I'm a noob and not a coder, is barely all new here , so probably I'm missing several concepts. The server launched from it's folder run just fine (despite some error - missing apt stuffs during the installation). I was also able to build under Arch also the STT server. I can connect to the server via client on Android app from remote and accomplish all the tests. All this is stored on a virtual machine well routed via free.ddns to a nas in my home. (I already host a Xmmp server for a personal chat system via ejabberd). I placed the sepia folder in the home user path as suggested here :"On Linux "~/SEPIA/" (no root!) is highly recommended if you want to use the boot and SSL scripts later: unzip SEPIA-Home.zip -d ~/SEPIA" . Tips? Thanks for your time and hard work :-)

[fquirin]

Hi giostark,

This project make me think that maybe some hope still could exist.

:smiley:

I would need some support to automate the start of the server on Arch Linux.

Ok lets see what we can do. I've been neglecting this topic for a while now since the 'reboot' Cronjob was an OK solution so far ('@reboot sleep 60 && ~/SEPIA/on-reboot.sh;') but it would certainly be nice to have a proper systemd service! Unfortunately I'm not a systemd expert as well but with some experimenting we should get this working ^^.

My best guess right now is that the scope is wrong when the scripts are called meaning that the service is probably called them from the wrong directory. Is this your SEPIA server location: /home/ios/Downloads/SEPIA-Home ? Usually I would recommend this: /home/ios/SEPIA if "ios" is the user that runs SEPIA. Otherwise you might get problems with the backup and update scripts (and maybe already during setup but that seems to have worked right?). Unfortunately the "~/SEPIA" path is hardcoded in some scripts :see_no_evil: .

There are some parameters in the Systemd description which might be able to fix this. Can you try this:

RootDirectory=/home/ios/Downloads
WorkingDirectory=/home/ios/Downloads/SEPIA-Home

Maybe ExecSearchPath can be used as well. I'm not sure.

If this doesn't work you can try to modify 'run-sepia.sh' to actually switch to the correct folder. Simply add something like cd $HOME/SEPIA or cd $HOME/Downloads/SEPIA-Home in the very first line.

There is a rather important line in the 'on-reboot' script that should be called at least once on system reboot as well (or made permanent somehow) or you might get memory problems with the Elasticsearch at some point:

# SETUP VM FOR ELASTICSEARCH - https://www.elastic.co/guide/en/elasticsearch/reference/5.3/vm-max-map-count.html
sudo sysctl -w vm.max_map_count=262144

[giostark]

Hi Quirin! (edit: finally success ! strait to 5:20 in the morning lol , few noobysh questions at the end)

Thanks for the fast response :D .
I'm not good at those stuff so it toke me long to face errors... I mange to make all start with 3 different services. One for command. At this I was unable to unify in one or even two service the 3 commands. If you will do some of better please let me know :-) For the server: the info about the hard coding pushed me to reinstall the server in /home/ios(user)/SEPIA/ . I was not capable to use "ExecStartPre=" or multiple "ExecStart=" , even root path didn't work I had to use fixed path. Boh , maybe was some my mistakes... again if you will produce some service file I'll test it soon as possible. By the way this service work for start , stop, restart the Sepia server:

[Unit] Description=SEPIA SERVER After=network.target

[Service] Type=oneshot RemainAfterExit=yes User=ios Group=users LimitNOFILE=65536 Restart=on-failure RestartSec=5 WatchdogSec=30 ExecStart=/home/ios/SEPIA/run-sepia.sh ExecStop=/home/ios/SEPIA/shutdown-sepia.sh ExecReload=/home/ios/SEPIA/restart-sepia.sh NotifyAccess=all PrivateDevices=true AmbientCapabilities=CAP_NET_BIND_SERVICE TimeoutSec=300

[Install] WantedBy=multi-user.target

The service status:

[ios@virtual-arch ~]$ sudo systemctl status sepia.service [sudo] password for ios:

• sepia.service - SEPIA SERVER Loaded: loaded (/etc/systemd/system/sepia.service; enabled; vendor preset: disabled) Active: active (exited) since Thu 2022-01-27 04:55:31 CET; 11min ago Process: 279 ExecStart=/home/ios/SEPIA/run-sepia.sh (code=exited, status=0/SUCCESS) Main PID: 279 (code=exited, status=0/SUCCESS) Tasks: 149 (limit: 6932) Memory: 815.1M CPU: 1min 45.146s CGroup: /system.slice/sepia.service |-399 /bin/java -Xms256m -Xmx256m -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccu> |-759 java -jar -Xms200m -Xmx200m sepia-assist-v2.5.2.jar --my |-823 java -jar -Xms200m -Xmx200m sepia-chat-v1.3.2.jar --my `-862 java -jar -Xms128m -Xmx128m sepia-teach-v2.2.2.jar --my

Jan 27 04:55:31 virtual-arch run-sepia.sh[942]: Example1: http://virtual-arch.local:20721/tools/index.html Jan 27 04:55:31 virtual-arch run-sepia.sh[942]: Example2: http://[IP]:20721/tools/index.html Jan 27 04:55:31 virtual-arch run-sepia.sh[942]: Example3: http://[IP]:20721/app/index.html Jan 27 04:55:31 virtual-arch run-sepia.sh[942]: If you've installed NGINX proxy with self-signed SSL try: Jan 27 04:55:31 virtual-arch run-sepia.sh[942]: Example4: https://virtual-arch.local:20726/sepia/assist/tools/index.html Jan 27 04:55:31 virtual-arch run-sepia.sh[942]: Example5: https://virtual-arch.local:20726/sepia/assist/app/index.html Jan 27 04:55:31 virtual-arch run-sepia.sh[942]: Please note: if this is a virtual machine the hostname might not work to contact the s> Jan 27 04:55:31 virtual-arch run-sepia.sh[942]: For more info about secure context and microphone access in the SEPIA client see: Jan 27 04:55:31 virtual-arch run-sepia.sh[942]: https://github.com/SEPIA-Framework/sepia-docs/wiki/SSL-for-your-Server Jan 27 04:55:31 virtual-arch systemd[1]: Finished SEPIA SERVER.

For the memory service at boot (just edited the run-sepia from the script you linked above):

[Unit] Description=Script

[Service] ExecStart=/home/ios/SEPIA/mem-fix.sh

[Install] WantedBy=multi-user.target

The status for the memory script (should be executed also if ExecStart after the execution stop the service) :

[ios@virtual-arch ~]$ sudo systemctl status mem-fix-sepia.service [sudo] password for ios:

• mem-fix-sepia.service - Script Loaded: loaded (/etc/systemd/system/mem-fix-sepia.service; enabled; vendor preset: disabled) Active: inactive (dead) since Thu 2022-01-27 04:54:28 CET; 19min ago Process: 264 ExecStart=/home/ios/SEPIA/mem-fix.sh (code=exited, status=0/SUCCESS) Main PID: 264 (code=exited, status=0/SUCCESS) CPU: 71ms

Jan 27 04:54:28 virtual-arch systemd[1]: Started Script. Jan 27 04:54:28 virtual-arch sudo[266]: root : PWD=/ ; USER=root ; COMMAND=/usr/bin/sysctl -w vm.max_map_count=262144 Jan 27 04:54:28 virtual-arch sudo[266]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=0) Jan 27 04:54:28 virtual-arch mem-fix.sh[270]: vm.max_map_count = 262144 Jan 27 04:54:28 virtual-arch sudo[266]: pam_unix(sudo:session): session closed for user root Jan 27 04:54:28 virtual-arch systemd[1]: mem-fix-sepia.service: Deactivated successfully.

Finally the server STT service that run the docker and stop the container (removed the "-it" for missing TTY).

[Unit] Description=Sepia STT Server Requires=docker.service After=docker.service

[Service]

Restart=always

ExecStart=sudo /usr/bin/docker run --rm --name=sepia-stt -p 20741:20741 sepia/stt-server:vosk_amd64 ExecStop=sudo /usr/bin/docker stop sepia/stt-server:vosk_amd64

[Install] WantedBy=default.target

The status of the sepiaSTT service:

[ios@virtual-arch ~]$ sudo systemctl status sepiaSTT.service

• sepiaSTT.service - Sepia STT Server Loaded: loaded (/etc/systemd/system/sepiaSTT.service; enabled; vendor preset: disabled) Active: active (running) since Thu 2022-01-27 04:54:33 CET; 22min ago Main PID: 559 (sudo) Tasks: 9 (limit: 6932) Memory: 62.5M CPU: 473ms CGroup: /system.slice/sepiaSTT.service |-559 sudo /usr/bin/docker run --rm --name=sepia-stt -p 20741:20741 sepia/stt-server:vosk_amd64 `-560 /usr/bin/docker run --rm --name=sepia-stt -p 20741:20741 sepia/stt-server:vosk_amd64

Jan 27 04:54:33 virtual-arch systemd[1]: Started Sepia STT Server. Jan 27 04:54:33 virtual-arch sudo[559]: root : PWD=/ ; USER=root ; COMMAND=/usr/bin/docker run --rm --name=sepia-stt -p 20741:2074> Jan 27 04:54:33 virtual-arch sudo[559]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=0) lines 1-14/14 (END)

All the 2 servers start , stop and restart fine. Also come well loaded at boot. Cheer !

1) After I start manually (in the console) the STT server, the console stay pending without returning to the cursor , is normal? I had to check in an other tab the status.

2)I had to compile Qt4 for install "fatrat" without whom the STT server service fail to start via sepiaSTT.service. Will be never ported the fatrat to qt5 right? (also fatrat seems a little obsolete in the arch aur repo but still functioning ).

[fquirin]

Hi @giostark

thanks for testing this and all the systemd files. It will be very helpful to build some "official" files!

Something I noticed is Active: active (exited) from SEPIA server. I think the 'exit' status might be because the run script is actually built to start the servers (assist, teach, chat) in the background and will exit right away.

For the memory service at boot (just edited the run-sepia from the script you linked above)

There might be a better solution according to Elasticsearch docs: To set this value permanently, update the vm.max_map_count setting in /etc/sysctl.conf. To verify after rebooting, run sysctl vm.max_map_count. This way you could skip the service :-).

After I start manually (in the console) the STT server, the console stay pending without returning to the cursor , is normal? I had to check in an other tab the status.

It depends on your Docker run flags. If you add -d it should go to the background right away but for the service script you probably want to keep using the "foreground" version so that systemd knows if its actually still "active". Otherwise you might get the same "exit" status as from SEPIA-Home server.

I had to compile Qt4 for install "fatrat" without whom the STT server service fail to start via sepiaSTT.service. Will be never ported the fatrat to qt5 right? (also fatrat seems a little obsolete in the arch aur repo but still functioning ).

SEPIA doesn't use Qt so this is probably a Docker or systemd issue :thinking: at least I've never seen that error before.

[giostark]

Ok Quirin , is clear all you wrote. 1) The sepia service execute several stuff so probably must be like it is (with final exit status) . 2) I'll add that line in sysctl.conf and disable the service. (this info could be added in the sepia-installation-and-setup and in the Wiki - now seems explained only in raspberry Pi installation guide) :-) 3) Fine I'll leave the things like this for the STT server (not exited service). 4) Yep qt4 it was for make docker run in background.

Eh... I would like to have the knowledge for build it . Maybe just manually building as I have done could be still helpful for some in the Wiki. I could try study better this but I can't guarantee a timeline. Digging in all the options of the service files and test it. My mind is fascinated by the scripting and I would like to come closer to this world. I have to face also other stuff as 3D printing , photography and other projects ("business" related). I'll try to stay closer to this project as I can :-) It is really fascinating ! I'm Italian , could I help with some translation?

[fquirin]

I'll add that line in sysctl.conf and disable the service

I've double-checked this and here is the correct command or at least what works for me ^^: sudo su -c "echo 'vm.max_map_count=262144' >> /etc/sysctl.d/99-sysctl.conf" I've also added a check and info message to the SEPIA run command for the next version :slightly_smiling_face:

Yep qt4 it was for make docker run in background

Docker is confusing sometimes :laughing:

Eh... I would like to have the knowledge for build it

You mean building the whole project from source code? Or some specific parts? The SEPIA-Home package in general is rather easy to build. There is some info about it here. I haven't tested these scripts for a while now :sweat_smile: ... my actual build machine is a Raspberry Pi 3 I haven't touched for some time :laughing: The most complicated part to build is probably the Android App because it currently requires some manual settings in Android Studio and specific version. I'm planning to optimize this to be able to upload the App to FDroid as well at some point.

My mind is fascinated by the scripting and I would like to come closer to this world. I have to face also other stuff as 3D printing , photography and other projects

The curse of our modern world ... too many possibilities, too many interesting stuff ... oh I know it, trust me. Partially the reason why updates to SEPIA usually take much longer than planned =)

I'm Italian , could I help with some translation?

Actually yes :-). Unfortunately implementing a new language is a very painful process :see_no_evil: mainly because there are a lot of language specific, hand-crafted rules, but there are some basics that can be done. The files in ~SEPIA/sepia-assist-server/Xtensions/Assistant/.. are the pre-defined answers of SEPIA and some "hard-coded" input commands. It's not trivial to translate those files but not too hard either. Let me know if you're interested ;-)

[giostark]

I'm in. I'll' translate the "answer" and "command" in Italian. I'll copy and open the english version and , line after line , I'll translate it in Italian. Ok , this will not take few minutes but I'll do it easily . When I'll have finish this task we will proceed with others ;-) We are messing this post lol with lots of OT . Where i can write to you (mail or pm for instruction and guide lines etc) ?

[fquirin]

Since translating SEPIA is a question that comes up from time to time I'd prefer to discuss this in the "Discussions" section: https://github.com/SEPIA-Framework/sepia-docs/discussions. Maybe we just start a topic called "Translating SEPIA" or something and try to put together a best-practices list or a set of instructions :-). If you still prefer to contact me via email you can use address linked somewhere in the https://sepia-framework.github.io/ "credits" section (info at ... I'll not write it down here to prevent spam ^^). There was another topic around where someone started to do a Spanish translation. I think there are some useful tips we should collect.

[giostark]

Ok updated to the new version , succeeded fine. My old service files , after I copied them to the SEPIA folder, they start again fine. I have to propose a good sepia.service and sepiaSTT.service files to be enabled during installation. During the update those files should be copied to the new SEPIA folder or regenerated. Also , always in the installer and the update file, could be useful check /etc/sysctl.d/99-sysctl.conf if exist and is correct. Thanks for your huge work ! ps I wrote you ;-) I'm proceeding with the translation.

[fquirin]

I have to propose a good sepia.service and sepiaSTT.service files to be enabled during installation. During the update those files should be copied to the new SEPIA folder or regenerated

Yes I think this would be very useful. Instead of copying them by default it could become a 'setup' option, something like 'Enable SEPIA service'. Btw I've changed some scripts (e.g. the run script) to set the working directory automatically to the script folder. This might help when you start SEPIA from different folders.

[fquirin]

Just a quick info if you're using services or scripts that auto-start the SEPIA server: If you're running the update script please deactivate any services in advance or strange things will happen as soon as the update script closes any running instances!

[giostark]

Thanks Florian, For habitude I already do that. (so also for the ejabberd XMPP server). Good to know anyways !