[BUG] Setup broken after latest file restructuring

[Romeo1984]

Describe the bug After running /scripts/setup.sh the /scripts/pikaraoke.sh script has broken paths To Reproduce Steps to reproduce the behavior:

1. git clone https://github.com/vicwomg/pikaraoke.git
2. cd pikaraoke
3. cd scripts
4. ./setup.sh
5. ./pikaraoke.sh
6. Error: pi@pikaraoke:~/pikaraoke/scripts $ ./pikaraoke.sh Warning: Running remotely via SSH. Setting DISPLAY=:0.0 to run on host display python3: can't open file '/home/pi/pikaraoke/scripts/app.py': [Errno 2] No such file or directory

Expected behavior It used to run in ~/pikaraoke/pikaraoke.sh just fine, now that the file was moved to the ./scripts dir, it no longer works.

Screenshots None - All text Platform (please complete the following information):

• Hardware: Raspberry Pi 4
• OS: Raspberry Pi OS
• OS version: Raspberry pi Debian Bookworm 64-bit / 32-bit
• Browser chrome
• Version [e.g. 22]

Additional context None

[vicwomg]

Apologies for the lack of prior notice about this, but wasn't really sure how to announce it.

The new preferred way to install pikaraoke is: pip install pikaraoke

Then to launch it, to simply run: pikaraoke

This makes pikaraoke a standard python module and removes the need for git and the need to access the source code. It's all explained in the README, which also explains that the scripts method may no longer work. Let me know if there is a specific reason you still need to launch pikaraoke this way.

[Romeo1984]

Great! As long as Pikaraoke can start as a systemd service then we are golden.

What is the best way to upgrade to the new version? Just nuke the old pikaraoke directory? It might be good to include upgrade/migration directions coming from the older versions.

[Romeo1984]

After installing pip, I get this error:

'pi@pikaraoke:~ $ pip install pikaraoke error: externally-managed-environment

× This environment is externally managed ╰─> To install Python packages system-wide, try apt install python3-xyz, where xyz is the package you are trying to install.

If you wish to install a non-Debian-packaged Python package,
create a virtual environment using python3 -m venv path/to/venv.
Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
sure you have python3-full installed.

For more information visit http://rptl.io/venv

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages. hint: See PEP 668 for the detailed specification. pi@pikaraoke:~ $'

@vicwomg What are the assumptions to get the new way of installing to work?

[vicwomg]

It's all explained in the readme, so I encourage you to check it out and let me know if something there is not clear, but you can choose to ignore that warning by passing --break-system-packages to the command. TBH you're probably fine if this is a dedicate pi device that's not running a bunch of python programs that will conflict with pikaraoke.

Otherwise, the "proper" way to install a pip package is to do it in an virtual environment. Again, see the Readme for specifics

[Romeo1984]

Yes - This is a dedicated karaoke device with HDMI out based on FullPageOS project. I am running pikaraoke as a systemd service in headless mode, then launching the pikaraoke splash page once X11 and Chromium loads. It is working very nicely on the older version.

[vicwomg]

Then run pip install pikaraoke --break-system-packages it will install fine, and you can launch pikaraoke --headless from anywhere, as it will be installed to your PATH

[Romeo1984]

During the 'pip install pikaraoke --break-system-packages' I got several of these:

'WARNING: The script unidecode is installed in '/home/pi/.local/bin' which is not on PATH. Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.'

I had to add 'export 'PATH="/home/pi/.local/bin:$PATH`

If I run it from SSH as pi, it works. But if I try to execute it from systemd service, I get an error:

pi@pikaraoke:~ $ sudo service pikaraoke status × pikaraoke.service - pikaraoke Loaded: loaded (/etc/systemd/system/pikaraoke.service; enabled; preset: enabled) Active: failed (Result: exit-code) since Sun 2024-09-01 12:41:59 MDT; 4s ago Duration: 59ms Process: 4191 ExecStart=/usr/bin/python3 /home/pi/.local/bin/pikaraoke --headless --download-path /home/pi/pikaraoke-songs --volume 0.75 (code> Main PID: 4191 (code=exited, status=1/FAILURE) CPU: 57ms

Sep 01 12:41:59 pikaraoke systemd[1]: Started pikaraoke.service - pikaraoke. Sep 01 12:41:59 pikaraoke python3[4191]: Traceback (most recent call last): Sep 01 12:41:59 pikaraoke python3[4191]: File "/home/pi/.local/bin/pikaraoke", line 5, in Sep 01 12:41:59 pikaraoke python3[4191]: from pikaraoke.app import main Sep 01 12:41:59 pikaraoke python3[4191]: ModuleNotFoundError: No module named 'pikaraoke' Sep 01 12:41:59 pikaraoke systemd[1]: pikaraoke.service: Main process exited, code=exited, status=1/FAILURE Sep 01 12:41:59 pikaraoke systemd[1]: pikaraoke.service: Failed with result 'exit-code'. lines 1-15/15 (END)

So, I added the same as su: I had to add 'export 'PATH="/home/pi/.local/bin:$PATH`

Still same error.

Here is my path as root: /home/pi/.local/lib/python3.11/site-packages/pikaraoke:/home/pi/.local/bin:/home/pi/.local/lib:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

[vicwomg]

ExecStart=/usr/bin/python3 /home/pi/.local/bin/pikaraoke --headless --download-path /home/pi/pikaraoke-songs --volume 0.75

Not sure this will make a difference, but Pikaraoke is a standalone executable so /home/pi/.local/bin/pikaraoke should not be previxed by "python3".

Should be more like: ExecStart=/home/pi/.local/bin/pikaraoke --headless --download-path /home/pi/pikaraoke-songs --volume 0.75

Other than that, I am not quite sure. @mariugul any suggestions? This sort of thing is what I was concerned with concering deprecation of the scripts launch method.

[Romeo1984]

@vicwomg okay. I followed your suggestion. I also added user=pi thinking it will use the $PATH as user pi in the systemd script. I now get this (can't find 'yt-dlp'):

pi@pikaraoke:~ $ sudo service pikaraoke status × pikaraoke.service - pikaraoke Loaded: loaded (/etc/systemd/system/pikaraoke.service; enabled; preset: enabled) Active: failed (Result: exit-code) since Sun 2024-09-01 13:37:11 MDT; 6s ago Duration: 1.275s Process: 4499 ExecStart=/home/pi/.local/bin/pikaraoke --headless --download-path /home/pi/pikaraoke-songs --volume 0.75 (code=exited, status=1> Main PID: 4499 (code=exited, status=1/FAILURE) CPU: 1.273s

Sep 01 13:37:10 pikaraoke pikaraoke[4499]: with Popen(*popenargs, **kwargs) as process: Sep 01 13:37:10 pikaraoke pikaraoke[4499]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Sep 01 13:37:10 pikaraoke pikaraoke[4499]: File "/usr/lib/python3.11/subprocess.py", line 1024, in init Sep 01 13:37:10 pikaraoke pikaraoke[4499]: self._execute_child(args, executable, preexec_fn, close_fds, Sep 01 13:37:10 pikaraoke pikaraoke[4499]: File "/usr/lib/python3.11/subprocess.py", line 1901, in _execute_child Sep 01 13:37:10 pikaraoke pikaraoke[4499]: raise child_exception_type(errno_num, err_msg, err_filename) Sep 01 13:37:10 pikaraoke pikaraoke[4499]: FileNotFoundError: [Errno 2] No such file or directory: 'yt-dlp' Sep 01 13:37:11 pikaraoke systemd[1]: pikaraoke.service: Main process exited, code=exited, status=1/FAILURE Sep 01 13:37:11 pikaraoke systemd[1]: pikaraoke.service: Failed with result 'exit-code'. Sep 01 13:37:11 pikaraoke systemd[1]: pikaraoke.service: Consumed 1.273s CPU time. lines 1-18/18 (END)

Reading up on running python apps as a systemd service, all paths in the python scripts must be absolute.

[vicwomg]

There is a --youtubedl-path if you can manage to track down the absolute path of youtube-dl which yt-dlp but I would agree this is getting way too complex. There must be a better way. Is it possible to install to a virtual env and have the ExecStart activate the venv before hand? That is essentially what the legacy launch script does.

[Romeo1984]

@vicwomg That did it! Here is my final systemd service file:

pi@pikaraoke:~ $ cat /etc/systemd/system/pikaraoke.service [Unit] Description=pikaraoke After=multi-user.target

[Service] WorkingDirectory=/home/pi/ Type=simple ExecStart=/home/pi/.local/bin/pikaraoke --headless --youtubedl-path /home/pi/.local/bin/yt-dlp --download-path /home/pi/pikaraoke-songs --volume 0.75 User=pi Group=pi Environment="PYTHONPATH=$PYTHONPATH:/home/pi/.local/lib/python3.11/site-packages"

[Install] WantedBy=multi-user.target

Maybe you can add it to the README to save others the time if they want to do the same?

[vicwomg]

I'm going to keep this open for other folks who might be researching this issue. It's difficult to add universal instructions to the readme because everyone's setups are different.

I may prioritize getting the scripts working again if all this becomes too much trouble.

[Romeo1984]

Is an acceptable practice to post an entire How To guide to the Discussion section? I was thinking a lot of people can benefit from turning a pi into a karaoke machine running in kiosk mode. On Sep 1, 2024, at 3:08 PM, Vicwomg @.***> wrote: I'm going to keep this open for other folks who might be researching this issue. It's difficult to add universal instructions to the readme because everyone's setups are different. I may prioritize getting the scripts working again if all this becomes too much trouble.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you modified the open/close state.Message ID: @.***>

[mariugul]

pikaraoke didn't describe anywhere how to use it as a systemd service, so I would say this was outside of scope of what to take into consideration when we did the upgrade to making pikaraoke a python package. Also, it's still possible to keep using tag v1.2.0 for legacy Since this was solved I would add it to a readme. Perhaps @vicwomg wants to setup a GitHub wiki, I don't think it should go into discussions. Due to a systemd service not being described by pikaraoke, we didn't test this at all when upgrading. Should we add it somewhere in a readme, then it would also probably be a tested feature. However, this was a good find!

Just some extra info: I think one issue here was that we subprocess yt-dlp now instead of a full path to where it's installed. This was part of refactor to using pathlib.Path instead of strings. However, that full PR is not in yet. Since systemd seems to want to use absolute paths, we could make yt-dlp resolve to the current environment like <current env>/yt-dlp. A part of that refactor was that it should be unecessary to specify yt-dlp path when it's already just a dependency in the python package.

[vicwomg]

While specifically running as a service is not called out in the README, it is well understood (to me) that auto-launching pikaraoke as a standalone appliance is a very common use case and I consider it a supported feature.

There really ought to be a clean way to do it, and python virtual environments make it harder.

I will set up a wiki for this sort of thing. Probably should move TROUBLESHOOTING.md there as well

[mariugul]

While specifically running as a service is not called out in the README, it is well understood (to me) that auto-launching pikaraoke as a standalone appliance is a very common use case and I consider it a supported feature.

There really ought to be a clean way to do it, and python virtual environments make it harder.

I will set up a wiki for this sort of thing. Probably should move TROUBLESHOOTING.md there as well

Then we can add it to testing. I'm pretty sure you can add a virtual env to the systemd service as well, it shouldn't be an issue. I just haven't tested that.

[vicwomg]

I'm pretty sure you can add a virtual env to the systemd service as well, it shouldn't be an issue. I just haven't tested that.

Some guides I've read says to just set ExecStart to directly run from the path as <venv_home>/bin/pikaraoke, but in my testing, the part that doesn't work is the call to update yt-dlp. It will try to run global pip, instead of the one in the <venv_home/bin, so it doesn't upgrade the correct binary

[mariugul]

I'm pretty sure you can add a virtual env to the systemd service as well, it shouldn't be an issue. I just haven't tested that.

Some guides I've read says to just set ExecStart to directly run from the path as <venv_home>/bin/pikaraoke, but in my testing, the part that doesn't work is the call to update yt-dlp. It will try to run global pip, instead of the one in the <venv_home/bin, so it doesn't upgrade the correct binary

That sounds correct with the venv path. I think the issue is that we call yt-dlp directly form PATH now. I can fix it when I do the pathlib.Path refactor. The systemd service doesn't really activate the venv I think, it only runs the app inside it. So because the app only calls yt-dlp in PATH it will try the global one is my guess.

[vicwomg]

This is separate from the yt-dlp path issue. I think it has less to do with yt-dlp path and more to do with pip's venv path. The update_ytdl function calls pip, and it doesn't seem to know to use pip in the venv to update the yt-dlp within that venv.

[mariugul]

This is separate from the yt-dlp path issue. I think it has less to do with yt-dlp path and more to do with pip's venv path. The update_ytdl function calls pip, and it doesn't seem to know to use pip in the venv to update the yt-dlp within that venv.

This is the code I find:

def upgrade_youtubedl(self):
        logging.info("Upgrading youtube-dl, current version: %s" % self.youtubedl_version)
        try:
            output = (
                check_output([self.youtubedl_path, "-U"], stderr=subprocess.STDOUT)
                .decode("utf8")
                .strip()
            )
        except CalledProcessError as e:
            output = e.output.decode("utf8")
        logging.info(output)
        if "You installed yt-dlp with pip or using the wheel from PyPi" in output:
            try:
                logging.info("Attempting youtube-dl upgrade via pip3...")
                output = check_output(["pip3", "install", "--upgrade", "yt-dlp"]).decode("utf8")
            except FileNotFoundError:
                logging.info("Attempting youtube-dl upgrade via pip...")
                output = check_output(["pip", "install", "--upgrade", "yt-dlp"]).decode("utf8")
            logging.info(output)
        self.get_youtubedl_version()
        logging.info("Done. New version: %s" % self.youtubedl_version)

From what I can tell it would first use yt-dlp -U to try to upgrade and only if that fails will it try to do pip install --upgrade yt-dlp, if it gets that far(?). There might be something here in this logic that creates a weird situation. Anyhow, that can also be solved if we first find the current environment before calling any commands. It's a path issue

[vicwomg]

@Romeo1984 I have added an auto-launch wiki

https://github.com/vicwomg/pikaraoke/wiki/Auto%E2%80%90launching-pikaraoke

That guide uses the autolaunch and not systemd, which I assume will be more simple and ok for most users. Let me know if there's a compelling reason I should include a systemd option

[Romeo1984]

[Unit] Description=pikaraoke After=multi-user.target

[Service] WorkingDirectory=/home/pi/ Type=simple ExecStart=/home/pi/.local/bin/pikaraoke --headless --youtubedl-path /home/pi/.local/bin/yt-dlp --download-path /home/pi/pikaraoke-songs --volume 0.75 User=pi Group=pi Environment="PYTHONPATH=$PYTHONPATH:/home/pi/.local/lib/python3.11/site-packages"

[Install] WantedBy=multi-user.target

@vicwomg I am using FullPageOS that is purpose-built for Kiosk mode that only launches Chromium. There is no desktop that loads, so there is no autolaunch option that is honored.