Have you considered to have docker support in order not to pollute the system? Here is my solution.

hero-intelligent commented 1 year ago

Have you considered to have docker support? It isn't very difficult to do so, as there is commands in Ubuntu, which can directly be the RUN command in Dockerfile. So just build an image FROM ubuntu:22.10 after which then add the RUN commands. The only thing that has to be put into consideration is display, graphics card and working directory to be mounted onto the docker container, which can be set up in docker-compose.yml file, which you programmers need to work out. It would be more convenient if you can wrap up the whole program into a deb package.

hero-intelligent commented 1 year ago

First, preinstall docker by the official command curl -fsSL https://get.docker.com | bash -s docker. and execute these commands:

git clone https://github.com/Anjok07/ultimatevocalremovergui.git uvr
cd uvr
touch Dockerfile
touch docker-compose.yml

then edit the two files newly made.

Execute these commands everytime you run this program. Change to super user mode if needed.

xhost +
docker compose up

Now this is the Dockerfile:

FROM ubuntu:22.10

COPY ./requirements.txt /uvr/requirements.txt

WORKDIR /uvr

RUN apt update && DEBIAN_FRONTEND=noninteractive apt install -y tzdata && \
    apt install -y ffmpeg python3-pip python3-tk freeglut3-dev x11-xserver-utils && \
    pip3 install -r requirements.txt -i https://pypi.mirrors.ustc.edu.cn/simple #This is 中科大 image repository of pip3.
#Change to your own image repository of pip3, or just delete the -i argument if you are not in China.

COPY . /uvr

VOLUME [ "/uvr/sources", "/uvr/results" ]

ENTRYPOINT [ "python3" ]

CMD [ "/uvr/UVR.py" ]

And docker-compose.yml:

services:
  uvr:
    build: .
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /tmp/.X11-unix:/tmp/.X11-unix
      - ./inputs:/uvr/inputs
      - ./outputs:/uvr/outputs
      - ./models:/uvr/models
    environment:
      - DISPLAY=unix:0.0
      - GDK_SCALE
      - GDK_DPI_SCALE
#    deploy:
#      resources:
#        reservations:
#          devices:
#            - driver: nvidia
#              device_ids: [ '0' ]
#              capabilities: [ gpu ]

hero-intelligent commented 1 year ago

First, preinstall docker by the official command and enable docker with non-user access.

curl -fsSL https://get.docker.com | bash -s docker
sudo groupadd docker
sudo usermod -aG docker $USER
newgrp docker

After that, you should need a reboot.

Then execute these commands in a directory where you want to place this software:

git clone https://github.com/Anjok07/ultimatevocalremovergui.git UVR
cd UVR
touch Dockerfile

then edit the Dockerfile newly made.

If you are an Nidia user, this is your Dockerfile.

# Linux x86_64 Minimum Required Driver Version for Nvidia >=525.60.13

FROM nvcr.io/nvidia/cuda:12.2.0-runtime-ubuntu22.04

# =============PYTHON INSTALL STARTS========================

RUN set -eux; \
    apt-get update; \
    apt-get install -y --no-install-recommends \
        ca-certificates \
        curl \
        gnupg \
        netbase \
        wget \
# https://bugs.debian.org/929417
        tzdata \
    ; \
    rm -rf /var/lib/apt/lists/*

# procps is very common in build systems, and is a reasonably small package
RUN apt-get update && apt-get install -y --no-install-recommends \
        git \
        mercurial \
        openssh-client \
        subversion \
        \
        procps \
    && rm -rf /var/lib/apt/lists/*

RUN set -ex; \
    apt-get update; \
    apt-get install -y --no-install-recommends \
        autoconf \
        automake \
        bzip2 \
        dpkg-dev \
        file \
        g++ \
        gcc \
        imagemagick \
        libbz2-dev \
        libc6-dev \
        libcurl4-openssl-dev \
        libdb-dev \
        libevent-dev \
        libffi-dev \
        libgdbm-dev \
        libglib2.0-dev \
        libgmp-dev \
        libjpeg-dev \
        libkrb5-dev \
        liblzma-dev \
        libmagickcore-dev \
        libmagickwand-dev \
        libmaxminddb-dev \
        libncurses5-dev \
        libncursesw5-dev \
        libpng-dev \
        libpq-dev \
        libreadline-dev \
        libsqlite3-dev \
        libssl-dev \
        libtool \
        libwebp-dev \
        libxml2-dev \
        libxslt-dev \
        libyaml-dev \
        make \
        patch \
        unzip \
        xz-utils \
        zlib1g-dev \
        \
# https://lists.debian.org/debian-devel-announce/2016/09/msg00000.html
        $( \
# if we use just "apt-cache show" here, it returns zero because "Can't select versions from package 'libmysqlclient-dev' as it is purely virtual", hence the pipe to grep
            if apt-cache show 'default-libmysqlclient-dev' 2>/dev/null | grep -q '^Version:'; then \
                echo 'default-libmysqlclient-dev'; \
            else \
                echo 'libmysqlclient-dev'; \
            fi \
        ) \
    ; \
    rm -rf /var/lib/apt/lists/*

# ensure local python is preferred over distribution python
ENV PATH /usr/local/bin:$PATH

# http://bugs.python.org/issue19846
# > At the moment, setting "LANG=C" on a Linux system *fundamentally breaks Python 3*, and that's not OK.
ENV LANG C.UTF-8

# runtime dependencies
RUN set -eux; \
    apt-get update; \
    apt-get install -y --no-install-recommends \
        libbluetooth-dev \
        tk-dev \
        uuid-dev \
    ; \
    rm -rf /var/lib/apt/lists/*

ENV GPG_KEY E3FF2839C048B25C084DEBE9B26995E310250568
ENV PYTHON_VERSION 3.9.18

RUN set -eux; \
    \
    wget -O python.tar.xz "https://www.python.org/ftp/python/${PYTHON_VERSION%%[a-z]*}/Python-$PYTHON_VERSION.tar.xz"; \
    wget -O python.tar.xz.asc "https://www.python.org/ftp/python/${PYTHON_VERSION%%[a-z]*}/Python-$PYTHON_VERSION.tar.xz.asc"; \
    GNUPGHOME="$(mktemp -d)"; export GNUPGHOME; \
    gpg --batch --keyserver hkps://keys.openpgp.org --recv-keys "$GPG_KEY"; \
    gpg --batch --verify python.tar.xz.asc python.tar.xz; \
    gpgconf --kill all; \
    rm -rf "$GNUPGHOME" python.tar.xz.asc; \
    mkdir -p /usr/src/python; \
    tar --extract --directory /usr/src/python --strip-components=1 --file python.tar.xz; \
    rm python.tar.xz; \
    \
    cd /usr/src/python; \
    gnuArch="$(dpkg-architecture --query DEB_BUILD_GNU_TYPE)"; \
    ./configure \
        --build="$gnuArch" \
        --enable-loadable-sqlite-extensions \
        --enable-optimizations \
        --enable-option-checking=fatal \
        --enable-shared \
        --with-system-expat \
        --without-ensurepip \
    ; \
    nproc="$(nproc)"; \
    EXTRA_CFLAGS="$(dpkg-buildflags --get CFLAGS)"; \
    LDFLAGS="$(dpkg-buildflags --get LDFLAGS)"; \
    make -j "$nproc" \
        "EXTRA_CFLAGS=${EXTRA_CFLAGS:-}" \
        "LDFLAGS=${LDFLAGS:-}" \
        "PROFILE_TASK=${PROFILE_TASK:-}" \
    ; \
# https://github.com/docker-library/python/issues/784
# prevent accidental usage of a system installed libpython of the same version
    rm python; \
    make -j "$nproc" \
        "EXTRA_CFLAGS=${EXTRA_CFLAGS:-}" \
        "LDFLAGS=${LDFLAGS:--Wl},-rpath='\$\$ORIGIN/../lib'" \
        "PROFILE_TASK=${PROFILE_TASK:-}" \
        python \
    ; \
    make install; \
    \
# enable GDB to load debugging data: https://github.com/docker-library/python/pull/701
    bin="$(readlink -ve /usr/local/bin/python3)"; \
    dir="$(dirname "$bin")"; \
    mkdir -p "/usr/share/gdb/auto-load/$dir"; \
    cp -vL Tools/gdb/libpython.py "/usr/share/gdb/auto-load/$bin-gdb.py"; \
    \
    cd /; \
    rm -rf /usr/src/python; \
    \
    find /usr/local -depth \
        \( \
            \( -type d -a \( -name test -o -name tests -o -name idle_test \) \) \
            -o \( -type f -a \( -name '*.pyc' -o -name '*.pyo' -o -name 'libpython*.a' \) \) \
        \) -exec rm -rf '{}' + \
    ; \
    \
    ldconfig; \
    \
    python3 --version

# make some useful symlinks that are expected to exist ("/usr/local/bin/python" and friends)
RUN set -eux; \
    for src in idle3 pydoc3 python3 python3-config; do \
        dst="$(echo "$src" | tr -d 3)"; \
        [ -s "/usr/local/bin/$src" ]; \
        [ ! -e "/usr/local/bin/$dst" ]; \
        ln -svT "$src" "/usr/local/bin/$dst"; \
    done

# if this is called "PIP_VERSION", pip explodes with "ValueError: invalid truth value '<VERSION>'"
ENV PYTHON_PIP_VERSION 23.0.1
# https://github.com/docker-library/python/issues/365
ENV PYTHON_SETUPTOOLS_VERSION 58.1.0
# https://github.com/pypa/get-pip
ENV PYTHON_GET_PIP_URL https://github.com/pypa/get-pip/raw/9af82b715db434abb94a0a6f3569f43e72157346/public/get-pip.py
ENV PYTHON_GET_PIP_SHA256 45a2bb8bf2bb5eff16fdd00faef6f29731831c7c59bd9fc2bf1f3bed511ff1fe

RUN set -eux; \
    \
    wget -O get-pip.py "$PYTHON_GET_PIP_URL"; \
    echo "$PYTHON_GET_PIP_SHA256 *get-pip.py" | sha256sum -c -; \
    \
    export PYTHONDONTWRITEBYTECODE=1; \
    \
    python get-pip.py \
        --disable-pip-version-check \
        --no-cache-dir \
        --no-compile \
        "pip==$PYTHON_PIP_VERSION" \
        "setuptools==$PYTHON_SETUPTOOLS_VERSION" \
    ; \
    rm -f get-pip.py; \
    \
    pip --version

CMD ["python3"]

# =============PYTHON INSTALL COMPLETE========================

# Set non-root user

ARG USERNAME=uvr
ARG USER_UID=1000
ARG USER_GID=$USER_UID

# Create the user
RUN groupadd --gid $USER_GID $USERNAME \
    && useradd --uid $USER_UID --gid $USER_GID -m $USERNAME \
    && apt-get update \
    && apt-get install -y sudo \
    && rm -rf /var/lib/apt/lists/* \
    && echo $USERNAME ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/$USERNAME \
    && chmod 0440 /etc/sudoers.d/$USERNAME

# Install UVR dependencies

RUN apt-get update \
    && apt-get install -y ffmpeg \
        x11-xserver-utils xwayland \
        libx11-6 libxext-dev libxrender-dev libxinerama-dev libxi-dev libxrandr-dev libxcursor-dev libxtst-dev tk-dev \
        freeglut3-dev libgirepository1.0-dev \
    && rm -rf /var/lib/apt/lists/*

# Set the default user.
WORKDIR /home/$USERNAME
USER $USERNAME

# Install UVR

# RUN git clone https://github.com/Anjok07/ultimatevocalremovergui.git UVR

WORKDIR /home/$USERNAME/UVR
COPY ./requirements.txt .
ENV PATH=/home/$USERNAME/.local/bin:$PATH

RUN pip3 install scikit-learn \
    && pip3 install -r requirements.txt
    && pip3 install pygobject

COPY . .

VOLUME [ "/home/$USERNAME/UVR/sources", "/home/$USERNAME/UVR/results" ]

CMD [ "python3" , "UVR.py" ]

If you are NOT an Nvidia user:


FROM python:3.9.18-bookworm

# Set non-root user

ARG USERNAME=uvr
ARG USER_UID=1000
ARG USER_GID=$USER_UID

# Create the user
RUN groupadd --gid $USER_GID $USERNAME \
    && useradd --uid $USER_UID --gid $USER_GID -m $USERNAME \
    && apt-get update \
    && apt-get install -y sudo \
    && rm -rf /var/lib/apt/lists/* \
    && echo $USERNAME ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/$USERNAME \
    && chmod 0440 /etc/sudoers.d/$USERNAME

# Install UVR dependencies

RUN apt-get update \
    && apt-get install -y ffmpeg \
        x11-xserver-utils xwayland \
        libx11-6 libxext-dev libxrender-dev libxinerama-dev libxi-dev libxrandr-dev libxcursor-dev libxtst-dev tk-dev \
        freeglut3-dev libgirepository1.0-dev \
    && rm -rf /var/lib/apt/lists/*

# Set the default user.
WORKDIR /home/$USERNAME
USER $USERNAME

# Install UVR

# RUN git clone https://github.com/Anjok07/ultimatevocalremovergui.git UVR

WORKDIR /home/$USERNAME/UVR
COPY ./requirements.txt .
ENV PATH=/home/$USERNAME/.local/bin:$PATH

RUN pip3 install scikit-learn \
    && pip3 install -r requirements.txt
    && pip3 install pygobject

COPY . .

VOLUME [ "/home/$USERNAME/UVR/sources", "/home/$USERNAME/UVR/results" ]

CMD [ "python3" , "UVR.py" ]

Then build the image.

docker build -t uvr .

Execute these commands everytime you run this program. Change to super user mode if needed.

xhost local:docker

docker run -it --rm \
    -e XDG_RUNTIME_DIR=/tmp \
    -e WAYLAND_DISPLAY=$WAYLAND_DISPLAY \
    -v $XDG_RUNTIME_DIR/$WAYLAND_DISPLAY:/tmp/$WAYLAND_DISPLAY  \
    --user=$(id -u):$(id -g) \
    -e XAUTHORITY=$XAUTHORITY \
    -v /tmp/.X11-unix:/tmp/.X11-unix \
    -v /etc/localtime:/etc/localtime \
    -e DISPLAY=unix$DISPLAY \
    -e GDK_SCALE \
    -e GDK_DPI_SCALE \
    --network=host \
    --name uvr \
    -v ./sources:/home/uvr/UVR/sources \
    -v ./results:/home/uvr/UVR/results \
    -v ./models:/home/uvr/UVR/models \
    --gpus all \
    uvr python3 UVR.py

NOTES:

The inputs can ONLY place at sources directory, and the outputs can ONLY place at results directory. Linux x86_64 Minimum Required Driver Version for Nvidia >=525.60.13, otherwise you won't be able to use cuda. --gpus all should be omitted when you have no Nvidia GPU installed.

sxiii commented 10 months ago

Hi @hero-intelligent !

Thanks for your Docker example. Build it OK with just 2 small changes, but when tried to run:

``docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]].

Also tried without --gpus all flag, then it can't find NVIDIA driver.

Also last section looked to me very like x11docker. So I decided to try it with it, but it also ended up not working. Got any ideas?

Thanks

hero-intelligent commented 10 months ago

@sxiii I Apologize, But I have no Nvdia drivers at hands right now. This is only an example or a workaround for encapsulating this program in a large container in order not to pollute the system, and only tested working on CPU only. I write the file for Nvidia GPU just assumed that it will work but I am not able to get it tested or fixed.

Besides, this program can only run on machines with X11 server as far as I know, but nowadays more and more machines will be using Wayland. Also, Containerizing this program will lead to huge power loss. So my version is kind of like a joke nowadays.

Possible solution: Change the cuda version in the first line to be version that is compatible with both your GPU model and GPU driver

I can only figure out this solution, But I'm still new in this field. Sorry again for that. If you get into other problems, search on the internet and try to solve it yourself. If you have other breakthroughs, i'll be appreciate it if you could comment in this thread down below.

dweb0 commented 10 months ago

Huge thanks to hero-intelligent for the docker setup! I was able to get it the non-NVIDIA version working on my 2015 Macbook Pro running 10.14.6 Mojave with the following steps. Hopefully this can help other macOS users.

Installed https://www.xquartz.org. I discovered that I had to reboot my computer for it to work.
Used hero-intelligent's Dockerfile for "NOT an Nvidia user", with one modification. Before the pip3 install step, added this statement to fix the scikit-learn error:

ENV SKLEARN_ALLOW_DEPRECATED_SKLEARN_PACKAGE_INSTALL=True

To build the docker image, used additional build-args so that USER_UID and USER_GID matched my own.

docker build -t uvr . \
    --build-arg USER_UID=$(id -u) \
    --build-arg USER_GID=$(id -g)

Added a line to /etc/hosts file. You probably need sudo for this.

echo "$(ipconfig getifaddr en0) $(hostname)" >> /etc/hosts

Used this command to run the container. Modify the mounted volumes (-v SOURCE:DEST) to include the directories you want UVR to be able to access.

export HOSTNAME=$(hostname)

xhost +${HOSTNAME}

docker run -it --rm \
    --user=$(id -u):$(id -g) \
    -v /tmp/.X11-unix:/tmp/.X11-unix \
    -v "${HOME}/.Xauthority":/tmp/.Xauthority \
    -e XAUTHORITY=/tmp/.Xauthority \
    -e DISPLAY="${HOSTNAME}":0 \
    -v /etc/localtime:/etc/localtime \
    -e XDG_RUNTIME_DIR=/tmp \
    -e PYGLET_SHADOW_WINDOW=0 \
    --net=host \
    --name uvr \
    -v "$(pwd)/sources":/home/uvr/UVR/sources \
    -v "$(pwd)/results":/home/uvr/UVR/results \
    -v "$(pwd)/models":/home/uvr/UVR/models \
    -v "${HOME}/Music":/home/uvr/UVR/music \
    uvr

Anjok07 / ultimatevocalremovergui

Have you considered to have docker support in order not to pollute the system? Here is my solution. #379