Running the "Real Time Voice Cloning" project in Docker

[seanlane]

Comments for my blog post about running the "Real Time Voice Cloning" project in Docker: https://sean.lane.sh/posts/2019/07/Running-the-Real-Time-Voice-Cloning-project-in-Docker/

[yxt132]

Thanks for the great work! I was able to have the docker image built and docker host running. However, when i try to use XQuartz to SSH into the docker container, I got the following error. It seems the connection was established and then something went wrong during authentication. But I double checked that I do have the file "id_rsa" in the /home/roy/.ssh folder. I wonder if it is because it needs sudo user access. But I failed to use switch to root user inside XQuartz although I can do that inside a normal terminal using the same password. Any idea why this has happened?

OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n 7 Dec 2017
debug1: Reading configuration data /home/roy/.ssh/config debug1: /home/roy/.ssh/config line 1: Applying options for voice debug1: Reading configuration data /etc/ssh/ssh_config debug1: /etc/ssh/ssh_config line 19: Applying options for * debug1: Connecting to localhost [127.0.0.1] port 2150. debug1: Connection established. debug1: identity file /home/roy/.ssh/id_rsa type 0 debug1: key_load_public: No such file or directory debug1: identity file /home/roy/.ssh/id_rsa-cert type -1 debug1: key_load_public: No such file or directory debug1: identity file /home/roy/.ssh/id_dsa type -1 debug1: key_load_public: No such file or directory debug1: identity file /home/roy/.ssh/id_dsa-cert type -1 debug1: key_load_public: No such file or directory debug1: identity file /home/roy/.ssh/id_ecdsa type -1 debug1: key_load_public: No such file or directory debug1: identity file /home/roy/.ssh/id_ecdsa-cert type -1 debug1: key_load_public: No such file or directory debug1: identity file /home/roy/.ssh/id_ed25519 type -1 debug1: key_load_public: No such file or directory debug1: identity file /home/roy/.ssh/id_ed25519-cert type -1 debug1: Local version string SSH-2.0-OpenSSH_7.6p1 Ubuntu-4ubuntu0.3 ssh_exchange_identification: read: Connection reset by peer

[yxt132]

btw, i am using macOS 10.13.6. the server is ubuntu 16.04

[seanlane]

@yxt132 Based on https://unix.stackexchange.com/a/321972, the public key file for all of the files mentioned in the debug1 messages aren't actual errors, just warnings, and the actual error is the last line: ssh_exchange_identification: read: Connection reset by peer This would indicate that something is occurring within the Docker host. It seems that the SSH server within the Docker container either isn't listening on port 22, your Docker host isn't connecting port 2150 to port 22 on the Docker container, or the SSH server in the container isn't running after all.

I would try modifying the Dockerfile to be the following (remember to replace your public key still as needed), noting that the differences are in the final two lines. Rebuild the image, and then open a second terminal on your Docker host. In the first terminal, start and run the container, which should display the verbose debug output from sshd. Then in the second one, ssh into the container from the Docker host, and you should get some more information about what is causing the issue in that output.

For reference, my Docker host is running Ubuntu Server 18.04 LTS, though I'm unaware of any differences that would be an issue.

FROM pytorch/pytorch

WORKDIR "/workspace"
RUN apt-get clean \
        && apt-get update \
        && apt-get install -y ffmpeg libportaudio2 openssh-server python3-pyqt5 xauth \
        && apt-get -y autoremove \
        && mkdir /var/run/sshd \
        && mkdir /root/.ssh \
        && chmod 700 /root/.ssh \
        && ssh-keygen -A \
        && sed -i "s/^.*PasswordAuthentication.*$/PasswordAuthentication no/" /etc/ssh/sshd_config \
        && sed -i "s/^.*X11Forwarding.*$/X11Forwarding yes/" /etc/ssh/sshd_config \
        && sed -i "s/^.*X11UseLocalhost.*$/X11UseLocalhost no/" /etc/ssh/sshd_config \
        && grep "^X11UseLocalhost" /etc/ssh/sshd_config || echo "X11UseLocalhost no" >> /etc/ssh/sshd_config
ADD Real-Time-Voice-Cloning/requirements.txt /workspace/requirements.txt
RUN pip install -r /workspace/requirements.txt
RUN echo "<REPLACE THIS SENTENCE (INCLUDING ARROWS) WITH YOUR SSH PUBLIC KEY ON THE DOCKER HOST>" \ 
    >> /root/.ssh/authorized_keys
RUN echo "export PATH=/opt/conda/bin:$PATH" >> /root/.profile
ENTRYPOINT ["sh", "-c", "/usr/sbin/sshd", "-D", "-d"]

[yxt132]

Thank you for your speedy response! I modified the Dockerfile as you suggested. However, when I tried to start the docker container and nothing happened and it seemed exited after after it was started, even though I used "-it" option to make it interactive. I checked the running container by "docker ps -a" and did not see anything related to it.

Then I added back "&& bash" right after "/usr/sbin/sshd" in the ENTRYPOINT. It seems to be working now. Thanks. I will keep you posted on the next steps!

[yxt132]

A quick update. It is working after I updated the docker image file. I guess @seanlane is right, perhaps SSH server in the container was not running. Everything works as expected now. Thanks again!

[seanlane]

Awesome, glad it worked out!

[scottbouma]

Thanks for the great article! I'm having the same trouble you were with /dev/snd. Can you by any chance share the code changes you made to save the output audio as a WAV file instead of playing within the application? Thanks!

[seanlane]

@scottbouma Sure, I changed the following in the toolbox/ui.py file:

1. Import soundfile somewhere at the top: import soundfile as sf
2. Find the play function and change from this:

def play(self, wav, 
    sd.stop()
    sd.play(wav, sample_rate)

to this

def play(self, wav, 
    sf.write('test_out.wav', wav, sample_rate)
    sd.stop()
    sd.play(wav, sample_rate)

It's not very clean, and I thought I saw some other methods that may already be there to write to disk, but this worked well enough for me to stop trying to get it to run and just play around with it.

[dfcv24]

good job! I use you dockerfile and connect between mac and remote ubuntu. but meet core dumped when input "python demo_toolbox.py" So I test like this

root@cd0e6b994966:~# python
Python 3.6.8 |Anaconda, Inc.| (default, Dec 30 2018, 01:22:34) 
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from PyQt5.QtWidgets import QApplication, QLabel
>>> app = QApplication([])
QFactoryLoader::QFactoryLoader() checking directory path "/opt/conda/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms" ...
QFactoryLoader::QFactoryLoader() looking at "/opt/conda/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqeglfs.so"
Found metadata in lib /opt/conda/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqeglfs.so, metadata=
{
    "IID": "org.qt-project.Qt.QPA.QPlatformIntegrationFactoryInterface.5.3",
    "MetaData": {
        "Keys": [
            "eglfs"
        ]
    },
    "archreq": 0,
    "className": "QEglFSIntegrationPlugin",
    "debug": false,
    "version": 331008
}

Got keys from plugin meta data ("eglfs")
QFactoryLoader::QFactoryLoader() looking at "/opt/conda/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqlinuxfb.so"
Found metadata in lib /opt/conda/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqlinuxfb.so, metadata=
{
    "IID": "org.qt-project.Qt.QPA.QPlatformIntegrationFactoryInterface.5.3",
    "MetaData": {
        "Keys": [
            "linuxfb"
        ]
    },
    "archreq": 0,
    "className": "QLinuxFbIntegrationPlugin",
    "debug": false,
    "version": 331008
}

Got keys from plugin meta data ("linuxfb")
QFactoryLoader::QFactoryLoader() looking at "/opt/conda/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqminimal.so"
Found metadata in lib /opt/conda/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqminimal.so, metadata=
{
    "IID": "org.qt-project.Qt.QPA.QPlatformIntegrationFactoryInterface.5.3",
    "MetaData": {
        "Keys": [
            "minimal"
        ]
    },
    "archreq": 0,
    "className": "QMinimalIntegrationPlugin",
    "debug": false,
    "version": 331008
}

Got keys from plugin meta data ("minimal")
QFactoryLoader::QFactoryLoader() looking at "/opt/conda/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqminimalegl.so"
Found metadata in lib /opt/conda/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqminimalegl.so, metadata=
{
    "IID": "org.qt-project.Qt.QPA.QPlatformIntegrationFactoryInterface.5.3",
    "MetaData": {
        "Keys": [
            "minimalegl"
        ]
    },
    "archreq": 0,
    "className": "QMinimalEglIntegrationPlugin",
    "debug": false,
    "version": 331008
}

Got keys from plugin meta data ("minimalegl")
QFactoryLoader::QFactoryLoader() looking at "/opt/conda/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqoffscreen.so"
Found metadata in lib /opt/conda/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqoffscreen.so, metadata=
{
    "IID": "org.qt-project.Qt.QPA.QPlatformIntegrationFactoryInterface.5.3",
    "MetaData": {
        "Keys": [
            "offscreen"
        ]
    },
    "archreq": 0,
    "className": "QOffscreenIntegrationPlugin",
    "debug": false,
    "version": 331008
}

Got keys from plugin meta data ("offscreen")
QFactoryLoader::QFactoryLoader() looking at "/opt/conda/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqvnc.so"
Found metadata in lib /opt/conda/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqvnc.so, metadata=
{
    "IID": "org.qt-project.Qt.QPA.QPlatformIntegrationFactoryInterface.5.3",
    "MetaData": {
        "Keys": [
            "vnc"
        ]
    },
    "archreq": 0,
    "className": "QVncIntegrationPlugin",
    "debug": false,
    "version": 331008
}

Got keys from plugin meta data ("vnc")
QFactoryLoader::QFactoryLoader() looking at "/opt/conda/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqwayland-egl.so"
Found metadata in lib /opt/conda/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqwayland-egl.so, metadata=
{
    "IID": "org.qt-project.Qt.QPA.QPlatformIntegrationFactoryInterface.5.3",
    "MetaData": {
        "Keys": [
            "wayland-egl"
        ]
    },
    "archreq": 0,
    "className": "QWaylandEglPlatformIntegrationPlugin",
    "debug": false,
    "version": 331008
}

Got keys from plugin meta data ("wayland-egl")
QFactoryLoader::QFactoryLoader() looking at "/opt/conda/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqwayland-generic.so"
Found metadata in lib /opt/conda/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqwayland-generic.so, metadata=
{
    "IID": "org.qt-project.Qt.QPA.QPlatformIntegrationFactoryInterface.5.3",
    "MetaData": {
        "Keys": [
            "wayland"
        ]
    },
    "archreq": 0,
    "className": "QWaylandIntegrationPlugin",
    "debug": false,
    "version": 331008
}

Got keys from plugin meta data ("wayland")
QFactoryLoader::QFactoryLoader() looking at "/opt/conda/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqwayland-xcomposite-egl.so"
Found metadata in lib /opt/conda/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqwayland-xcomposite-egl.so, metadata=
{
    "IID": "org.qt-project.Qt.QPA.QPlatformIntegrationFactoryInterface.5.3",
    "MetaData": {
        "Keys": [
            "wayland-xcomposite-egl"
        ]
    },
    "archreq": 0,
    "className": "QWaylandXCompositeEglPlatformIntegrationPlugin",
    "debug": false,
    "version": 331008
}

Got keys from plugin meta data ("wayland-xcomposite-egl")
QFactoryLoader::QFactoryLoader() looking at "/opt/conda/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqwayland-xcomposite-glx.so"
Found metadata in lib /opt/conda/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqwayland-xcomposite-glx.so, metadata=
{
    "IID": "org.qt-project.Qt.QPA.QPlatformIntegrationFactoryInterface.5.3",
    "MetaData": {
        "Keys": [
            "wayland-xcomposite-glx"
        ]
    },
    "archreq": 0,
    "className": "QWaylandXCompositeGlxPlatformIntegrationPlugin",
    "debug": false,
    "version": 331008
}

Got keys from plugin meta data ("wayland-xcomposite-glx")
QFactoryLoader::QFactoryLoader() looking at "/opt/conda/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqwebgl.so"
Found metadata in lib /opt/conda/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqwebgl.so, metadata=
{
    "IID": "org.qt-project.Qt.QPA.QPlatformIntegrationFactoryInterface.5.3",
    "MetaData": {
        "Keys": [
            "webgl"
        ]
    },
    "archreq": 0,
    "className": "QWebGLIntegrationPlugin",
    "debug": false,
    "version": 331008
}

Got keys from plugin meta data ("webgl")
QFactoryLoader::QFactoryLoader() looking at "/opt/conda/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqxcb.so"
Found metadata in lib /opt/conda/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqxcb.so, metadata=
{
    "IID": "org.qt-project.Qt.QPA.QPlatformIntegrationFactoryInterface.5.3",
    "MetaData": {
        "Keys": [
            "xcb"
        ]
    },
    "archreq": 0,
    "className": "QXcbIntegrationPlugin",
    "debug": false,
    "version": 331008
}

Got keys from plugin meta data ("xcb")
QFactoryLoader::QFactoryLoader() checking directory path "/opt/conda/bin/platforms" ...
loaded library "/opt/conda/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqxcb.so"
qt.qpa.xcb: could not connect to display 
qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, wayland-egl, wayland, wayland-xcomposite-egl, wayland-xcomposite-glx, webgl, xcb.

Aborted (core dumped)

I am not familiar to Qt5, have you any idea for this?

[seanlane]

@dfcv24 I'm not very familiar with Qt5 either, but my initial thought is that maybe Qt5 was updated and somehow broke between when I created this Dockerfile and when you tried running it. One possible option is to attempt to use a previous version of Qt5.

I don't have time at the moment to debug this, but I will try to get back to this when I can.

[dfcv24]

@seanlane Yes! your thought is right, In the requirement.txt, PyQt5 has no version control, so I install newest 5.13.0 by default, I change it to 5.11.3 and success.

[seanlane]

@dfcv24 Thanks for figuring it out and letting me know, glad it worked! I'll make a note of that change in the blog post for others to follow.

[gabrielmontagne]

Note to selves, on newer nvidia container runtimes (I think 19.10 and newer) use --gpus all instead of --runtime=nvidia.

[saraBadawi]

Did you try to install these packages on AWS what your recommendation of service providing servers with GPU

[seanlane]

@saraBadawi No, I didn't use a cloud provider, I was using a private server with Ubuntu 16.04 LTS. That said, because it's running with a Docker image, I would imagine it wouldn't take too much work to get it going in a cloud environment, assuming that you're familiar with getting NVIDIA CUDA working.

[gabrielmontagne]

I ran @seanlane 's tuto successfully on one of google's nvidia machines, https://console.cloud.google.com/marketplace/browse?q=nvidia some come with cuda configured and even the nvidia docker container runtime.

[swimnbird]

I was following some different instructions and tried installing Cuda on my Mac OS X Mojave and had an issue where I kept getting ‘CUDA_ERROR_NOT_INITILIZED’ errors when trying to load the real-time-vocal-cloner. It also gave me a weird error in the OS System Preferences panel where it says it wasn’t up to date but wouldn’t let me update it. I bring it up because I don’t want to just get the same error.

I’d like to follow these instructions but I’ve read that step 1 isn’t possible unless I have high Sierra 10.13, is that true? If so I will downgrade but I want to make sure I’m not making a different mistake.

Ps. I am running a mid-2014 MacBook Pro, 2.5 ghz Core i7, with a NVIDIA GeForce GT 750M with 2GB of GDDR5 memory. I don’t know if all of that I relevant but I should be able to run real-time-voice-cloner sense I have a supported graphics card for it, right?

[seanlane]

To be honest, I'm not sure. I don't have a mac with an nVidia GPU, so I was running this on a remote server that had a compatible GPU. Unfortunately, according to this issue, it appears that step 1 is impossible no matter the macOS version: https://github.com/NVIDIA/nvidia-docker/issues/101

One option you could pursue is what @gabrielmontagne mentioned with using a virtual machine from Google: https://console.cloud.google.com/marketplace/browse?q=nvidia

[swimnbird]

To be honest, I'm not sure. I don't have a mac with an nVidia GPU, so I was running this on a remote server that had a compatible GPU. Unfortunately, according to this issue, it appears that step 1 is impossible no matter the macOS version: NVIDIA/nvidia-docker#101

One option you could pursue is what @gabrielmontagne mentioned with using a virtual machine from Google: https://console.cloud.google.com/marketplace/browse?q=nvidia

So update, I downgraded to high Sierra 10.13.6 and got nvidia drivers up to date no problems with the help of some YouTube tutorials, however I encountered an error with TensorFlow not being able to download on my Mac so I gave up and restored. I’m gonna try and put Ubuntu on a bootable drive and see if I have any luck with that and if not I’ll try a google server. Do you know if you have to pay for them? They seem pretty pricey unless there’s a free option.

EDIT found no way to get ubuntu working, it froze every installation so I continued on the mac, this is the specific error I'd get and am getting

ERROR: Could not find a version that satisfies the requirement tensorflow==2.0.1 (from -r requirements.txt (line 48)) (from versions: 2.2.0rc3, 2.2.0rc4, 2.2.0)
ERROR: No matching distribution found for tensorflow==2.0.1 (from -r requirements.txt (line 48))

[seanlane]

I've heard anecdotally that there is some free GPU access via https://colab.research.google.com/ but I haven't used it much myself. I had access to a Linux server with GPUs, so I never personally used either a mac machine nor the linked machines above. They are just possible avenues to explore in case you can't get it running on your mac machine.

[Surefyre]

Not got it working yet, trying on my AWS Blender GPU instance...

If you get 'cannot connect to the Docker daemon. Is 'docker daemon' running on this host?' then do: sudo chmod 666 /var/run/docker.sock

Which wil allow non-root to connect to it

[0xrushi]

I get gcc no such file or directory error. Changing the Dockerfile works

FROM pytorch/pytorch

WORKDIR "/workspace"
RUN apt-get clean \
        && apt-get update \
        && apt-get install -y ffmpeg libportaudio2 openssh-server python3-pyqt5 xauth gcc build-essential\
        && apt-get -y autoremove \
        && mkdir /var/run/sshd \
        && mkdir /root/.ssh \
        && chmod 700 /root/.ssh \
        && ssh-keygen -A \
        && sed -i "s/^.*PasswordAuthentication.*$/PasswordAuthentication no/" /etc/ssh/sshd_config \
        && sed -i "s/^.*X11Forwarding.*$/X11Forwarding yes/" /etc/ssh/sshd_config \
        && sed -i "s/^.*X11UseLocalhost.*$/X11UseLocalhost no/" /etc/ssh/sshd_config \
        && grep "^X11UseLocalhost" /etc/ssh/sshd_config || echo "X11UseLocalhost no" >> /etc/ssh/sshd_config
ADD Real-Time-Voice-Cloning/requirements.txt /workspace/requirements.txt
RUN pip install -r /workspace/requirements.txt
RUN echo "<REPLACE THIS SENTENCE (INCLUDING ARROWS) WITH YOUR SSH PUBLIC KEY ON THE DOCKER HOST>" \ 
    >> /root/.ssh/authorized_keys
RUN echo "export PATH=/opt/conda/bin:$PATH" >> /root/.profile
ENTRYPOINT ["sh", "-c", "/usr/sbin/sshd && bash"]
CMD ["bash"]

[seanlane]

Thanks for posting the comment, I'll try to find some time and update the post.

[an-internet-user001]

Hi, I know I am late to the party on this one, but curious if there has been any progress on being able to use a docker image on MacOS, and still being able to leverage GPU's for processing RVC? I know there is an NVIDIA Docker add-on(?) but not sure if this can be used on docker that's being ran natively on MacOS. Chat-GPT tells me I could use VM-ware but was curious if anyone has a recommended solution?