README.md instructions are not sufficient for a fresh install

[harshhpareek]

I tried setting this up on a fresh linux install and found that the steps in the readme are incomplete to set this up from scratch. The steps mentioned are:

npm install
npm start
# source $VIRTUALENV_HOME/bin/activate # optional
pip3 install -r requirements.txt

I needed to follow the following additional steps to get this working:

• Electron: Some dependency is missing. Setting up the electron example project by following the instructions here installed it: https://github.com/electron/electron-quick-start
• npm audit fix Not sure why, but it doesn't work without this.
• If using GPU, Install cuda by following the instructions here: https://docs.nvidia.com/cuda/#installation-guides
• Install pytorch (if using GPU, with cuda) by following the instructions here: https://pytorch.org/get-started/locally/
• Add at least one model file to the models/ directory. Make sure the game name directory is the same as the game name in the voice set json file. Without this it throws an error when enabling use gpu in the settings:

  Traceback (most recent call last):
  File "./server.py", line 129, in do_POST
  fastpitch_model.device = torch.device('cuda' if use_gpu else 'cpu')
  AttributeError: 'int' object has no attribute 'device'
  {'device': 'gpu'}

Without doing this, the loading fastpitch model dialog appears but does not progress. Electron should show an error dialog here instead of just printing the error in a log file, but this is a separate issue from the one being discussed here. I believe the above steps are sufficient because the DEBUG* files and the xVA-Synth\server.log file do not have any errors when I try to generate sounds. (Side note: I am on linux, so the \ above caused the file to be created in the parent directory. use pathlib in python 3 to make this work across platforms)

Resolution:

• I'm not familiar with NPM packages to point out which packages are missing for NPM to get electron working. Most likely changing the package requirements file will fix this.
  • What version of electron is needed? The npm install command installs v2.0.18, while it looks like the current version is 11.x.x
• For the cuda and pytorch dependencies, it is probably better for the user to install them manually. The links above should be added to the README
  • What version of CUDA is recommended? For me, this is working with cuda 10.2, the current pytorch default in pip.

[harshhpareek]

In v1.0.3, I had to add models/waveglow_256channels_universal_v4.pt from the nexus page manually to get this working.

[DanRuta]

Thank you for your detailed post regarding this. I admit, this section was a little bare.

It is true, CUDA and pytorch are necessary dependencies. I am unsure what the Electron quick-start is required for.

Models can be downloaded from the nexus pages, and they have the correct file structure required. The gpu error is caused by an earlier error in the code causing some other code not to run. The earlier error is the mising waveglow checkpoint which you correctly point out must be downloaded from nvidia and placed in the models folder.

I use v2.0.0 (this is an old project). Keep it there for behaviour consistent to mine. I use CUDA 10.1 but this shouldn't matter, so long as it matches the pytorch version you download.

[harshhpareek]

Regarding Electron quick start: i meant that some dependency is missing, which that project installed

[datatypevoid]

Thank you @harshhpareek, this really got me rolling. A few more notes for anyone else who steps down this path:

• Electron 2 will not launch, it crashes repeatedly; upgrading to 4 fixed this, so change it in package.json accordingly.
• Ensure ffmpeg is installed via your system package manager, eg. apt install ffmpeg
• npm run install
• npm audit fix
• pip3 install ffmpeg-python is needed for output to conclude properly, or you need to disable it in settings (it was enabled by default for me)
  • If you want to use ffmpeg, you need to change the ffmpeg command call(s) in python/audio_post.py
  • Wherever you find ffmpeg_path=... set it to ffmpeg_path="ffmpeg" so that it finds the version installed in your system PATH.
• The waveglow models are archived on nexus (I used this; not sure if there are implications I am unaware of, but it does generate
• The requirements files have a few dependencies not available on linux (pywin32; pywin32-ctypes). I commented these out and did pip3 install requirements.txt

[DanRuta]

For the ffmpeg thing, I ship the ffmpeg.exe in the compiled version on Steam / Nexusmods. For linux, I suppose it wouldn't help anyway even if I did include it in the repo. I don't use linux, but maybe an easier thing to do rather than changing code might be to place a simlink between the place where the exe would be, and wherever your ffmpeg is installed on linux (maybe - I'm not sure).

The waveglow models are a legacy thing, they are no longer needed (not since v2.0). You can still use them if you wish, however. Nowadays, the per-voice individual HiFi-GAN vocoders are preferable, as they are fine-tuned specifically to the voice, rather than using some off-the-shelf waveglow one-size-fits-all vocoder (and they are faster). In upcoming model versions, I think this will go away also, with the models being end-to-end.

I'm working on the next major version (v3), I'll finally update the Electron version to make that easier. Are you sure about tqdm not being available on linux?

[datatypevoid]

Yeah, you're probably right about symlinking, I was going for quick and dirty to make sure it would work on my setup. You are also correct about tqdm, that misunderstanding was the result of something else -- having installed on a second machine this morning, I realized that.

On the topic of waveglow, I don't have options for any models besides WaveGlow, Big WaveGlow, and quick and dirty? How can I enabled the HiFi-GAN vocoders (sorry if I've just missed the instructions, I will have another look of course)? For testing, I am only using the Female Dunmer voice from Morrowind, just to note in case that has something to do with it?

EDIT: Also, how rude of me to not lead with this, but thank you for open-sourcing this incredible project. That is mighty generous, and then sharing your models and research tidbits too. Words can't even express the gratitude.

[DanRuta]

The hifigan models are the xxxxx.hg.pt files, where the xxxx is the voice ID. Most if not all new voices since about v1.1 of the app have been trained and released together with this additional file (on nexus, the description will show "Model: FastPitch1.1+HiFi"). When using the app, the default behaviour (can be changed in settings) is to use hifigan when the voice is loaded if available, though you can change the vocoder in the "Vocoder" dropdown (the lightning bolt next to it shows that it's available).

[datatypevoid]

Thank you kindly. I will try to route any other questions through discord to prevent cluttering github with non-issues.