Colab notebook: Where are weights?

drscotthawley commented 2 years ago

Thanks for sharing the notebook. Could it perhaps be documented a bit more to make it a bit easier for new users (like me) to make it work without crashing? I can't figure out how to fill in the missing information about where to find the weights.

Below is a log of my run...

!nvidia-smi -L
GPU 0: Tesla P100-PCIE-16GB (UUID: GPU-5218d88a-592a-b7c2-d10c-ff61031ab247)

Mount your drive

Mounted at /content/drive

Install Word2Wave, import necessary packages

Cloning into 'word2wave'...
remote: Enumerating objects: 349, done.
remote: Counting objects: 100% (349/349), done.
remote: Compressing objects: 100% (311/311), done.
remote: Total 349 (delta 185), reused 81 (delta 33), pack-reused 0
Receiving objects: 100% (349/349), 1.10 MiB | 5.21 MiB/s, done.
Resolving deltas: 100% (185/185), done.
/content/word2wave
Requirement already satisfied: matplotlib>=2.2.4 in /usr/local/lib/python3.7/dist-packages (from -r requirements.txt (line 1)) (3.2.2)
Requirement already satisfied: numpy>=1.16.3 in /usr/local/lib/python3.7/dist-packages (from -r requirements.txt (line 2)) (1.19.5)
Collecting librosa==0.6.3
  Downloading librosa-0.6.3.tar.gz (1.6 MB)
     |████████████████████████████████| 1.6 MB 5.0 MB/s 
Collecting pescador>=2.0.1
  Downloading pescador-2.1.0.tar.gz (20 kB)
Requirement already satisfied: torch>=1.1.0 in /usr/local/lib/python3.7/dist-packages (from -r requirements.txt (line 5)) (1.10.0+cu111)
Requirement already satisfied: tqdm>=4.32.1 in /usr/local/lib/python3.7/dist-packages (from -r requirements.txt (line 6)) (4.62.3)
Collecting numba==0.49.0
  Downloading numba-0.49.0-cp37-cp37m-manylinux2014_x86_64.whl (3.6 MB)
     |████████████████████████████████| 3.6 MB 36.1 MB/s 
Collecting torchaudio==0.8.1
  Downloading torchaudio-0.8.1-cp37-cp37m-manylinux1_x86_64.whl (1.9 MB)
     |████████████████████████████████| 1.9 MB 64.3 MB/s 
Requirement already satisfied: audioread>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from librosa==0.6.3->-r requirements.txt (line 3)) (2.1.9)
Requirement already satisfied: scipy>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from librosa==0.6.3->-r requirements.txt (line 3)) (1.4.1)
Requirement already satisfied: scikit-learn!=0.19.0,>=0.14.0 in /usr/local/lib/python3.7/dist-packages (from librosa==0.6.3->-r requirements.txt (line 3)) (1.0.1)
Requirement already satisfied: joblib>=0.12 in /usr/local/lib/python3.7/dist-packages (from librosa==0.6.3->-r requirements.txt (line 3)) (1.1.0)
Requirement already satisfied: decorator>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from librosa==0.6.3->-r requirements.txt (line 3)) (4.4.2)
Requirement already satisfied: six>=1.3 in /usr/local/lib/python3.7/dist-packages (from librosa==0.6.3->-r requirements.txt (line 3)) (1.15.0)
Requirement already satisfied: resampy>=0.2.0 in /usr/local/lib/python3.7/dist-packages (from librosa==0.6.3->-r requirements.txt (line 3)) (0.2.2)
Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from numba==0.49.0->-r requirements.txt (line 7)) (57.4.0)
Collecting llvmlite<=0.33.0.dev0,>=0.31.0.dev0
  Downloading llvmlite-0.32.1-cp37-cp37m-manylinux1_x86_64.whl (20.2 MB)
     |████████████████████████████████| 20.2 MB 1.3 MB/s 
Collecting torch>=1.1.0
  Downloading torch-1.8.1-cp37-cp37m-manylinux1_x86_64.whl (804.1 MB)
     |████████████████████████████████| 804.1 MB 2.6 kB/s 
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from torch>=1.1.0->-r requirements.txt (line 5)) (3.10.0.2)
Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=2.2.4->-r requirements.txt (line 1)) (2.8.2)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=2.2.4->-r requirements.txt (line 1)) (3.0.6)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=2.2.4->-r requirements.txt (line 1)) (1.3.2)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=2.2.4->-r requirements.txt (line 1)) (0.11.0)
Requirement already satisfied: pyzmq>=15.0 in /usr/local/lib/python3.7/dist-packages (from pescador>=2.0.1->-r requirements.txt (line 4)) (22.3.0)
Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn!=0.19.0,>=0.14.0->librosa==0.6.3->-r requirements.txt (line 3)) (3.0.0)
Building wheels for collected packages: librosa, pescador
  Building wheel for librosa (setup.py) ... done
  Created wheel for librosa: filename=librosa-0.6.3-py3-none-any.whl size=1573336 sha256=fff7ac07e9d03aa008fcf3f1f369f8acffdb27082d75ffead993dfeb62fb468d
  Stored in directory: /root/.cache/pip/wheels/de/c1/94/619fb8b04ee1f567115662d26650677ecf79bc7d8e462d21f8
  Building wheel for pescador (setup.py) ... done
  Created wheel for pescador: filename=pescador-2.1.0-py3-none-any.whl size=21104 sha256=4a7aaeaff3c65a1913ee3bad1cbd83c1c6e541790056b38a2848a31f5568f4e9
  Stored in directory: /root/.cache/pip/wheels/f0/e3/c6/32d30d5eb5292dac352d2fca4ebf393aa94e09b9b8b4b0f341
Successfully built librosa pescador
Installing collected packages: llvmlite, numba, torch, torchaudio, pescador, librosa
  Attempting uninstall: llvmlite
    Found existing installation: llvmlite 0.34.0
    Uninstalling llvmlite-0.34.0:
      Successfully uninstalled llvmlite-0.34.0
  Attempting uninstall: numba
    Found existing installation: numba 0.51.2
    Uninstalling numba-0.51.2:
      Successfully uninstalled numba-0.51.2
  Attempting uninstall: torch
    Found existing installation: torch 1.10.0+cu111
    Uninstalling torch-1.10.0+cu111:
      Successfully uninstalled torch-1.10.0+cu111
  Attempting uninstall: torchaudio
    Found existing installation: torchaudio 0.10.0+cu111
    Uninstalling torchaudio-0.10.0+cu111:
      Successfully uninstalled torchaudio-0.10.0+cu111
  Attempting uninstall: librosa
    Found existing installation: librosa 0.8.1
    Uninstalling librosa-0.8.1:
      Successfully uninstalled librosa-0.8.1
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
torchvision 0.11.1+cu111 requires torch==1.10.0, but you have torch 1.8.1 which is incompatible.
torchtext 0.11.0 requires torch==1.10.0, but you have torch 1.8.1 which is incompatible.
kapre 0.3.6 requires librosa>=0.7.2, but you have librosa 0.6.3 which is incompatible.
Successfully installed librosa-0.6.3 llvmlite-0.32.1 numba-0.49.0 pescador-2.1.0 torch-1.8.1 torchaudio-0.8.1
/usr/local/lib/python3.7/dist-packages/librosa/util/decorators.py:9: NumbaDeprecationWarning: An import was requested from a module that has moved location.
Import requested from: 'numba.decorators', please update to use 'numba.core.decorators' or pin to Numba version 0.48.0. This alias will not be present in Numba version 0.50.0.
  from numba.decorators import jit as optional_jit
/usr/local/lib/python3.7/dist-packages/librosa/util/decorators.py:9: NumbaDeprecationWarning: An import was requested from a module that has moved location.
Import of 'jit' requested from: 'numba.decorators', please update to use 'numba.core.decorators' or pin to Numba version 0.48.0. This alias will not be present in Numba version 0.50.0.
  from numba.decorators import jit as optional_jit

But then part that says...

Copy the pre-trained WaveGAN and COALA weights from drive

drive_path:  "/content/drive/<path/to/word2wave_files/>"

...it's not clear from the notebook what to enter in this string.

I see further up in the output that it seems to have installed into /content/word2wave, but when I type that into the prompt, I get

cp: cannot stat '/content/word2wavewavegan': No such file or directory
cp: cannot stat '/content/word2wavecoala': No such file or directory
mv: cannot stat '/content/word2wave/coala/coala/': No such file or directory

Looking around on Drive, I see...

!ls /content/drive

MyDrive  Shareddrives

If I just ignore the above errors and try to run the notebook with no changes, then when it comes to the part for generating with "firework", I see:

NameError                                 Traceback (most recent call last)
<ipython-input-9-cf00c4445550> in <module>()
      9 id2tag = json.load(open('/content/word2wave/coala/id2token_top_1000.json', 'rb'))
     10 
---> 11 check_text_input(text)

<ipython-input-7-0cff9a5913d2> in check_text_input(text)
     28 
     29 def check_text_input(text):
---> 30   _, words_in_dict, words_not_in_dict = word2wave.tokenize_text(text)
     31   if not words_in_dict:
     32       raise Exception("All the words in the text prompt are out-of-vocabulary, please try with another prompt")

NameError: name 'word2wave' is not defined

drscotthawley commented 2 years ago

One other issue:

Generate audio from text

Clicking on "laughing" produces:

NameError: name 'widgets' is not defined

This happens because you're using widgets before you import it (which happens in the next cell below). So the line

import ipywidgets as widgets

should be moved upward.

ilaria-manco commented 2 years ago

Hi Scott, thank you for reporting this!

I have updated the notebook and all the above issues should now be solved.

A summary of the changes:

The notebook was previously written assuming users would first train their own WaveGAN models. I realise this would be an unnecessary barrier to playing around with the notebook straight away, so a few of the models I trained can now be found in a pre-release. The notebook now has code that automatically downloads the weights from there and imports them to Colab upon running the cell, without any further input from the user. Custom WaveGAN models can still be imported via Google Drive if the weights are uploaded to a folder and the path copied into the text box in the cell:
```
drive_path = "" #@param {type:"string"}
if drive_path:
!cp -r -v {drive_path} "/content/word2wave/"
else:
!mkdir "/content/word2wave/wavegan/"
%cd /content/word2wave/wavegan/
!wget https://github.com/ilaria-manco/word2wave/releases/download/0.1/gan_fs_loop_3.tar
!wget https://github.com/ilaria-manco/word2wave/releases/download/0.1/gan_fs_loop_4.tar
!wget https://github.com/ilaria-manco/word2wave/releases/download/0.1/gan_drum.tar
%cd /content/word2wave/
```
Changed check_text_input(text) to take an instance of the Word2Wave class as an additional argument. This solves the name 'word2wave' is not defined error
Moved import ipywidgets as widgets to the previous cell

I will add some further documentation to make it more intuitive as soon as I have time, but hopefully the above changes help for now. Let me know if there are any other issues.

drscotthawley commented 2 years ago

Ilaria, thanks for your reply! I ended up really busy this week and haven't yet had a chance to try out your revised notebook. Based on the changes you describe, I'll leave it up to you as to whether you want to keep this Issue open until I get a chance to try the notebook or close this Issue right now anyway. Thanks!

ilaria-manco commented 2 years ago

I'll close it for now, but please feel free to open a new one if you notice any other issue. Thank you!

ilaria-manco / word2wave