No module named 'g2pNp2g_simple'

[hieu28022000]

Khi thực hiện run repos, tôi đã gặp một lỗi No module named 'g2pNp2g_simple' Tôi phải làm như thế nào để giải quyết vấn đề này đây.

[dangvansam]

Khi thực hiện run repos, tôi đã gặp một lỗi No module named 'g2pNp2g_simple' Tôi phải làm như thế nào để giải quyết vấn đề này đây.

bạn comment bỏ dòng đó đi. k cần import nữa là được nhé

[hieu28022000]

Tôi đã gặp một vắn đề khác ModuleNotFoundError: BeamSearchDecoderWithLM requires the installation of ctc_decoders from nemo/scripts/install_decoders.py. Tôi phát hiện ra rằng không có đường dẫn nào như vậy. Tôi nghĩ nó phải là nemo/scripts/install_decoders.sh Tôi đã cố gắng cài đặt nó bằng cách bash nemo/scripts/install_decoders.sh hoặc sh nemo/scripts/install_decoders.sh. Tuy nhiên tôi đã không thể khắc phục được vấn đề đó, tôi vẫn gặp lỗi ModuleNotFoundError: BeamSearchDecoderWithLM requires the installation of ctc_decoders from nemo/scripts/install_decoders.py Tôi phải làm như thế nào đây.

[detrin]

I am not able to understand what you are talking about, but I got the same error messages. I spent about 7 hours of solving that issue and preparing dockerfile where I can run the model. I will share it after it is working perfectly.

@dangvansam98 Btw, many thanks for the model. If this is a demo are you thinking of making version 2 of it? Also if you have any suggestions for different speech2text or speech2vec models for VN I will be grateful! I would like to stay in contact, but I am afraid my background is stronger in ML rather than NLP.

[detrin]

Oh god, I built KenLM in docker about 15 times, it was painful, but I am finally able to set the environment for your model properly in docker. Now, I am preparing Dockerfile with https://github.com/PaddlePaddle/DeepSpeech as you have in your repo, looks like NeMo used this in scripts/install_decoders.sh as some kind of temporal solution, but I didn't know that until I found https://gitlab.nrp-nautilus.io/ar-noc/nemo/-/blob/master/scripts/install_decoders.sh, which is NeMo at v0.9.0. You have NeMo v0.10.0 in repo.

Now I have options

1. create setup with NeMo v0.10.0 as it is in your local setup
2. to try to setup it with NeMo latest, because I saw this notebook https://github.com/NVIDIA/NeMo/blob/main/tutorials/asr/Offline_ASR.ipynb where they use https://github.com/NVIDIA/NeMo/blob/main/scripts/asr_language_modeling/ngram_lm/install_beamsearch_decoders.sh to install ctc_decoders

I will go for now with the updated version of NeMo, because there is probably no reason to do otherwise.

[detrin]

I am unable to create the exact same environment as you created in nemo folder, could you provide versions of python packages you have installed in connection with this folder?

[detrin]

This issue helped me to resolve _swig_decoders missing error from ctc_decoders_test.py from a newer version of NeMo (v1.5.3 to this date). The issue happens to be also, that while running python setup.py it doesn't install ctc_decoders globally. Copying decoders/build/lib.linux-x86_64-3.8/_swig_decoders.cpython-38-x86_64-linux-gnu.so into decoders folder solved this issue and while installing in docer pip install . has to be used rather than python setup.py. https://github.com/PaddlePaddle/PaddleSpeech/issues/421

I think I am going to fork this repo and prepare everything there so that it can be installed locally and in docker.

[detrin]

Alright, this is Dockerfile for running this model on CPU. You can use it in order to make some kind of Installation.md instructions.

FROM ubuntu:20.04

# https://github.com/anibali/docker-torch/blob/master/no-cuda/Dockerfile
# Use Tini as the init process with PID 1
ADD https://github.com/krallin/tini/releases/download/v0.10.0/tini /tini
RUN chmod +x /tini
ENTRYPOINT ["/tini", "--"]

RUN apt-get update 

# Install dependencies for OpenBLAS, and Torch
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y \
    build-essential git gfortran \
    # python3.8 as default on Ubuntu 20.04
    python3-numpy python3-nose python3-pandas \
    python3 python3-setuptools python3-dev \
    python3-h5py \
    pep8 python3-pip python3-wheel \
    python3-sphinx \
    # for python libs
    curl wget unzip libreadline-dev libjpeg-dev libpng-dev ncurses-dev \
    imagemagick gnuplot gnuplot-x11 libssl-dev libzmq3-dev graphviz \
    # OpenBLAS
    swig libopenblas-base \
    # HDF5
    libhdf5-dev \
    # cmake
    build-essential cmake libboost-system-dev libboost-thread-dev \
    libboost-program-options-dev libboost-test-dev libeigen3-dev \
    zlib1g-dev libbz2-dev liblzma-dev libboost-all-dev && \
    pip install --upgrade pip && \
    pip install --upgrade setuptools 

# Clone demo_vietasr, NeMo and libs for installing ctc_decoders
WORKDIR /home/root/speech2text
RUN git clone https://github.com/NVIDIA/NeMo.git --branch r1.5.0 
RUN git clone https://github.com/dangvansam98/demo_vietasr.git

# Create simlink
RUN ln -s /usr/bin/python3.8 /usr/bin/python

# Installation of ctc_decoders
WORKDIR /home/root/speech2text/NeMo/scripts/asr_language_modeling/ngram_lm/
# https://github.com/NVIDIA/NeMo/blob/main/scripts/asr_language_modeling/ngram_lm/install_beamsearch_decoders.sh
# libraries are installed above
RUN git clone https://github.com/NVIDIA/OpenSeq2Seq -b ctc-decoders
RUN mv OpenSeq2Seq/decoders .
RUN rm -rf OpenSeq2Seq

# copy of install_beamsearch_decoders.sh
WORKDIR /home/root/speech2text/NeMo/scripts/asr_language_modeling/ngram_lm/decoders
RUN git clone https://github.com/kpu/kenlm.git 
RUN wget http://www.openfst.org/twiki/pub/FST/FstDownload/openfst-1.6.3.tar.gz
RUN tar -xzvf openfst-1.6.3.tar.gz
RUN git clone https://github.com/progschj/ThreadPool.git

WORKDIR /home/root/speech2text/NeMo/scripts/asr_language_modeling/ngram_lm/decoders/kenlm
RUN mkdir build
WORKDIR /home/root/speech2text/NeMo/scripts/asr_language_modeling/ngram_lm/decoders/kenlm/build
RUN cmake ..
RUN make -j

WORKDIR /home/root/speech2text/NeMo/scripts/asr_language_modeling/ngram_lm/decoders/
# python setup.py doesn't work properly in docker
RUN pip install .
# _swig_decoders missing error while importing ctc_decoders
RUN cp build/lib.linux-x86_64-3.8/_swig_decoders.cpython-38-x86_64-linux-gnu.so .
RUN python ctc_decoders_test.py

WORKDIR /home/root/speech2text/demo_vietasr
# seems like it works with v1.5.1, also it installs all needed packages
# but we still have to use the locally cloned repo in demo_vietasr
RUN pip install nemo_toolkit[all]==1.5.1
RUN sed -i "s/placement=nemo.core.DeviceType.GPU,/placement=nemo.core.DeviceType.CPU,/g" infer.py
RUN sed -i "s/lm_path=\"NeMo/lm_path=\"nemo/g" infer.py

[detrin]

I am getting this error while trying to use latest stable NeMo

Traceback (most recent call last):
  File "model_test.py", line 3, in <module>
    from infer import restore_model, load_audio
  File "/home/root/speech2text/demo_vietasr/infer.py", line 7, in <module>
    from nemo.backends.pytorch.nm import DataLayerNM
ModuleNotFoundError: No module named 'nemo.backends'

I found out that there are no backends after v1.0.0 of NeMo and no DataLayerNM class, it was probably some kind of temporal solution. Probably it would be better to just stick to your clone of NeMo in the repository.

Also, I found out that apex can be cloned and installed locally as you probably did yourself. Also, when left in the repo, it just works.

[dangvansam]

I am getting this error while trying to use latest stable NeMo

Traceback (most recent call last):
  File "model_test.py", line 3, in <module>
    from infer import restore_model, load_audio
  File "/home/root/speech2text/demo_vietasr/infer.py", line 7, in <module>
    from nemo.backends.pytorch.nm import DataLayerNM
ModuleNotFoundError: No module named 'nemo.backends'

I found out that there are no backends after v1.0.0 of NeMo and no DataLayerNM class, it was probably some kind of temporal solution. Probably it would be better to just stick to your clone of NeMo in the repository.

Also, I found out that apex can be cloned and installed locally as you probably did yourself. Also, when left in the repo, it just works.

I apologize for the issue you were having; I haven't had time to update the repository I built for a very long time. I've currently upgraded to the most recent version, which fixes the issues that users had with installation and testing.