open-mmlab / Amphion

Amphion (/æmˈfaɪən/) is a toolkit for Audio, Music, and Speech Generation. Its purpose is to support reproducible research and help junior researchers and engineers get started in the field of audio, music, and speech generation research and development.
https://openhlt.github.io/amphion/
MIT License
4.45k stars 379 forks source link

[BUG]: G2P module fails to initialize when using `pypinyin` as backend #138

Closed yuantuo666 closed 6 months ago

yuantuo666 commented 6 months ago

Describe the bug

The G2P module fails to initialize due to using an unassigned variable.

How To Reproduce

Steps to reproduce the behavior:

  1. Config/File changes: using valle model and editing phone_extractor to pypinyin
  2. Run command: sh egs/tts/VALLE/run.sh --stage 1
  3. See error:
    Traceback (most recent call last):
    File "/app/bins/tts/preprocess.py", line 244, in <module>
    main()
    File "/app/bins/tts/preprocess.py", line 240, in main
    preprocess(cfg, args)
    File "/app/bins/tts/preprocess.py", line 226, in preprocess
    extract_phonme_sequences(dataset, output_path, cfg, dataset_types)
    File "/app/bins/tts/preprocess.py", line 91, in extract_phonme_sequences
    phone_extractor.extract_utt_phone_sequence(cfg, metadata)
    File "/app/processors/phone_extractor.py", line 114, in extract_utt_phone_sequence
    phone_extractor = phoneExtractor(cfg, dataset_name)
    File "/app/processors/phone_extractor.py", line 44, in __init__
    self.g2p_module = G2PModule(backend=cfg.preprocess.phone_extractor)
    File "/app/text/g2p_module.py", line 113, in __init__
    self.backend = self._initialize_backend(
    File "/app/text/g2p_module.py", line 150, in _initialize_backend
    punctuation_marks=punctuation_marks + self.separator.word,
    AttributeError: 'G2PModule' object has no attribute 'separator'

Expected behavior

The error should not occur.

Additional context

https://github.com/open-mmlab/Amphion/blob/main/text/g2p_module.py

class G2PModule:
    """Phonemize Text."""

    # We support espeak to extract IPA (International Phonetic Alphabet), which supports 100 languages,
    # https://github.com/espeak-ng/espeak-ng/blob/master/docs/languages.md

    def __init__(
        self,
        language="en-us",
        backend="espeak",
        separator=Separator(word="_", syllable="-", phone="|"),
        preserve_punctuation=True,
        punctuation_marks: Union[str, Pattern] = Punctuation.default_marks(),
        with_stress: bool = False,
        tie: Union[bool, str] = False,
        language_switch: LanguageSwitch = "keep-flags",
        words_mismatch: WordMismatch = "ignore",
    ) -> None:
        self.backend = self._initialize_backend(
            backend,
            language,
            punctuation_marks,
            preserve_punctuation,
            with_stress,
            tie,
            language_switch,
            words_mismatch,
        )
        self.separator = separator # variable self.separator is assigned after calling _initialize_backend()

    def _initialize_backend(
        self,
        backend,
        language,
        punctuation_marks,
        preserve_punctuation,
        with_stress,
        tie,
        language_switch,
        words_mismatch,
    ):
        if backend == "espeak":
            return EspeakBackend(
                language,
                punctuation_marks=punctuation_marks,
                preserve_punctuation=preserve_punctuation,
                with_stress=with_stress,
                tie=tie,
                language_switch=language_switch,
                words_mismatch=words_mismatch,
            )
        elif backend in ["pypinyin", "pypinyin_initials_finals"]:
            if language != "cmn":
                raise ValueError(
                    f"{language} is not supported for pypinyin and pypinyin_initials_finals."
                )
            return PypinyinBackend(
                backend=backend,
                punctuation_marks=punctuation_marks + self.separator.word, # variable self.separator is used here before assign
            )
        else:
            raise NotImplementedError(f"{backend}")

Variable self.separator is used in _initialize_backend() before assigned in init()

Possible Solution

Move self.separator = separator in front of self.backend = self._initialize_backend(...)