[ImportError] hydra_train.py 실행 시, openspeech.decoders의 circular import문제

[jskwak98]

❓ Questions & Help

Details

안녕하세요 Openspeech를 처음 사용해보는 학생입니다. Custom Dataset으로 학습을 해보고자 Issue와 Langcon 영상을 참고해서, preprocess와 tokenizer 부분을 따로 구현한 뒤, 가이드를 참고해 hydra_train.py를 실행했더니, 제가 추가한 code와 관련 없는 initialization 부분에서 ImportError가 나서 질문을 남깁니다.

관련된 다른 issue가 없어, 제 미숙함으로 인해 발생한 오류일 수 있으리라 생각해, 위 상황과 별개로 처음부터 오류를 재현해보았습니다.

1. conda로 openspeech라는 이름의 가상환경을 만들었습니다.
2. pip install openspeech-core로 관련 package를 설치했습니다.
3. git clone을 통해 repository를 clone하였습니다.
4. ./install.sh를 실행해 설치를 완료했습니다.
5. 이후 README.md의 example을 그대로 command line에 실행합니다.(PATH 관련된 코드 실행 이전에 오류가 발생해서 그대로 실행했습니다.)

그러면 아래와 같은 error가 발생합니다.

Traceback (most recent call last): File "./openspeech_cli/hydra_train.py", line 30, in from openspeech.dataclass.initialize import hydra_train_init File "/workspace/openspeech/openspeech/init.py", line 27, in import openspeech.decoders File "/workspace/openspeech/openspeech/decoders/init.py", line 23, in from .lstm_attention_decoder import LSTMAttentionDecoder File "/workspace/openspeech/openspeech/decoders/lstm_attention_decoder.py", line 29, in from openspeech.decoders import OpenspeechDecoder ImportError: cannot import name 'OpenspeechDecoder' from partially initialized module 'openspeech.decoders' (most likely due to a circular import) (/workspace/openspeech/openspeech/decoders/init.py)

openspeech/openspeech/init.py에서 import openspeech.decoders를 실행하고,

openspeech/openspeech/decoders/lstm_attention_decoder.py가 from openspeech.decoders import OpenspeechDecoder를 실행하며 순환참조가 일어나 발생하는 오류로 추정하는데, 혹시 제가 무언가 잘못했다면 어떤 부분을 잘못했는지, 고쳐야한다면 어떤 부분을 고쳐야 할지 궁금합니다.

감사합니다.

[ccyang123]

Hello, jskwak98,

You may try to modify "/workspace/openspeech/openspeech/decoders/lstm_attention_decoder.py", line 29 as

from .openspeech_decoder import OpenspeechDecoder

from openspeech.decoders import OpenspeechDecoder

[jskwak98]

Hello, jskwak98,

You may try to modify "/workspace/openspeech/openspeech/decoders/lstm_attention_decoder.py", line 29 as

from .openspeech_decoder import OpenspeechDecoder #from openspeech.decoders import OpenspeechDecoder

+) Edit Oh I misunderstood your comment how stupid I am. That actually worked. Thank you so much.

I eventually changed all init.py as akthddus22 suggested thank you for your help

Previous comment

Hello ccyang123,

Thank you for your sincere reply.

I've actually tried that before posting this issue, but since lstm_attention_decoder.py actually imports OpenspeechDecoder class as the parent class of LSTMAttentionDecoder(at line 40) NameError: name 'OpenspeechDecoder' is not defined occurs.

Again, thank you for your help on my issue.

[akthddus22]

Im not sure it will be helpful for you. But in my case, changing order of some imports solved the same problem

1. /models/init.py import importlib import os

from .openspeech_model import OpenspeechModel from .openspeech_ctc_model import OpenspeechCTCModel from .openspeech_encoder_decoder_model import OpenspeechEncoderDecoderModel from .openspeech_transducer_model import OpenspeechTransducerModel

2. /models/openspeech_model.py import pytorch_lightning as pl import torch import torch.nn as nn from omegaconf import DictConfig from torch import Tensor from torch.optim import ASGD, SGD, Adadelta, Adagrad, Adam, Adamax, AdamW

from openspeech.criterion import CRITERION_REGISTRY from openspeech.metrics import CharacterErrorRate, WordErrorRate from openspeech.optim.adamp import AdamP from openspeech.optim.novograd import Novograd from openspeech.optim.radam import RAdam from openspeech.optim.scheduler import SCHEDULER_REGISTRY from openspeech.tokenizers.tokenizer import Tokenizer

3. encoders/init.py from .conformer_encoder import ConformerEncoder from .contextnet_encoder import ContextNetEncoder from .openspeech_encoder import OpenspeechEncoder from .convolutional_lstm_encoder import ConvolutionalLSTMEncoder from .convolutional_transformer_encoder import ConvolutionalTransformerEncoder from .jasper import Jasper from .lstm_encoder import LSTMEncoder from .rnn_transducer_encoder import RNNTransducerEncoder from .squeezeformer_encoder import SqueezeformerEncoder from .transformer_encoder import TransformerEncoder from .transformer_transducer_encoder import TransformerTransducerEncoder

4. decoders/init.py from .openspeech_decoder import OpenspeechDecoder from .lstm_attention_decoder import LSTMAttentionDecoder from .rnn_transducer_decoder import RNNTransducerDecoder from .transformer_decoder import TransformerDecoder from .transformer_transducer_decoder import TransformerTransducerDecoder

i hope it works for you too

[parkmy123]

from openspeech.decoders import OpenspeechDecoder를

from openspeech.decoders.openspeech_decoder import OpenspeechDecoder로 수정해도 안되나요?

[jskwak98]

Im not sure it will be helpful for you. But in my case, changing order of some imports solved the same problem

1. /models/init.py import importlib import os

from .openspeech_model import OpenspeechModel from .openspeech_ctc_model import OpenspeechCTCModel from .openspeech_encoder_decoder_model import OpenspeechEncoderDecoderModel from .openspeech_transducer_model import OpenspeechTransducerModel

2. /models/openspeech_model.py import pytorch_lightning as pl import torch import torch.nn as nn from omegaconf import DictConfig from torch import Tensor from torch.optim import ASGD, SGD, Adadelta, Adagrad, Adam, Adamax, AdamW

from openspeech.criterion import CRITERION_REGISTRY from openspeech.metrics import CharacterErrorRate, WordErrorRate from openspeech.optim.adamp import AdamP from openspeech.optim.novograd import Novograd from openspeech.optim.radam import RAdam from openspeech.optim.scheduler import SCHEDULER_REGISTRY from openspeech.tokenizers.tokenizer import Tokenizer

3. encoders/init.py from .conformer_encoder import ConformerEncoder from .contextnet_encoder import ContextNetEncoder from .openspeech_encoder import OpenspeechEncoder from .convolutional_lstm_encoder import ConvolutionalLSTMEncoder from .convolutional_transformer_encoder import ConvolutionalTransformerEncoder from .jasper import Jasper from .lstm_encoder import LSTMEncoder from .rnn_transducer_encoder import RNNTransducerEncoder from .squeezeformer_encoder import SqueezeformerEncoder from .transformer_encoder import TransformerEncoder from .transformer_transducer_encoder import TransformerTransducerEncoder
4. decoders/init.py from .openspeech_decoder import OpenspeechDecoder from .lstm_attention_decoder import LSTMAttentionDecoder from .rnn_transducer_decoder import RNNTransducerDecoder from .transformer_decoder import TransformerDecoder from .transformer_transducer_decoder import TransformerTransducerDecoder

i hope it works for you too

It worked like a charm, I should've more closely investigate the import hierarchy. Thank you so much.

[jskwak98]

from openspeech.decoders import OpenspeechDecoder를

from openspeech.decoders.openspeech_decoder import OpenspeechDecoder로 수정해도 안되나요?

좋은 comment를 잘못 알아들었네요 말씀대로 이런 순서로 오류 나는 곳을 고치면 (Decoder -> Encoder -> ... ) 되는 것 같습니다. 일단은 @akthddus22 님께서 말씀해주신대로 했습니다. 도와주셔서 너무 감사합니다!