search

jzhang38 / TinyLlama

The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.
Apache License 2.0
7.7k stars 453 forks source link

Resolve circular dependency and import issues #170

Closed keeeeenw closed 6 months ago

keeeeenw commented 6 months ago

Resolved these two issues while preparing for pretraining datasets.

Traceback (most recent call last):
  File "/home/user/workspace/TinyLlama/scripts/prepare_redpajama.py", line 14, in <module>
    import lit_gpt.packed_dataset as packed_dataset
  File "/home/user/workspace/TinyLlama/lit_gpt/__init__.py", line 2, in <module>
    from lit_gpt.config import Config
  File "/home/user/workspace/TinyLlama/lit_gpt/config.py", line 7, in <module>
    import lit_gpt.model
  File "/home/user/workspace/TinyLlama/lit_gpt/model.py", line 14, in <module>
    from lit_gpt.config import Config
ImportError: cannot import name 'Config' from partially initialized module 'lit_gpt.config' (most likely due to a circular import) (/home/user
/workspace/TinyLlama/lit_gpt/config.py)
Traceback (most recent call last):
  File "/home/user/workspace/TinyLlama/scripts/prepare_redpajama.py", line 15, in <module>
    from lit_gpt.config import Config, Tokenizer
ImportError: cannot import name 'Tokenizer' from 'lit_gpt.config' (/home/user/workspace/TinyLlama/lit_gpt/config.py)