search

openhackathons-org / End-to-End-LLM

This repository is an AI Bootcamp material that consist of a workflow for LLM
Apache License 2.0
68 stars 33 forks source link

Issue: Nemo_primer.ipynb imports not working. #21

Closed pablodonisebri closed 4 months ago

pablodonisebri commented 9 months ago

In Nemo_primer.ipynb when doing import nemo.collections.asr as nemo_asr, import nemo.collections.nlp as nemo_nlp and import nemo.collections.tts as nemo_tts I get the following error ImportError: tokenizers>=0.11.1,!=0.11.3,<0.14 is required for a normal functioning of this module, but found tokenizers==0.15.2.

If I try to solve it by doing pip install tokenizers==0.13.1 I get this other error File /usr/local/lib/python3.10/dist-packages/pytorch_lightning/_graveyard/utilities.py:25 17 def _get_gpu_memory_map() -> None: 18 # TODO: Remove in v2.0.0 19 raise RuntimeError( 20 "pytorch_lightning.utilities.memory.get_gpu_memory_map was deprecated in v1.5 and is no longer supported" 21 " as of v1.9. Use pytorch_lightning.accelerators.cuda.get_nvidia_gpu_stats instead." 22 ) ---> 25 pl.utilities.memory.get_gpu_memory_map = _get_gpu_memory_map

AttributeError: partially initialized module 'pytorch_lightning' has no attribute 'utilities' (most likely due to a circular import)

It might be helpful to specify the desired package versions in the pip install inside the Dockerfile_nemo because it might be that doing RUN pip install lightning RUN pip install megatron.core RUN pip install --upgrade nemoguardrails RUN pip install openai RUN pip install ujson RUN pip install --upgrade --no-cache-dir gdown is installing new and uncompatible versions of the libraries (I mean uncompatibles with the tutorials showed in the notebooks).

programmah commented 8 months ago

@pablodonisebri Thank you for your observations and recommendations. Resolving the issue will be prioritize in our next update.

programmah commented 4 months ago

The issues is addressed but lab is now deprecated.