Nemo_toolkit 2.0.0.rc0 installation failure

[Subbu-ssr]

I have tried to install nemo_toolkit==2.0.0.rc0 but it shows error:

RuntimeError: causal_conv1d is only supported on CUDA 11.6 and above. Note: make sure nvcc has a supported version by running nvcc -V.

  torch.__version__  = 2.3.0+cu121

  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed

× Encountered error while generating package metadata. ╰─> See above for output.

note: This is an issue with the package mentioned above, not pip. hint: See above for details.

can anyone help me to resolve this issue??

[titu1994]

Thanks for the information, we're taking a look at whether we can make it an optional dependency.

[ericharper]

I just tried installing in a clean conda env and it worked for me:

conda create env --name test_r2.0.0rc0 python==3.10.12
conda activate test_r2.0.0rc0
conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia
pip install Cython
pip install packaging
pip install --pre nemo_toolkit[all]

Could you try it?

Or if you use the container nvcr.io/nvidia/pytorch:24.02-py3 just

pip install --pre nemo_toolkit[all]

should work.

[Subbu-ssr]

Thank you so much for your help

On Fri, 7 Jun, 2024, 11:31 Eric Harper, @.***> wrote:

I just tried installing in a clean conda env and it worked for me:

conda create env --name test_r2.0.0rc0 python==3.10.12 conda activate test_r2.0.0rc0 conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia pip install Cython pip install packaging pip install --pre nemo_toolkit[all]

Could you try it?

Or if you use the container nvcr.io/nvidia/pytorch:24.02-py3 just

pip install --pre nemo_toolkit[all]

should work.

— Reply to this email directly, view it on GitHub https://github.com/NVIDIA/NeMo/issues/9397#issuecomment-2154140932, or unsubscribe https://github.com/notifications/unsubscribe-auth/BI7V4KOCJ4NR4FU2HARAF43ZGFEFHAVCNFSM6AAAAABI4MMH36VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNJUGE2DAOJTGI . You are receiving this because you authored the thread.Message ID: @.***>

-- The information in this message may be proprietary and/or confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, please notify ATMECS and delete it from your computer.

[ericharper]

Please let us know if this resolves your issue.

[Subbu-ssr]

ollecting nemo_toolkit[all] Downloading https://files.pythonhosted.org/packages/cd/94/1031a231453146b5ec302a6c2282dc90c2e18844deeee2ff867900b90368/nemo_toolkit-2.0.0rc0-py3-none-any.whl (3.6MB) 100% |████████████████████████████████| 3.6MB 5.4MB/s eta 0:00:01 Collecting ruamel.yaml (from nemo_toolkit[all]) Downloading https://files.pythonhosted.org/packages/4b/5d/678d7d15071816cb9a5e015fcd090ddc59a6a195ce8965363313e2044595/ruamel.yaml-0.18.3-py3-none-any.whl (114kB) 100% |████████████████████████████████| 122kB 12.2MB/s ta 0:00:01 Collecting numpy>=1.22 (from nemo_toolkit[all]) Could not find a version that satisfies the requirement numpy>=1.22 (from nemo_toolkit[all]) (from versions: 1.3.0, 1.4.1, 1.5.0, 1.5.1, 1.6.0, 1.6.1, 1.6.2, 1.7.0, 1.7.1, 1.7.2, 1.8.0, 1.8.1, 1.8.2, 1.9.0, 1.9.1, 1.9.2, 1.9.3, 1.10.0.post2, 1.10.1, 1.10.2, 1.10.4, 1.11.0, 1.11.1, 1.11.2, 1.11.3, 1.12.0, 1.12.1, 1.13.0, 1.13.1, 1.13.3, 1.14.0, 1.14.1, 1.14.2, 1.14.3, 1.14.4, 1.14.5, 1.14.6, 1.15.0, 1.15.1, 1.15.2, 1.15.3, 1.15.4, 1.16.0, 1.16.1, 1.16.2, 1.16.3, 1.16.4, 1.16.5, 1.16.6, 1.17.0, 1.17.1, 1.17.2, 1.17.3, 1.17.4, 1.17.5, 1.18.0, 1.18.1, 1.18.2, 1.18.3, 1.18.4, 1.18.5, 1.19.0, 1.19.1, 1.19.2, 1.19.3, 1.19.4, 1.19.5) No matching distribution found for numpy>=1.22 (from nemo_toolkit[all] This is the error i got now after runnig the command The numpy i installed in my system in 1.11.3 version . can you please help me to resolve this issue

[Subbu-ssr]

i tried installing nemo-toolkit with the command pip install nemo-toolkit[all]==2.0.0rc0. It shows the nemo version and only asr collections and tts collections are importing,nlp collections are not importing. This is the error i got ImportError Traceback (most recent call last) /tmp/ipykernel_53908/447078856.py in ----> 1 import nemo.collections.nlp as nemo_nlp

~/.local/lib/python3.10/site-packages/nemo/collections/nlp/init.py in 13 # limitations under the License. 14 ---> 15 from nemo.collections.nlp import data, losses, models, modules 16 from nemo.package_info import version 17

~/.local/lib/python3.10/site-packages/nemo/collections/nlp/data/init.py in 40 BertTokenClassificationInferDataset, 41 ) ---> 42 from nemo.collections.nlp.data.zero_shot_intent_recognition.zero_shot_intent_dataset import ( 43 ZeroShotIntentDataset, 44 ZeroShotIntentInferenceDataset,

~/.local/lib/python3.10/site-packages/nemo/collections/nlp/data/zero_shot_intent_recognition/init.py in 14 15 ---> 16 from nemo.collections.nlp.data.zero_shot_intent_recognition.zero_shot_intent_dataset import ( 17 ZeroShotIntentInferenceDataset, 18 calc_class_weights_from_dataloader,

~/.local/lib/python3.10/site-packages/nemo/collections/nlp/data/zero_shot_intent_recognition/zero_shot_intent_dataset.py in 28 from nemo.collections.nlp.data.glue_benchmark.data_processors import InputExample 29 from nemo.collections.nlp.data.glue_benchmark.glue_benchmark_dataset import GLUEDataset ---> 30 from nemo.collections.nlp.parts.utils_funcs import tensor2list 31 from nemo.core.neural_types import CategoricalValuesType, ChannelType, MaskType, NeuralType 32 from nemo.utils import logging

~/.local/lib/python3.10/site-packages/nemo/collections/nlp/parts/init.py in 15 16 from nemo.collections.nlp.parts.megatron_lr_schedulers import CosineAnnealingExp ---> 17 from nemo.collections.nlp.parts.utils_funcs import list2str, tensor2list

~/.local/lib/python3.10/site-packages/nemo/collections/nlp/parts/utils_funcs.py in 35 from torch import Tensor 36 ---> 37 from nemo.collections.nlp.modules.common.megatron.utils import erf_gelu 38 from nemo.collections.nlp.modules.common.megatron.utils import openai_gelu as openai_gelu_func 39 from nemo.collections.nlp.modules.common.megatron.utils import squared_relu

~/.local/lib/python3.10/site-packages/nemo/collections/nlp/modules/init.py in 14 15 ---> 16 from nemo.collections.nlp.modules.common import ( 17 AlbertEncoder, 18 BertEncoder,

~/.local/lib/python3.10/site-packages/nemo/collections/nlp/modules/common/init.py in 16 17 from nemo.collections.nlp.modules.common.bert_module import BertModule ---> 18 from nemo.collections.nlp.modules.common.huggingface import ( 19 AlbertEncoder, 20 BertEncoder,

~/.local/lib/python3.10/site-packages/nemo/collections/nlp/modules/common/huggingface/init.py in 17 from nemo.collections.nlp.modules.common.huggingface.camembert import CamembertEncoder 18 from nemo.collections.nlp.modules.common.huggingface.distilbert import DistilBertEncoder ---> 19 from nemo.collections.nlp.modules.common.huggingface.huggingface_utils import ( 20 get_huggingface_lm_model, 21 get_huggingface_pretrained_lm_models_list,

~/.local/lib/python3.10/site-packages/nemo/collections/nlp/modules/common/huggingface/huggingface_utils.py in 16 from typing import List, Optional 17 ---> 18 from transformers import ( 19 ALBERT_PRETRAINED_MODEL_ARCHIVE_LIST, 20 BERT_PRETRAINED_MODEL_ARCHIVE_LIST,

ImportError: cannot import name 'ALBERT_PRETRAINED_MODEL_ARCHIVE_LIST' from 'transformers' (/home/subramanyam/.local/lib/python3.10/site-packages/transformers/init.py)

I upgrade transformers version to 4.41.2 but it does not importing the nlp collections of nemo-toolkit. Please help me to resolve this issue also

[titu1994]

You need to reduce your HF transformers version, not increase it. 4.40.0 is the supported version for 2.0 rc0

[Subbu-ssr]

ok i will try and install it

[Subbu-ssr]

ImportError: cannot import name 'packaging' from 'pkg_resources' (/home/subramanyam/.local/lib/python3.10/site-packages/pkg_resources/init.py) now i got this error i also updated the package version into 24.1 and setup tools version but iam getting the same error

[titu1994]

Please see the readme - you need to install Cython and packaging before installing Nemo

[titu1994]

Please do not update random libraries, we do not support setup tools 70.x, you'll need to downgrade that first to 69 or lower

[Subbu-ssr]

ok i already installed cython and packaging before installing the nemo toolkit but if i run the command import nemo.collections.nlp as nlp it shows this error ImportError: cannot import name 'packaging' from 'pkg_resources' (/home/subramanyam/.local/lib/python3.10/site-packages/pkg_resources/init.py) iam using jupyter notebook