Colab notebook has stopped working

[Preetika764]

I've recently started getting this ValueError while running prepare_dataset.sh.

[Preetika764]

This happens when I run with Colab.

[eldarkurtic]

Could you provide more details about the model you are trying to load?

[Preetika764]

I'm using "mistralai/Mistral-7B-Instruct-v0.2" and 4bit model precision. This used to work fine earlier. Now the session crashes after the couple of import statements at top and it restarts to use the newer versions. I tried again and wasn't able to reproduce the above error but I got this right now -

/usr/local/lib/python3.10/dist-packages/langchain_core/_api/deprecation.py:139: LangChainDeprecationWarning: The class HuggingFaceEmbeddings was deprecated in LangChain 0.2.2 and will be removed in 0.3.0. An updated version of the class exists in the langchain-huggingface package and should be used instead. To use it run pip install -U langchain-huggingface and import as from langchain_huggingface import HuggingFaceEmbeddings. warn_deprecated( 2024-07-05 20:17:15.133573: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered 2024-07-05 20:17:15.133638: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered 2024-07-05 20:17:15.135791: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered 2024-07-05 20:17:16.850078: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/transformers/utils/import_utils.py", line 1510, in _get_module return importlib.import_module("." + module_name, self.name) File "/usr/lib/python3.10/importlib/init.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1050, in _gcd_import File "", line 1027, in _find_and_load File "", line 1006, in _find_and_load_unlocked File "", line 688, in _load_unlocked File "", line 883, in exec_module File "", line 241, in _call_with_frames_removed File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 209, in from peft import PeftModel File "/usr/local/lib/python3.10/dist-packages/peft/init.py", line 22, in from .auto import ( File "/usr/local/lib/python3.10/dist-packages/peft/auto.py", line 31, in from .mapping import MODEL_TYPE_TO_PEFT_MODEL_MAPPING File "/usr/local/lib/python3.10/dist-packages/peft/mapping.py", line 23, in from .mixed_model import PeftMixedModel File "/usr/local/lib/python3.10/dist-packages/peft/mixed_model.py", line 27, in from peft.tuners.mixed import COMPATIBLE_TUNER_TYPES File "/usr/local/lib/python3.10/dist-packages/peft/tuners/init.py", line 22, in from .rosa import RosaConfig, RosaModel, RosaScheduler File "/usr/local/lib/python3.10/dist-packages/peft/tuners/rosa/init.py", line 19, in from .layer import Embedding, Linear, RosaLayer File "/usr/local/lib/python3.10/dist-packages/peft/tuners/rosa/layer.py", line 30, in import spops File "/content/PanzaMail/scripts/PanzaMail/scripts/spops/spops/init.py", line 1, in from spops.spmm import spmm File "/content/PanzaMail/scripts/PanzaMail/scripts/spops/spops/spmm.py", line 2, in import spops_backend ImportError: /content/PanzaMail/scripts/PanzaMail/scripts/spops/spops_backend.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN3c104cuda14ExchangeDeviceEa

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/content/PanzaMail/scripts/../src/panza/data_preparation/create_vector_store.py", line 75, in main() File "/content/PanzaMail/scripts/../src/panza/data_preparation/create_vector_store.py", line 61, in main embeddings_model = rag.get_embeddings_model(args.embedding_model) File "/content/PanzaMail/src/panza/utils/rag.py", line 11, in get_embeddings_model embeddings_model = HuggingFaceEmbeddings( File "/usr/local/lib/python3.10/dist-packages/langchain_core/_api/deprecation.py", line 203, in warn_if_direct_instance return wrapped(self, *args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/langchain_community/embeddings/huggingface.py", line 71, in init import sentence_transformers File "/usr/local/lib/python3.10/dist-packages/sentence_transformers/init.py", line 15, in from sentence_transformers.trainer import SentenceTransformerTrainer File "/usr/local/lib/python3.10/dist-packages/sentence_transformers/trainer.py", line 10, in from transformers import EvalPrediction, PreTrainedTokenizerBase, Trainer, TrainerCallback File "", line 1075, in _handle_fromlist File "/usr/local/lib/python3.10/dist-packages/transformers/utils/import_utils.py", line 1500, in getattr module = self._get_module(self._class_to_module[name]) File "/usr/local/lib/python3.10/dist-packages/transformers/utils/import_utils.py", line 1512, in _get_module raise RuntimeError( RuntimeError: Failed to import transformers.trainer because of the following error (look up to see its traceback): /content/PanzaMail/scripts/PanzaMail/scripts/spops/spops_backend.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN3c104cuda14ExchangeDeviceEa