evar18
commented
3 weeks ago cc: @ParmuSingh would you be able to help with the above?
Open evar18 opened 3 weeks ago
evar18
commented
3 weeks ago cc: @ParmuSingh would you be able to help with the above?
System Info
Hi There, I'm running into the below error when trying to run the notebook. I'm fairly new to python and jupyter notebook so I'm stuck really. Can you help me solve these errors and get this working? Much appreciated
Information
🐛 Describe the bug
Command that fails:
jupyter nbconvert --execute --to notebook WhatsApp_Finetune.ipynbError logs
[NbConvertApp] Converting notebook WhatsApp_Finetune.ipynb to notebook
Traceback (most recent call last): File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/bin/jupyter-nbconvert", line 8, in
sys.exit(main())
^^^^^^
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/jupyter_core/application.py", line 283, in launch_instance
super().launch_instance(argv=argv, kwargs)
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/traitlets/config/application.py", line 1075, in launch_instance
app.start()
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/nbconvert/nbconvertapp.py", line 420, in start
self.convert_notebooks()
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/nbconvert/nbconvertapp.py", line 597, in convert_notebooks
self.convert_single_notebook(notebook_filename)
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/nbconvert/nbconvertapp.py", line 563, in convert_single_notebook
output, resources = self.export_single_notebook(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/nbconvert/nbconvertapp.py", line 487, in export_single_notebook
output, resources = self.exporter.from_filename(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/nbconvert/exporters/exporter.py", line 201, in from_filename
return self.from_file(f, resources=resources, kw)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/nbconvert/exporters/exporter.py", line 220, in from_file
return self.from_notebook_node(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/nbconvert/exporters/notebook.py", line 36, in from_notebook_node
nb_copy, resources = super().from_notebook_node(nb, resources, **kw)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/nbconvert/exporters/exporter.py", line 154, in from_notebook_node
nb_copy, resources = self._preprocess(nb_copy, resources)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/nbconvert/exporters/exporter.py", line 353, in _preprocess
nbc, resc = preprocessor(nbc, resc)
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/nbconvert/preprocessors/base.py", line 48, in call
return self.preprocess(nb, resources)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/nbconvert/preprocessors/execute.py", line 103, in preprocess
self.preprocess_cell(cell, resources, index)
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/nbconvert/preprocessors/execute.py", line 124, in preprocess_cell
cell = self.execute_cell(cell, index, store_history=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/jupyter_core/utils/init.py", line 165, in wrapped
return loop.run_until_complete(inner)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.12/asyncio/base_events.py", line 687, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/nbclient/client.py", line 1062, in async_execute_cell
await self._check_raise_for_error(cell, cell_index, exec_reply)
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/nbclient/client.py", line 918, in _check_raise_for_error
raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)
nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
import torch from transformers import LlamaForCausalLM, LlamaTokenizer
model_id="./models_hf/7B"
tokenizer = LlamaTokenizer.from_pretrained(model_id)
----- stderr -----
A module that was compiled using NumPy 1.x cannot be run in NumPy 2.1.2 as it may crash. To support both 1.x and 2.x versions of NumPy, modules must be compiled with NumPy 2.0. Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.
If you are a user of the module, the easiest solution will be to downgrade to 'numpy<2' or try to upgrade the affected module. We expect that some modules will need time to support NumPy 2.
Traceback (most recent call last): File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/ipykernel_launcher.py", line 18, in
app.launch_new_instance()
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/traitlets/config/application.py", line 1075, in launch_instance
app.start()
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/ipykernel/kernelapp.py", line 739, in start
self.io_loop.start()
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/tornado/platform/asyncio.py", line 205, in start
self.asyncio_loop.run_forever()
File "/usr/lib64/python3.12/asyncio/base_events.py", line 641, in run_forever
self._run_once()
File "/usr/lib64/python3.12/asyncio/base_events.py", line 1986, in _run_once
handle._run()
File "/usr/lib64/python3.12/asyncio/events.py", line 88, in _run
self._context.run(self._callback, self._args)
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/ipykernel/kernelbase.py", line 545, in dispatch_queue
await self.process_one()
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/ipykernel/kernelbase.py", line 534, in process_one
await dispatch(args)
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/ipykernel/kernelbase.py", line 437, in dispatch_shell
await result
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/ipykernel/ipkernel.py", line 362, in execute_request
await super().execute_request(stream, ident, parent)
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/ipykernel/kernelbase.py", line 778, in execute_request
reply_content = await reply_content
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/ipykernel/ipkernel.py", line 449, in do_execute
res = shell.run_cell(
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/ipykernel/zmqshell.py", line 549, in run_cell
return super().run_cell(*args, *kwargs)
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/IPython/core/interactiveshell.py", line 3075, in run_cell
result = self._run_cell(
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/IPython/core/interactiveshell.py", line 3130, in _run_cell
result = runner(coro)
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/IPython/core/async_helpers.py", line 128, in _pseudo_sync_runner
coro.send(None)
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/IPython/core/interactiveshell.py", line 3334, in run_cell_async
has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/IPython/core/interactiveshell.py", line 3517, in run_ast_nodes
if await self.runcode(code, result, async=asy):
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/IPython/core/interactiveshell.py", line 3577, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "/tmp/ipykernel_282115/308316149.py", line 1, in
import torch
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/torch/init.py", line 1471, in
from .functional import # noqa: F403
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/torch/functional.py", line 9, in
import torch.nn.functional as F
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/torch/nn/init.py", line 1, in
from .modules import * # noqa: F403
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/torch/nn/modules/init.py", line 35, in
from .transformer import TransformerEncoder, TransformerDecoder, \
File "/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/torch/nn/modules/transformer.py", line 20, in
device: torch.device = torch.device(torch._C._get_default_device()), # torch.device('cpu'),
/home/earn-tab/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/torch/nn/modules/transformer.py:20: UserWarning: Failed to initialize NumPy: _ARRAY_API not found (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:84.)
device: torch.device = torch.device(torch._C._get_default_device()), # torch.device('cpu'),
HFValidationError Traceback (most recent call last) File ~/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/transformers/utils/hub.py:403, in cached_file(path_or_repo_id, filename, cache_dir, force_download, resume_download, proxies, token, revision, local_files_only, subfolder, repo_type, user_agent, _raise_exceptions_for_gated_repo, _raise_exceptions_for_missing_entries, _raise_exceptions_for_connection_errors, _commit_hash, **deprecated_kwargs) 401 try: 402 # Load from URL or cache if already cached --> 403 resolved_file = hf_hub_download( 404 path_or_repo_id, 405 filename, 406 subfolder=None if len(subfolder) == 0 else subfolder, 407 repo_type=repo_type, 408 revision=revision, 409 cache_dir=cache_dir, 410 user_agent=user_agent, 411 force_download=force_download, 412 proxies=proxies, 413 resume_download=resume_download, 414 token=token, 415 local_files_only=local_files_only, 416 ) 417 except GatedRepoError as e:
File ~/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/huggingface_hub/utils/_validators.py:106, in validate_hf_hub_args.._inner_fn(*args, **kwargs)
105 if arg_name in ["repo_id", "from_id", "to_id"]:
--> 106 validate_repo_id(arg_value)
108 elif arg_name == "token" and arg_value is not None:
File ~/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/huggingface_hub/utils/_validators.py:154, in validate_repo_id(repo_id) 153 if repo_id.count("/") > 1: --> 154 raise HFValidationError( 155 "Repo id must be in the form 'repo_name' or 'namespace/repo_name':" 156 f" '{repo_id}'. Use
repo_typeargument if needed." 157 ) 159 if not REPO_ID_REGEX.match(repo_id):HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': './models_hf/7B'. Use
repo_typeargument if needed.The above exception was the direct cause of the following exception:
OSError Traceback (most recent call last) Cell In[5], line 6 2 from transformers import LlamaForCausalLM, LlamaTokenizer 4 model_id="./models_hf/7B" ----> 6 tokenizer = LlamaTokenizer.from_pretrained(model_id)
File ~/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/transformers/tokenization_utils_base.py:2132, in PreTrainedTokenizerBase.from_pretrained(cls, pretrained_model_name_or_path, cache_dir, force_download, local_files_only, token, revision, trust_remote_code, *init_inputs, **kwargs) 2129 if "tokenizer_file" in vocab_files: 2130 # Try to get the tokenizer config to see if there are versioned tokenizer files. 2131 fast_tokenizer_file = FULL_TOKENIZER_FILE -> 2132 resolved_config_file = cached_file( 2133 pretrained_model_name_or_path, 2134 TOKENIZER_CONFIG_FILE, 2135 cache_dir=cache_dir, 2136 force_download=force_download, 2137 resume_download=resume_download, 2138 proxies=proxies, 2139 token=token, 2140 revision=revision, 2141 local_files_only=local_files_only, 2142 subfolder=subfolder, 2143 user_agent=user_agent, 2144 _raise_exceptions_for_gated_repo=False, 2145 _raise_exceptions_for_missing_entries=False, 2146 _raise_exceptions_for_connection_errors=False, 2147 _commit_hash=commit_hash, 2148 ) 2149 commit_hash = extract_commit_hash(resolved_config_file, commit_hash) 2150 if resolved_config_file is not None:
File ~/Documents/src/WhatsApp-Llama/.env/lib64/python3.12/site-packages/transformers/utils/hub.py:469, in cached_file(path_or_repo_id, filename, cache_dir, force_download, resume_download, proxies, token, revision, local_files_only, subfolder, repo_type, user_agent, _raise_exceptions_for_gated_repo, _raise_exceptions_for_missing_entries, _raise_exceptions_for_connection_errors, _commit_hash, **deprecated_kwargs) 467 raise EnvironmentError(f"There was a specific connection error when trying to load {path_or_repo_id}:\n{err}") 468 except HFValidationError as e: --> 469 raise EnvironmentError( 470 f"Incorrect path_or_model_id: '{path_or_repo_id}'. Please provide either the path to a local folder or the repo_id of a model on the Hub." 471 ) from e 472 return resolved_file
OSError: Incorrect path_or_model_id: './models_hf/7B'. Please provide either the path to a local folder or the repo_id of a model on the Hub.
Expected behavior
jupyter notbook to start properly so I can chat with the trained model.