Open Xiaom961 opened 1 year ago
Rubiksman78
commented
1 year ago I've updated the zip from the Drive very recently and there is indeed an issue with torch version for GPU. Correcting this asap and reuploading it.
You can fix it right now and it will do the same thing by running this command from a terminal opened in the RenAI folder:
"libs/pythonlib/python.exe" -m pip install --force torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117
Xiaom961
commented
1 year ago Ohh, I was already going crazy thinking that I messed up some file. I spent the whole afternoon looking for a solution. Now I'm heading to work, so in a few more hours, I will try to fix it.
Thank you very much for the quick response. I will let you know how it went with the problem.
Rubiksman78
commented
1 year ago I've updated the file, you can download the zip from the Drive again and replace your previous one.
Xiaom961
commented
1 year ago I have managed to fix the problem. Honestly, the new files you updated didn't work for me; I was still experiencing the same issue. However, I was able to solve it by following the steps you provided to use Huggingface models with int8: C:\Users\MyName\AppData\Local\Programs\Python\Python39\Lib\site-packages\bitsandbytes
I had to do the same thing, but in the Drive\RenAI-Chat-v1.0\libs\pythonlib\Lib\site-packages\bitsandbytes files that I downloaded (replacing files and modifying Main.py).
When I tried to do it from the C drive as you mentioned, it didn't detect CUDA. Therefore, it seems necessary to perform these steps specifically using the files from the DRIVE
Now I can run the program with Pygmalion without any problem, but the issue I have now is that occasionally, when asking something to my chatbot, the program gives me traceback errors. They occur very randomly. But atleast the chatbot works.
Rubiksman78
commented
1 year ago Indeed, I should have precised the potential issue with int8 if your GPU is an old generation. What is the traceback for when it crashes?
Xiaom961
commented
1 year ago hapenned 10 mins ago, here the full traceback:
I'm sorry, but an uncaught exception occurred.
While running game code:
File "game/char_chat.rpy", line 122, in script call
call char_get_actions from _call_char_get_actions_1
File "game/char_chat.rpy", line 137, in script call
call is_talking from _call_is_talking
File "game/char_chat.rpy", line 165, in script
if sentences_list[0] == "server_ok":
File "game/char_chat.rpy", line 165, in
-- Full Traceback ------------------------------------------------------------
Full traceback:
File "game/char_chat.rpy", line 122, in script call
call char_get_actions from _call_char_get_actions_1
File "game/char_chat.rpy", line 137, in script call
call is_talking from _call_is_talking
File "game/char_chat.rpy", line 165, in script
if sentences_list[0] == "server_ok":
File "G:\AI+\renpy\ast.py", line 2115, in execute
if renpy.python.py_eval(condition):
File "G:\AI+\renpy\python.py", line 1080, in py_eval
return py_eval_bytecode(code, globals, locals)
File "G:\AI+\renpy\python.py", line 1073, in py_eval_bytecode
return eval(bytecode, globals, locals)
File "game/char_chat.rpy", line 165, in
Windows-10-10.0.19041 AMD64 Ren'Py 8.0.1.22070801 RenAI Chat 1.0 Thu May 25 12:06:13 2023
sacredbone
commented
10 months ago Hey, unfortunately I got the same or a very similar problem
I have a RTX4060 and CUDA 11.7.0 installed, tried to create a separate environment with the same result:
warn("The installed version of bitsandbytes was compiled without GPU support. " ╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ Z:\RenAIChat\RenAIChat-1.0-pc\main.py:57 in
│ │ │ │ 54 model_name = CHAT_CONFIG["model_name"] │ │ 55 gc.collect() │ │ 56 torch.cuda.empty_cache() │ │ ❱ 57 chat_model, tokenizer = build_model_and_tokenizer_for(model_name) │ │ 58 │ │ 59 generation_settings = { │ │ 60 │ "max_new_tokens": CHAT_CONFIG["max_new_tokens"], │ │ │ │ Z:\RenAIChat\RenAIChat-1.0-pc\chatbot\model.py:29 in build_model_and_tokenizer_for │ │ │ │ 26 │ │ for bad_word in _build_bad_words_list_for(model_name) │ │ 27 │ ] │ │ 28 │ logger.info(f"Loading the {model_name} model") │ │ ❱ 29 │ model = transformers.AutoModelForCausalLM.from_pretrained( │ │ 30 │ │ model_name, │ │ 31 │ │ bad_words_ids=bad_words_ids, │ │ 32 │ │ device_map="auto", │ │ │ │ Z:\RenAIChat\RenAIChat-1.0-pc\libs\pythonlib\lib\site-packages\transformers\models\auto\auto_fac │ │ tory.py:464 in from_pretrained │ │ │ │ 461 │ │ │ ) │ │ 462 │ │ elif type(config) in cls._model_mapping.keys(): │ │ 463 │ │ │ model_class = _get_model_class(config, cls._model_mapping) │ │ ❱ 464 │ │ │ return model_class.from_pretrained( │ │ 465 │ │ │ │ pretrained_model_name_or_path, *model_args, config=config, hub_kwargs, │ │ 466 │ │ │ ) │ │ 467 │ │ raise ValueError( │ │ │ │ Z:\RenAIChat\RenAIChat-1.0-pc\libs\pythonlib\lib\site-packages\transformers\modeling_utils.py:24 │ │ 78 in from_pretrained │ │ │ │ 2475 │ │ │ │ mismatched_keys, │ │ 2476 │ │ │ │ offload_index, │ │ 2477 │ │ │ │ error_msgs, │ │ ❱ 2478 │ │ │ ) = cls._load_pretrained_model( │ │ 2479 │ │ │ │ model, │ │ 2480 │ │ │ │ state_dict, │ │ 2481 │ │ │ │ loaded_state_dict_keys, # XXX: rename? │ │ │ │ Z:\RenAIChat\RenAIChat-1.0-pc\libs\pythonlib\lib\site-packages\transformers\modeling_utils.py:27 │ │ 94 in _load_pretrained_model │ │ │ │ 2791 │ │ │ │ ) │ │ 2792 │ │ │ │ │ │ 2793 │ │ │ │ if low_cpu_mem_usage: │ │ ❱ 2794 │ │ │ │ │ new_error_msgs, offload_index, state_dict_index = _load_state_dict_i │ │ 2795 │ │ │ │ │ │ model_to_load, │ │ 2796 │ │ │ │ │ │ state_dict, │ │ 2797 │ │ │ │ │ │ loaded_keys, │ │ │ │ Z:\RenAIChat\RenAIChat-1.0-pc\libs\pythonlib\lib\site-packages\transformers\modeling_utils.py:66 │ │ 5 in _load_state_dict_into_meta_model │ │ │ │ 662 │ │ │ # For backward compatibility with older versions of accelerate│ │ 663 │ │ │ set_module_tensor_to_device(model, param_name, param_device, set_module_kw │ │ 664 │ │ else: │ │ ❱ 665 │ │ │ set_module_8bit_tensor_to_device(model, param_name, param_device, value=para │ │ 666 │ │ │ 667 │ return error_msgs, offload_index, state_dict_index │ │ 668 │ │ │ │ Z:\RenAIChat\RenAIChat-1.0-pc\libs\pythonlib\lib\site-packages\transformers\utils\bitsandbytes.p │ │ y:77 in set_module_8bit_tensor_to_device │ │ │ │ 74 │ │ if value is None: │ │ 75 │ │ │ new_value = old_value.to(device) │ │ 76 │ │ elif isinstance(value, torch.Tensor): │ │ ❱ 77 │ │ │ new_value = value.to(device) │ │ 78 │ │ else: │ │ 79 │ │ │ new_value = torch.tensor(value, device=device) │ │ 80 │ │ │ │ Z:\RenAIChat\RenAIChat-1.0-pc\libs\pythonlib\lib\site-packages\torch\cuda__init__.py:289 in │ │ _lazy_init │ │ │ │ 286 │ │ │ │ "multiprocessing, you must use the 'spawn' start method" │ │ 287 │ │ │ ) │ │ 288 │ │ if not hasattr(torch._C, "_cuda_getDeviceCount"): │ │ ❱ 289 │ │ │ raise AssertionError("Torch not compiled with CUDA enabled") │ │ 290 │ │ if _cudart is None: │ │ 291 │ │ │ raise AssertionError( │ │ 292 │ │ │ │ "libcudart functions unavailable. It looks like you have a broken build? │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ AssertionError: Torch not compiled with CUDA enabled
I would appreciate every help to get it running.
hello, Im having a rough time trying to this program works, I was using Monika AI just fine, so I wanted to try this, but Im having a hard time trying to make it works:
I follow all the steps, installed cuda , libraries etc and nothing.