search

Jittor / JittorLLMs

计图大模型推理库，具有高性能、配置要求低、中文支持好、可移植等特点
Apache License 2.0
2.36k stars 180 forks source link

windows下运行后会报错AttributeError: 'Embedding' object has no attribute 'astype' #150

Open netstat1grep80 opened 1 year ago

netstat1grep80 commented 1 year ago

先说一下我的环境,win10,conda环境安装的python 3.10.11,根据requirements安装的包如下 pip list Package Version

astunparse 1.6.3 certifi 2023.5.7 charset-normalizer 3.1.0 colorama 0.4.6 cpm-kernels 1.0.11 filelock 3.12.2 fsspec 2023.6.0 huggingface-hub 0.15.1 icetk 0.0.4 idna 3.4 jittor 1.3.8.4 jtorch 0.1.6 numpy 1.25.0 packaging 23.1 Pillow 9.5.0 pip 23.1.2 protobuf 3.19.5 pywin32 306 PyYAML 6.0 regex 2023.6.3 requests 2.31.0 sentencepiece 0.1.99 setuptools 67.8.0 six 1.16.0 tokenizers 0.13.3 torch 2.0.0 torchvision 0.15 tqdm 4.65.0 transformers 4.26.1 typing_extensions 4.7.0 urllib3 2.0.3 wheel 0.38.4

接下来是我运行python .\cli_demo.py chatglm 的结果

[i 0629 10:25:35.739000 40 compiler.py:955] Jittor(1.3.8.4) src: c:\programdata\anaconda3\envs\jittor\lib\site-packages\jittor [i 0629 10:25:35.775000 40 compiler.py:956] cl at C:\Users\laoz.cache\jittor\msvc\VC_____\bin\cl.exe(19.29.30133) [i 0629 10:25:35.776000 40 compiler.py:957] cache_path: C:\Users\laoz.cache\jittor\jt1.3.8\cl\py3.10.11\Windows-10-10.x50\IntelRXeonRCPUxbf\default [i 0629 10:25:35.780000 40 install_cuda.py:93] cuda_driver_version: [11, 2, 0] [i 0629 10:25:35.849000 40 init.py:411] Found C:\Users\laoz.cache\jittor\jtcuda\cuda11.2_cudnn8_win\bin\nvcc.exe(11.2.67) at C:\Users\laoz.cache\jittor\jtcuda\cuda11.2_cudnn8_win\bin\nvcc.exe. [i 0629 10:25:36.031000 40 compiler.py:1010] cuda key:cu11.2.67 [i 0629 10:25:36.037000 40 init.py:227] Total mem: 31.92GB, using 10 procs for compiling. [i 0629 10:25:37.537000 40 jit_compiler.cc:28] Load ccpath: C:\Users\laoz.cache\jittor\msvc\VC____\\bin\cl.exe [i 0629 10:25:37.550000 40 init.cc:62] Found cuda archs: [52,] Explicitly passing a revision is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision. Explicitly passing a revision is encouraged when loading a configuration with custom code to ensure no malicious code has been contributed in a newer revision. Explicitly passing a revision is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision. Traceback (most recent call last): File "D:\ai\JittorLLMs\cli_demo.py", line 8, in model = models.get_model(args) File "D:\ai\JittorLLMs\models__init.py", line 46, in get_model return module.get_model(args) File "D:\ai\JittorLLMs\models\chatglm__init__.py", line 48, in get_model return ChatGLMMdoel(args) File "D:\ai\JittorLLMs\models\chatglm__init.py", line 22, in init__ self.model = AutoModel.from_pretrained(os.path.dirname(file), trust_remote_code=True) File "C:\ProgramData\Anaconda3\envs\jittor\lib\site-packages\transformers\models\auto\auto_factory.py", line 459, in from_pretrained return model_class.from_pretrained( File "C:\ProgramData\Anaconda3\envs\jittor\lib\site-packages\transformers\modeling_utils.py", line 2362, in from_pretrained model = cls(config, *model_args, **model_kwargs) File "C:\Users\laoz/.cache\huggingface\modules\transformers_modules\local\modeling_chatglm.py", line 1019, in init self.transformer = ChatGLMModel(config) File "C:\Users\laoz/.cache\huggingface\modules\transformers_modules\local\modeling_chatglm.py", line 770, in init self.word_embeddings = skip_init( File "C:\ProgramData\Anaconda3\envs\jittor\lib\site-packages\jtorch\init.py", line 50, in inner ret.astype(dtype) File "C:\ProgramData\Anaconda3\envs\jittor\lib\site-packages\jittor\init.py", line 1721, in getattr return object.getattribute__(self, key) AttributeError: 'Embedding' object has no attribute 'astype'

cocowy1 commented 1 year ago

+1请问这个怎么解决呀

cocowy1 commented 1 year ago

@cjld @zjp-shadow @lzhengning 请问大佬这是什么原因导致的呀

netstat1grep80 commented 1 year ago

pip install jittor==1.3.7.12 jtorch==0.1.3 -i https://pypi.jittor.org/simple 将版本可解。虽然astype解决了，但是我又遇到了一个新问题 . 和这个人遇到的问题一样 莫非我的显卡也不支持吗

TcDhlPro commented 1 year ago

pip install jittor==1.3.7.12 jtorch==0.1.3 -i https://pypi.jittor.org/simple 将版本可解。虽然astype解决了，但是我又遇到了一个新问题 . 看样是无解的

照你这个降版本后, 是可以启动起来了, 作者给的例子有很大问题

netstat1grep80 commented 1 year ago

我把我的日志都贴出来 ，等待哪位大师帮着解决一下

laoz@DESKTOP-GGGAJN3:~/JittorLLMs$ python web_demo.py chatglm [i 0630 11:48:23.728447 80 compiler.py:955] Jittor(1.3.7.12) src: /home/laoz/anaconda3/envs/jittor/lib/python3.10/site-packages/jittor [i 0630 11:48:23.732523 80 compiler.py:956] g++ at /usr/bin/g++(9.4.0) [i 0630 11:48:23.732720 80 compiler.py:957] cache_path: /home/laoz/.cache/jittor/jt1.3.7/g++9.4.0/py3.10.11/Linux-5.15.90.xc6/IntelRXeonRCPUxbf/default [i 0630 11:48:24.040999 80 install_cuda.py:93] cuda_driver_version: [11, 8] [i 0630 11:48:24.041609 80 install_cuda.py:81] restart /home/laoz/anaconda3/envs/jittor/bin/python ['web_demo.py', 'chatglm'] [i 0630 11:48:26.785346 00 compiler.py:955] Jittor(1.3.7.12) src: /home/laoz/anaconda3/envs/jittor/lib/python3.10/site-packages/jittor [i 0630 11:48:26.789626 00 compiler.py:956] g++ at /usr/bin/g++(9.4.0) [i 0630 11:48:26.789738 00 compiler.py:957] cache_path: /home/laoz/.cache/jittor/jt1.3.7/g++9.4.0/py3.10.11/Linux-5.15.90.xc6/IntelRXeonRCPUxbf/default [i 0630 11:48:27.111910 00 install_cuda.py:93] cuda_driver_version: [11, 8] [i 0630 11:48:27.118317 00 init.py:411] Found /home/laoz/.cache/jittor/jtcuda/cuda11.2_cudnn8_linux/bin/nvcc(11.2.152) at /home/laoz/.cache/jittor/jtcuda/cuda11.2_cudnn8_linux/bin/nvcc. [i 0630 11:48:27.267855 00 init.py:411] Found addr2line(2.34) at /usr/bin/addr2line. [i 0630 11:48:27.440078 00 compiler.py:1010] cuda key:cu11.2.152 [i 0630 11:48:28.435296 00 init.py:227] Total mem: 15.58GB, using 5 procs for compiling. [i 0630 11:48:29.220412 00 jit_compiler.cc:28] Load cc_path: /usr/bin/g++ [i 0630 11:48:30.108674 00 init.cc:62] Found cuda archs: [52,] [i 0630 11:48:30.352613 00 compile_extern.py:522] mpicc not found, distribution disabled. Explicitly passing a revision is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision. Explicitly passing a revision is encouraged when loading a configuration with custom code to ensure no malicious code has been contributed in a newer revision. Explicitly passing a revision is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision. Loading checkpoint shards: 100%|██████████████████████████████████████████████| 8/8 [01:46<00:00, 13.36s/it][i 0630 11:50:29.952318 00 cuda_flags.cc:39] CUDA enabled. /home/laoz/anaconda3/envs/jittor/lib/python3.10/site-packages/gradio/components/textbox.py:259: UserWarning: The style method is deprecated. Please set these arguments in the constructor instead. warnings.warn( Running on local URL: http://0.0.0.0:51234

To create a public link, set share=True in launch().

Compiling Operators(9/9) used: 16.3s eta: 0s

Compiling Operators(2/2) used: 6.31s eta: 0s

Compiling Operators(1/1) used: 5.88s eta: 0s /home/laoz/.cache/jittor/jt1.3.7/g++9.4.0/py3.10.11/Linux-5.15.90.xc6/IntelRXeonRCPUxbf/default/cu11.2.152/jit/opkey0_broadcast_toTx_float16DIM_3__BCAST_3opkey1_binary__Tx_float16Ty_float16___hash_f3d27a3882838ef3_op.cc(52): error: more than one conversion function from "jittor::float16" to a built-in type applies: function "half::operator float() const" function "half::operator short() const" function "half::operator unsigned short() const" function "half::operator int() const" function "half::operator unsigned int() const" function "half::operator long long() const" function "half::operator unsigned long long() const" function "half::operator __nv_bool() const"

/home/laoz/.cache/jittor/jt1.3.7/g++9.4.0/py3.10.11/Linux-5.15.90.xc6/IntelRXeonRCPUxbf/default/cu11.2.152/jit/opkey0_broadcast_toTx_float16DIM_3__BCAST_3opkey1_binary__Tx_float16Ty_float16___hash_f3d27a3882838ef3_op.cc(52): error: more than one conversion function from "jittor::float16" to a built-in type applies: function "half::operator float() const" function "half::operator short() const" function "half::operator unsigned short() const" function "half::operator int() const" function "half::operator unsigned int() const" function "half::operator long long() const" function "half::operator unsigned long long() const" function "half::operator __nv_bool() const"

2 errors detected in the compilation of "/home/laoz/.cache/jittor/jt1.3.7/g++9.4.0/py3.10.11/Linux-5.15.90.xc6/IntelRXeonRCPUxbf/default/cu11.2.152/jit/opkey0_broadcast_toTx_float16DIM_3__BCAST_3opkey1_binaryTx_float16Ty_float16___hash_f3d27a3882838ef3_op.cc". Traceback (most recent call last): File "/home/laoz/anaconda3/envs/jittor/lib/python3.10/site-packages/gradio/routes.py", line 437, in run_predict output = await app.get_blocks().process_api( File "/home/laoz/anaconda3/envs/jittor/lib/python3.10/site-packages/gradio/blocks.py", line 1352, in process_api result = await self.call_function( File "/home/laoz/anaconda3/envs/jittor/lib/python3.10/site-packages/gradio/blocks.py", line 1093, in call_function prediction = await utils.async_iteration(iterator) File "/home/laoz/anaconda3/envs/jittor/lib/python3.10/site-packages/gradio/utils.py", line 341, in async_iteration return await iterator.anext() File "/home/laoz/anaconda3/envs/jittor/lib/python3.10/site-packages/gradio/utils.py", line 334, in anext return await anyio.to_thread.run_sync( File "/home/laoz/anaconda3/envs/jittor/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "/home/laoz/anaconda3/envs/jittor/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread return await future File "/home/laoz/anaconda3/envs/jittor/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run result = context.run(func, *args) File "/home/laoz/anaconda3/envs/jittor/lib/python3.10/site-packages/gradio/utils.py", line 317, in run_sync_iterator_async return next(iterator) File "/home/laoz/JittorLLMs/web_demo.py", line 12, in predict for response, history in model.run_web_demo(input, history): File "/home/laoz/JittorLLMs/models/chatglm/init.py", line 42, in run_web_demo yield self.run(input_text, history=history) File "/home/laoz/JittorLLMs/models/chatglm/init.py", line 45, in run return self.model.chat(self.tokenizer, text, history=history) File "/home/laoz/anaconda3/envs/jittor/lib/python3.10/site-packages/jittor/init.py", line 118, in inner ret = func(*args, kw) File "/home/laoz/.cache/huggingface/modules/transformers_modules/local/modeling_chatglm.py", line 1233, in chat outputs = self.generate(input_ids, gen_kwargs) File "/home/laoz/anaconda3/envs/jittor/lib/python3.10/site-packages/jittor/init.py", line 118, in inner ret = func(*args, *kw) File "/home/laoz/anaconda3/envs/jittor/lib/python3.10/site-packages/transformers/generation/utils.py", line 1437, in generate return self.sample( File "/home/laoz/anaconda3/envs/jittor/lib/python3.10/site-packages/transformers/generation/utils.py", line 2443, in sample outputs = self( File "/home/laoz/anaconda3/envs/jittor/lib/python3.10/site-packages/jtorch/nn/init.py", line 16, in call return self.forward(args, kw) File "/home/laoz/.cache/huggingface/modules/transformers_modules/local/modeling_chatglm.py", line 1138, in forward transformer_outputs = self.transformer( File "/home/laoz/anaconda3/envs/jittor/lib/python3.10/site-packages/jtorch/nn/init.py", line 16, in call return self.forward(*args, *kw) File "/home/laoz/.cache/huggingface/modules/transformers_modules/local/modeling_chatglm.py", line 973, in forward layer_ret = layer( File "/home/laoz/anaconda3/envs/jittor/lib/python3.10/site-packages/jtorch/nn/init.py", line 16, in call return self.forward(args, **kw) File "/home/laoz/.cache/huggingface/modules/transformers_modules/local/modeling_chatglm.py", line 614, in forward attention_outputs = self.attention( File "/home/laoz/anaconda3/envs/jittor/lib/python3.10/site-packages/jtorch/nn/init.py", line 16, in call__ return self.forward(*args, kw) File "/home/laoz/.cache/huggingface/modules/transformers_modules/local/modeling_chatglm.py", line 454, in forward cos, sin = self.rotary_emb(q1, seq_len=position_ids.max() + 1) File "/home/laoz/anaconda3/envs/jittor/lib/python3.10/site-packages/jtorch/nn/init.py", line 16, in call return self.forward(*args, *kw) File "/home/laoz/.cache/huggingface/modules/transformers_modules/local/modeling_chatglm.py", line 202, in forward t = torch.arange(seq_len, device=x.device, dtype=self.inv_freq.dtype) File "/home/laoz/anaconda3/envs/jittor/lib/python3.10/site-packages/jtorch/init.py", line 31, in inner ret = func(args, kw) File "/home/laoz/anaconda3/envs/jittor/lib/python3.10/site-packages/jittor/misc.py", line 809, in arange if isinstance(start, jt.Var): start = start.item() RuntimeError: Wrong inputs arguments, Please refer to examples(help(jt.item)).

Types of your inputs are: self = Var, args = (),

The function declarations are: ItemData item()

Failed reason:[f 0630 11:52:09.671526 68 parallel_compiler.cc:330] Error happend during compilation: [Error] source file location:/home/laoz/.cache/jittor/jt1.3.7/g++9.4.0/py3.10.11/Linux-5.15.90.xc6/IntelRXeonRCPUxbf/default/cu11.2.152/jit/opkey0_broadcast_toTx_float16DIM_3__BCAST_3opkey1_binary__Tx_float16Ty_float16___hash_f3d27a3882838ef3_op.cc Compile fused operator(15/18)failed:[Op(12261:0:1:1:i1:o1:s0,broadcast_to->12262),Op(12263:0:1:1:i2:o1:s0,binary.add->12264),]

Reason: [f 0630 11:52:07.998326 40:C3 log.cc:608] Check failed ret(256) == 0(0) Run cmd failed: "/home/laoz/.cache/jittor/jtcuda/cuda11.2_cudnn8_linux/bin/nvcc" "/home/laoz/.cache/jittor/jt1.3.7/g++9.4.0/py3.10.11/Linux-5.15.90.xc6/IntelRXeonRCPUxbf/default/cu11.2.152/jit/opkey0_broadcast_toTx_float16DIM_3__BCAST_3opkey1_binaryTx_float16Ty_float16___hash_f3d27a3882838ef3_op.cc" -std=c++14 -Xcompiler -fPIC -Xcompiler -march=native -Xcompiler -fdiagnostics-color=always -lstdc++ -ldl -shared -I"/home/laoz/anaconda3/envs/jittor/lib/python3.10/site-packages/jittor/src" -I/home/laoz/anaconda3/envs/jittor/include/python3.10 -I/home/laoz/anaconda3/envs/jittor/include/python3.10 -DHAS_CUDA -DIS_CUDA -I"/home/laoz/.cache/jittor/jtcuda/cuda11.2_cudnn8_linux/include" -I"/home/laoz/anaconda3/envs/jittor/lib/python3.10/site-packages/jittor/extern/cuda/inc" -lcudart -L"/home/laoz/.cache/jittor/jtcuda/cuda11.2_cudnn8_linux/lib64" -Xlinker -rpath="/home/laoz/.cache/jittor/jtcuda/cuda11.2_cudnn8_linux/lib64" -I"/home/laoz/.cache/jittor/jt1.3.7/g++9.4.0/py3.10.11/Linux-5.15.90.xc6/IntelRXeonRCPUxbf/default/cu11.2.152" -L"/home/laoz/.cache/jittor/jt1.3.7/g++9.4.0/py3.10.11/Linux-5.15.90.xc6/IntelRXeonRCPUxbf/default/cu11.2.152" -Xlinker -rpath="/home/laoz/.cache/jittor/jt1.3.7/g++9.4.0/py3.10.11/Linux-5.15.90.xc6/IntelRXeonRCPUxbf/default/cu11.2.152" -L"/home/laoz/.cache/jittor/jt1.3.7/g++9.4.0/py3.10.11/Linux-5.15.90.xc6/IntelRXeonRCPUxbf/default" -Xlinker -rpath="/home/laoz/.cache/jittor/jt1.3.7/g++9.4.0/py3.10.11/Linux-5.15.90.xc6/IntelRXeonRCPUxbf/default" -l:"jit_utils_core.cpython-310-x86_64-linux-gnu".so -l:"jittor_core.cpython-310-x86_64-linux-gnu".so -x cu --cudart=shared -ccbin="/usr/bin/g++" --use_fast_math -w -I"/home/laoz/anaconda3/envs/jittor/lib/python3.10/site-packages/jittor/extern/cuda/inc" -arch=compute_52 -code=sm_52 -o "/home/laoz/.cache/jittor/jt1.3.7/g++9.4.0/py3.10.11/Linux-5.15.90.xc6/IntelRXeonRCPUxbf/default/cu11.2.152/jit/opkey0_broadcast_toTx_float16__DIM_3BCAST_3opkey1_binary__Tx_float16Ty_float16_____hash_f3d27a3882838ef3_op.so"

opkey0_broadcast_toTx_float16DIM_3__BCAST_3opkey1_binary__Tx_float16Ty_float16___hash_f3d27a3882838ef3_op.cc 文件内容如下

define JIT 1

define JIT_cuda 1

include

include

include "var.h"

include "ops/broadcast_to_op.h"

include "ops/op_register.h"

include

include "var.h"

include "ops/binary_op.h"

include "ops/broadcast_to_op.h"

include "ops/op_register.h"

include

include "fused_op.h"

include "type/fp16_compute.h"

define op0_broadcast_to

define op0_Tx float16

define op0_DIM 3

define op0_BCAST 3

define op0_index_t int32

define op1_binary

define op1_Tx float16

define op1_Ty float16

define op1_Tz float16

define op1_OP add

define op1_index_t int32

include "misc/cuda_atomic.h"

include "misc/cuda_limits.h"

include "helper_cuda.h"

using namespace jittor;

define INLINE_FUNC inline static void

launch_bounds(1024) global__ void func_f3d27a3882838ef3_0(op1_index_t range0, op1_index_t range1, op1_index_t range2, op0_Tx* restrict op0_xp, op1_Tx* restrict op1_xp, op1_Tz* restrict__ op1_zp, int tn1, int tn0) { int thread_id = blockIdx.x blockDim.x + threadIdx.x; int tn2 = 0; int tnum0 = 1<<(tn0-tn1); int tid0 = (thread_id>>tn1) & (tnum0-1); int tnum1 = 1<<(tn1-tn2); int tid1 = (thread_id>>tn2) & (tnum1-1); op0_index_t op0_xstride2 = 1; auto op0_xstride1 = op0_xstride2 range2; auto op0_xstride0 = op0_xstride1 1; op1_index_t op1_zstride2 = 1; auto op1_zstride1 = op1_zstride2 range2; auto op1_zstride0 = op1_zstride1 range1; op1_index_t range01 = range1 range0; for (op1_index_t id1 = tid0; id1<range01; id1+=tnum0) { for (op1_index_t id2 = tid1; id2<range2; id2+=tnum1) { auto op0_xid = + 0 op0_xstride0 + 0 op0_xstride1 + id2 op0_xstride2; auto op0_zd = op0_xp[op0_xid]; op1_index_t op1_i = + 0 op1_zstride0 + id1 op1_zstride1 + id2 op1_zstride2; op1_zp[op1_i] = ((op1_xp[op1_i])+(op0_zd )); //就是这行报错吧 } } }

pragma GCC diagnostic ignored "-Wunused-function"

inline static int get_thread_range_log(int& thread_num, int64 range) { int nbits = NanoVector::get_nbits(std::min((int64)thread_num, range)) - 2; thread_num >>= nbits; return nbits; } void jittor::FusedOp::jit_run() { auto op0_x = ((BroadcastToOp)(ops[0]))->x; auto op1_x = ((BinaryOp)(ops[1]))->x; auto op1_z = ((BinaryOp)(ops[1]))->z; op1_index_t range0 = op1_x->shape[0]; op1_index_t range1 = op1_x->shape[1]; op1_index_t range2 = op1_x->shape[2]; auto restrict op0_xp = op0_x->ptr(); auto restrict op1_xp = op1_x->ptr(); auto restrict op1_zp = op1_z->ptr(); { int thread_num = 262144; int thread_num_left = thread_num; int tn1 = get_thread_range_log(thread_num_left, range2); int tn0 = get_thread_range_log(thread_num_left, range1 * range0); tn0=tn0+tn1; tn0=std::max(tn0, 5); thread_num=1<<tn0; int p1 = std::max(thread_num/1024, 1); int p2 = std::min(thread_num, 1024); func_f3d27a3882838ef3_0<<<p1,p2>>>(range0,range1,range2,op0_xp,op1_xp,op1_zp,tn1,tn0); } }

我的显卡是Quadro M5000

TcDhlPro commented 1 year ago

web_demo我也没法用, 能用的就是api那个, 但是发了消息等模型回答要很久 image

TcDhlPro commented 1 year ago

image 我还有个问题就是, 如果装了这个, 启动都会报错

Jittor commented 1 year ago

更新最新版本可以解决该问题