PaddlePaddle / PaddleNLP

👑 Easy-to-use and powerful NLP and LLM library with 🤗 Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including 🗂Text Classification, 🔍 Neural Search, ❓ Question Answering, ℹ️ Information Extraction, 📄 Document Intelligence, 💌 Sentiment Analysis etc.
https://paddlenlp.readthedocs.io
Apache License 2.0
12.19k stars 2.95k forks source link

[Question]: (ResourceExhausted) Fail to alloc memory of 4165888745472 size, error code is 12. [Hint: Expected error == 0, but received error:12 != 0:0.] (at /paddle/paddle/fluid/memory/allocation/cpu_allocator.cc:50) [operator < fill_constant > error] #8208

Closed Hughhuh closed 6 months ago

Hughhuh commented 8 months ago

请提出你的问题

schema = ['姓名', '毕业院校', '职位', '月收入', '身体状况'] ie.set_schema(schema) ie('兹证明凌霄为本单位职工,已连续在我单位工作5 年。学历为嘉利顿大学毕业,目前在我单位担任总经理助理 职位。近一年内该员工在我单位平均月收入(税后)为 12000 元。该职工身体状况良好。本单位仅此承诺上述表述是正确的,真实的。')

/usr/local/python3.7.0/lib/python3.7/site-packages/paddlenlp/transformers/tokenizer_utils_base.py:2541: FutureWarning: The max_seq_len argument is deprecated and will be removed in a future version, please use max_length instead. FutureWarning, /usr/local/python3.7.0/lib/python3.7/site-packages/paddlenlp/transformers/tokenizer_utils_base.py:1944: FutureWarning: The pad_to_max_length argument is deprecated and will be removed in a future version, use padding=True or padding='longest' to pad to the longest sequence in the batch, or use padding='max_length' to pad to a max length. In this case, you can give a specific length with max_length (e.g. max_length=45) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert). FutureWarning,

MemoryError Traceback (most recent call last) /tmp/ipykernel_621/2600741435.py in 1 schema = ['姓名', '毕业院校', '职位', '月收入', '身体状况'] 2 ie.set_schema(schema) ----> 3 ie('兹证明凌霄为本单位职工,已连续在我单位工作5 年。学历为嘉利顿大学毕业,目前在我单位担任总经理助理 职位。近一年内该员工在我单位平均月收入(税后)为 12000 元。该职工身体状况良好。本单位仅此承诺上述表述是正确的,真实的。')

/usr/local/python3.7.0/lib/python3.7/site-packages/paddlenlp/taskflow/taskflow.py in call(self, *inputs, kwargs) 820 The main work function in the taskflow. 821 """ --> 822 results = self.task_instance(inputs, kwargs) 823 return results 824

/usr/local/python3.7.0/lib/python3.7/site-packages/paddlenlp/taskflow/task.py in call(self, *args, kwargs) 525 def call(self, *args, *kwargs): 526 inputs = self._preprocess(args) --> 527 outputs = self._run_model(inputs, kwargs) 528 results = self._postprocess(outputs) 529 return results

/usr/local/python3.7.0/lib/python3.7/site-packages/paddlenlp/taskflow/information_extraction.py in _run_model(self, inputs) 1066 raw_inputs = inputs["text"] 1067 _inputs = self._parse_inputs(raw_inputs) -> 1068 results = self._multi_stage_predict(_inputs) 1069 inputs["result"] = results 1070 return inputs

/usr/local/python3.7.0/lib/python3.7/site-packages/paddlenlp/taskflow/information_extraction.py in _multi_stage_predict(self, data) 1164 result_list = [] 1165 else: -> 1166 result_list = self._single_stage_predict(examples) 1167 1168 if not node.parent_relations:

/usr/local/python3.7.0/lib/python3.7/site-packages/paddlenlp/taskflow/information_extraction.py in _single_stage_predict(self, inputs) 977 self.input_handles[2].copy_from_cpu(pos_ids.numpy()) 978 self.input_handles[3].copy_from_cpu(att_mask.numpy()) --> 979 self.predictor.run() 980 start_prob = self.output_handle[0].copy_to_cpu().tolist() 981 end_prob = self.output_handle[1].copy_to_cpu().tolist()

MemoryError: (ResourceExhausted) Fail to alloc memory of 4165888745472 size, error code is 12. [Hint: Expected error == 0, but received error:12 != 0:0.] (at /paddle/paddle/fluid/memory/allocation/cpu_allocator.cc:50) [operator < fill_constant > error]

DesmonDay commented 8 months ago

你用的paddle版本和paddlenlp版本是什么呢?

xiaose7118 commented 6 months ago

最后是怎么解决的

lingshi91 commented 6 months ago

同样遇到了这个问题,怎么解决的