Hit the memory issue when running simple_ui.py

after all requirements have been satisfied, but hit memory malloc fails, as following output, while trying to run simple_ui.py is there a minimum hbm memory requirement for the device? and i tried it within Orange Pi AI Pro device which has 16G memory.

Thanks!


root@orangepiaipro:/home/HwHiAiUser/llm/ChatPDF# python simple_ui.py
Building prefix dict from the default dictionary ...
Loading model from cache /tmp/jieba.cache
Loading model cost 2.425 seconds.
Prefix dict has been built successfully.
2024-05-13 16:19:55.977 | INFO     | __main__:<module>:32 - Namespace(gen_model_type='auto', gen_model_name='01-ai/Yi-6B-Chat', lora_model=None, rerank_model_name=None, corpus_files='sample.pdf', int4=False, int8=False, chunk_size=220, chunk_overlap=0, num_expand_context_chunk=1, server_name='0.0.0.0', server_port=8082, share=False)
Traceback (most recent call last):
  File "/home/HwHiAiUser/llm/ChatPDF/simple_ui.py", line 34, in <module>
    model = ChatPDF(
  File "/home/HwHiAiUser/llm/ChatPDF/chatpdf.py", line 165, in __init__
    m1 = BertSimilarity(model_name_or_path="shibing624/text2vec-base-multilingual")
  File "/usr/local/miniconda3/lib/python3.9/site-packages/msimilarities/bert_similarity.py", line 49, in __init__
    self.sentence_model = SentenceModel(
  File "/usr/local/miniconda3/lib/python3.9/site-packages/ms2vec/sentence_model.py", line 76, in __init__
    self.bert = AutoModel.from_pretrained(model_name_or_path)
  File "/usr/local/miniconda3/lib/python3.9/site-packages/mindnlp/transformers/models/auto/auto_factory.py", line 100, in from_pretrained
    return model_class.from_pretrained(
  File "/usr/local/miniconda3/lib/python3.9/site-packages/mindnlp/transformers/modeling_utils.py", line 1037, in from_pretrained
    model = cls(config, *model_args, **model_kwargs)
  File "/usr/local/miniconda3/lib/python3.9/site-packages/mindnlp/transformers/models/bert/modeling_bert.py", line 699, in __init__
    self.embeddings = BertEmbeddings(config)
  File "/usr/local/miniconda3/lib/python3.9/site-packages/mindnlp/transformers/models/bert/modeling_bert.py", line 118, in __init__
    self.token_type_ids = ops.zeros(self.position_ids.shape, dtype=mindspore.int64)
  File "/usr/local/miniconda3/lib/python3.9/site-packages/mindspore/common/_stub_tensor.py", line 85, in shape
    self.stub_shape = self.stub.get_shape()
RuntimeError: Ascend kernel runtime initialization failed. The details refer to 'Ascend Error Message'.

----------------------------------------------------
- Framework Error Message:
----------------------------------------------------
Malloc device memory failed, free memory size is less than half of total memory size.Device 0 Device HBM total size:16367894528 Device HBM free size:5145612288 may be other processes occupying this card, check as: ps -ef|grep python

----------------------------------------------------
- C++ Call Stack: (For framework developers)
----------------------------------------------------
mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:389 Init
mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_adapter.cc:73 Initialize

lvyufeng / ChatPDF

Hit the memory issue when running simple_ui.py #2