Bus error (core dumped)

yoh240404 commented 3 weeks ago

用的示例代码

from FlagEmbedding import FlagAutoModel
import argparse
import json
import random
import numpy as np
import faiss
from tqdm import tqdm
from FlagEmbedding import FlagModel

if __name__` == '__main__':
    model_path = "./base-model/BAAI/bge-base-en-v1.5"

    # model = FlagAutoModel.from_finetuned(model_path,
    #                                       query_instruction_for_retrieval="Represent this sentence for searching relevant passages:",
    #                                       use_fp16=True)

    model = FlagModel(model_path, query_instruction_for_retrieval="Represent this sentence for searching relevant passages:")

    sentences_1 = ["I love NLP", "I love machine learning"]
    sentences_2 = ["I love BGE", "I love text retrieval"]
    print(3)
    embeddings_1 = model.encode(sentences_1, batch_size=256)
    print(4)
    embeddings_2 = model.encode(sentences_2, batch_size=256)
    print(5)
    similarity = embeddings_1 @ embeddings_2.T
    print(similarity)

报错如下：

>>> 3
Bus error (core dumped)

旧的版本没有问题，今天clone的新版本出现这个问题。

545999961 commented 2 weeks ago

测试了下代码，运行起来是没问题的，看起来像是访问地址有问题

yoh240404 commented 2 weeks ago

请问访问地址有问题是啥意思

chaofan @.***>于2024年11月4日周一14:56写道：

测试了下代码，运行起来是没问题的，看起来像是访问地址有问题

— Reply to this email directly, view it on GitHub https://github.com/FlagOpen/FlagEmbedding/issues/1182#issuecomment-2453938689, or unsubscribe https://github.com/notifications/unsubscribe-auth/BHSZF44MWKHZ5TYDISQIU33Z64LABAVCNFSM6AAAAABRAQY422VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJTHEZTQNRYHE . You are receiving this because you authored the thread.Message ID: @.***>

545999961 commented 2 weeks ago

Bus error (core dumped)

看起来像是内存访问的问题，调用huggingface模块或者sentence transformers的代码可以正常运行吗

yoh240404 commented 2 weeks ago

可以的，那应该是开发机的问题

chaofan @.***>于2024年11月4日周一15:14写道：

Bus error (core dumped)

看起来像是内存访问的问题，调用huggingface https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/inference/embedder#1-normal-model 模块或者sentence transformers https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/inference/embedder#using-sentence-transformers 的代码可以正常运行吗

— Reply to this email directly, view it on GitHub https://github.com/FlagOpen/FlagEmbedding/issues/1182#issuecomment-2453962569, or unsubscribe https://github.com/notifications/unsubscribe-auth/BHSZF44J5ZZGVWXI4O7IVN3Z64NF3AVCNFSM6AAAAABRAQY422VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJTHE3DENJWHE . You are receiving this because you authored the thread.Message ID: @.***>

ShenweiXie commented 1 week ago

I've encounter the same issues, have you fixed this?

545999961 commented 1 week ago

I've encounter the same issues, have you fixed this?

May I ask what the model of the machine is?

yoh240404 commented 1 week ago

I've encounter the same issues, have you fixed this?

脚本运行前指定卡号即可

hanhainebula commented 1 week ago

I've encounter the same issues, have you fixed this?

脚本运行前指定卡号即可

你好，方便问一下是使用的多GPU还是单GPU吗，我们这边现在想知道这个报错的具体原因看是否需要做特定的修复，谢谢！

yoh240404 commented 1 week ago

I've encounter the same issues, have you fixed this?

脚本运行前指定卡号即可

你好，方便问一下是使用的多GPU还是单GPU吗，我们这边现在想知道这个报错的具体原因看是否需要做特定的修复，谢谢！

单卡多卡都会出现，脚本运行前设一下系统变量就可以：

export CUDA_VISIBLE_DEVICES=1 python ./test.py [[0.6543 0.7573] [0.6562 0.7227]]

FlagOpen / FlagEmbedding

Bus error (core dumped) #1182