使用示例代码出错 Invalid device: cuda:0

[oyy2000]

Code

import os from trustllm.generation.generation import LLMGeneration

llm_gen = LLMGeneration( model_path="meta-llama/Llama-2-7b-chat-hf", test_type="safety", data_path="eval_data", online_model=False, use_deepinfra=False, use_replicate=False, repetition_penalty=1.0, num_gpus=2, max_new_tokens=512, debug=True, device='cuda:0' )

llm_gen.generation_results()

Response:

Beginning generation with safety evaluation at temperature 0. Evaluation target model: llama2-7b Test function failed on attempt 1: Invalid device: cuda:0 Retrying in 3 seconds... Beginning generation with safety evaluation at temperature 0. Evaluation target model: llama2-7b Test function failed on attempt 2: Invalid device: cuda:0 Retrying in 3 seconds... Beginning generation with safety evaluation at temperature 0. Evaluation target model: llama2-7b Test function failed on attempt 3: Invalid device: cuda:0 Retrying in 3 seconds...

使用其他 transformer model 时使用 cuda:0是没问题的，请问如何解决，提前感谢您的回复🙏

[HowieHwong]

您好，感谢您的反馈， @nauyisu022 已经做了相应的修改，您可以重新运行代码看下是否还有错误。

[oyy2000]

非常感谢您们如此快的回复❤️