search

Unispac / Visual-Adversarial-Examples-Jailbreak-Large-Language-Models

Repository for the Paper (AAAI 2024, Oral) --- Visual Adversarial Examples Jailbreak Large Language Models
165 stars 12 forks source link

Some problems about llava_llama_v2 #21

Open jiaxiaojunQAQ opened 3 months ago

jiaxiaojunQAQ commented 3 months ago

When I run python llava_llama_v2_visual_attack.py --n_iters 5000 --constrained --save_dir results_llava_llama_v2_constrained_16 --eps 16 --alpha 1, I meet following problems.

model = /mnt/local/LLM_model/llava-llama-2-13b-chat-lightning-preview llava-llama-2-13b-chat-lightning-preview Traceback (most recent call last): File "/mnt/local/jxj/Jailbreak_MLLM/VAE_Jailbreak/llava_llama_v2_visual_attack.py", line 42, in tokenizer, model, image_processor, model_name = get_model(args) File "/mnt/local/jxj/Jailbreak_MLLM/VAE_Jailbreak/llava_llama_2/utils.py", line 146, in get_model tokenizer, model, image_processor, context_len = load_pretrained_model(args.model_path, args.model_base, model_name) File "/mnt/local/jxj/Jailbreak_MLLM/VAE_Jailbreak/llava_llama_2/model/builder.py", line 119, in load_pretrained_model model = AutoModelForCausalLM.from_pretrained(model_path, low_cpu_mem_usage=True, **kwargs) File "/home/jxj/anaconda3/envs/llm_new/lib/python3.9/site-packages/transformers/models/auto/auto_factory.py", line 474, in from_pretrained raise ValueError( ValueError: Unrecognized configuration class <class 'llava_llama_2.model.builder.load_pretrained_model..LlavaConfig'> for this kind of AutoModel: AutoModelForCausalLM. Model type should be one of BartConfig, BertConfig, BertGenerationConfig, BigBirdConfig, BigBirdPegasusConfig, BioGptConfig, BlenderbotConfig, BlenderbotSmallConfig, BloomConfig, CamembertConfig, CodeGenConfig, CpmAntConfig, CTRLConfig, Data2VecTextConfig, ElectraConfig, ErnieConfig, GitConfig, GPT2Config, GPT2Config, GPTBigCodeConfig, GPTNeoConfig, GPTNeoXConfig, GPTNeoXJapaneseConfig, GPTJConfig, LlamaConfig, MarianConfig, MBartConfig, MegaConfig, MegatronBertConfig, MvpConfig, OpenAIGPTConfig, OPTConfig, PegasusConfig, PLBartConfig, ProphetNetConfig, QDQBertConfig, ReformerConfig, RemBertConfig, RobertaConfig, RobertaPreLayerNormConfig, RoCBertConfig, RoFormerConfig, Speech2Text2Config, TransfoXLConfig, TrOCRConfig, XGLMConfig, XLMConfig, XLMProphetNetConfig, XLMRobertaConfig, XLMRobertaXLConfig, XLNetConfig, XmodConfig, LlavaConfig, LlavaMPTConfig.

What should I do?

xlnn commented 3 weeks ago

You should rename "llava-llama-2-13b-chat-lightning-preview",Thank you!