search

HKUDS / GraphGPT

[SIGIR'2024] "GraphGPT: Graph Instruction Tuning for Large Language Models"
https://arxiv.org/abs/2310.13023
Apache License 2.0
493 stars 36 forks source link

AttributeError: 'str' object has no attribute 'requires_grad_' #38

Closed zhihui-shao closed 6 months ago

zhihui-shao commented 6 months ago 
Traceback (most recent call last):
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_mem.py", line 15, in <module>
    train()
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_graph.py", line 863, in train
    model_graph_dict = model.get_model().initialize_graph_modules(
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/model/GraphLlama.py", line 148, in initialize_graph_modules
    graph_tower.requires_grad_(False)
AttributeError: 'str' object has no attribute 'requires_grad_'

我认为可能是vicuna的config.json中的问题：

"graph_hidden_size": 128, 
"pretrain_graph_model_path": "/home/shaozhihui/szh/GraphGPT/graph_transformer/"

我的目录结构如下： image

这是我的运行命令

model_path=./vicuna-7b-v1.5-16k
instruct_ds=./data/stage_1/graph_matching.json
graph_data_path=./graph_data/all_graph_data.pt
pretra_gnn=./graph_transformer/clip_gt_arxiv_pub.pkl
output_model=./checkpoints/stage_1

wandb offline
CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.run --nnodes=1 --nproc_per_node=2 --master_port=20001 \
    graphgpt/train/train_mem.py \
    --model_name_or_path ${model_path} \
    --version v1 \
    --data_path ${instruct_ds} \
    --graph_content ./arxiv_ti_ab.json \
    --graph_data_path ${graph_data_path} \
    --graph_tower ${pretra_gnn} \
    --tune_graph_mlp_adapter True \
    --graph_select_layer -2 \
    --use_graph_start_end \
    --bf16 False \
    --output_dir ${output_model} \
    --num_train_epochs 3 \
    --per_device_train_batch_size 2 \
    --per_device_eval_batch_size 2 \
    --gradient_accumulation_steps 1 \
    --evaluation_strategy "no" \
    --save_strategy "steps" \
    --save_steps 2400 \
    --save_total_limit 1 \
    --learning_rate 2e-3 \
    --weight_decay 0. \
    --warmup_ratio 0.03 \
    --lr_scheduler_type "cosine" \
    --logging_steps 1 \
    --tf32 False \
    --model_max_length 2048 \
    --gradient_checkpointing True \
    --lazy_preprocess True \
    --report_to wandb

有人能帮帮我吗，谢谢

linwhitehat commented 6 months ago 
Traceback (most recent call last):
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_mem.py", line 15, in <module>
    train()
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_graph.py", line 863, in train
    model_graph_dict = model.get_model().initialize_graph_modules(
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/model/GraphLlama.py", line 148, in initialize_graph_modules
    graph_tower.requires_grad_(False)
AttributeError: 'str' object has no attribute 'requires_grad_'

我认为可能是vicuna的config.json中的问题：

"graph_hidden_size": 128, 
"pretrain_graph_model_path": "/home/shaozhihui/szh/GraphGPT/graph_transformer/"

我的目录结构如下： image

这是我的运行命令

model_path=./vicuna-7b-v1.5-16k
instruct_ds=./data/stage_1/graph_matching.json
graph_data_path=./graph_data/all_graph_data.pt
pretra_gnn=./graph_transformer/clip_gt_arxiv_pub.pkl
output_model=./checkpoints/stage_1

wandb offline
CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.run --nnodes=1 --nproc_per_node=2 --master_port=20001 \
    graphgpt/train/train_mem.py \
    --model_name_or_path ${model_path} \
    --version v1 \
    --data_path ${instruct_ds} \
    --graph_content ./arxiv_ti_ab.json \
    --graph_data_path ${graph_data_path} \
    --graph_tower ${pretra_gnn} \
    --tune_graph_mlp_adapter True \
    --graph_select_layer -2 \
    --use_graph_start_end \
    --bf16 False \
    --output_dir ${output_model} \
    --num_train_epochs 3 \
    --per_device_train_batch_size 2 \
    --per_device_eval_batch_size 2 \
    --gradient_accumulation_steps 1 \
    --evaluation_strategy "no" \
    --save_strategy "steps" \
    --save_steps 2400 \
    --save_total_limit 1 \
    --learning_rate 2e-3 \
    --weight_decay 0. \
    --warmup_ratio 0.03 \
    --lr_scheduler_type "cosine" \
    --logging_steps 1 \
    --tf32 False \
    --model_max_length 2048 \
    --gradient_checkpointing True \
    --lazy_preprocess True \
    --report_to wandb

有人能帮帮我吗，谢谢

我解决了，作者写的的配置不对，应该改成pretra_gnn=clip_gt_arxiv

zhihui-shao commented 6 months ago 
Traceback (most recent call last):
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_mem.py", line 15, in <module>
    train()
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_graph.py", line 863, in train
    model_graph_dict = model.get_model().initialize_graph_modules(
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/model/GraphLlama.py", line 148, in initialize_graph_modules
    graph_tower.requires_grad_(False)
AttributeError: 'str' object has no attribute 'requires_grad_'

我认为可能是vicuna的config.json中的问题：

"graph_hidden_size": 128, 
"pretrain_graph_model_path": "/home/shaozhihui/szh/GraphGPT/graph_transformer/"

我的目录结构如下： image 这是我的运行命令

model_path=./vicuna-7b-v1.5-16k
instruct_ds=./data/stage_1/graph_matching.json
graph_data_path=./graph_data/all_graph_data.pt
pretra_gnn=./graph_transformer/clip_gt_arxiv_pub.pkl
output_model=./checkpoints/stage_1

wandb offline
CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.run --nnodes=1 --nproc_per_node=2 --master_port=20001 \
    graphgpt/train/train_mem.py \
    --model_name_or_path ${model_path} \
    --version v1 \
    --data_path ${instruct_ds} \
    --graph_content ./arxiv_ti_ab.json \
    --graph_data_path ${graph_data_path} \
    --graph_tower ${pretra_gnn} \
    --tune_graph_mlp_adapter True \
    --graph_select_layer -2 \
    --use_graph_start_end \
    --bf16 False \
    --output_dir ${output_model} \
    --num_train_epochs 3 \
    --per_device_train_batch_size 2 \
    --per_device_eval_batch_size 2 \
    --gradient_accumulation_steps 1 \
    --evaluation_strategy "no" \
    --save_strategy "steps" \
    --save_steps 2400 \
    --save_total_limit 1 \
    --learning_rate 2e-3 \
    --weight_decay 0. \
    --warmup_ratio 0.03 \
    --lr_scheduler_type "cosine" \
    --logging_steps 1 \
    --tf32 False \
    --model_max_length 2048 \
    --gradient_checkpointing True \
    --lazy_preprocess True \
    --report_to wandb

有人能帮帮我吗，谢谢

我解决了，作者写的的配置不对，应该改成pretra_gnn=clip_gt_arxiv

我现在修改为pretra_gnn=clip_gt_arxiv_pub，但还是报错了：

Traceback (most recent call last):
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_mem.py", line 13, in <module>
    train()
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_graph.py", line 866, in train
    model_graph_dict = model.get_model().initialize_graph_modules(
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/model/GraphLlama.py", line 145, in initialize_graph_modules
    clip_graph, args= load_model_pretrained(CLIP, self.config.pretrain_graph_model_path) 
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/model/GraphLlama.py", line 54, in load_model_pretrained
    assert osp.exists(osp.join(pretrain_model_path, 'config.json')), 'config.json missing'
AssertionError: config.json missing

我发现我的pretrain_model_path: ./graph_transformer/clip_gt_arxiv_pub，而我的GNN模型目录结构为： image

我发现train_graph.py的804行model.config.pretrain_graph_model_path = model.config.pretrain_graph_model_path + model_args.graph_tower 它似乎将pretrain_graph_model_path和pretra_gnn拼接在了一起，导致找不到config文件。 请问您知道应该如何解决吗，不胜感激！

linwhitehat commented 6 months ago 
Traceback (most recent call last):
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_mem.py", line 15, in <module>
    train()
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_graph.py", line 863, in train
    model_graph_dict = model.get_model().initialize_graph_modules(
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/model/GraphLlama.py", line 148, in initialize_graph_modules
    graph_tower.requires_grad_(False)
AttributeError: 'str' object has no attribute 'requires_grad_'

我认为可能是vicuna的config.json中的问题：

"graph_hidden_size": 128, 
"pretrain_graph_model_path": "/home/shaozhihui/szh/GraphGPT/graph_transformer/"

我的目录结构如下： image 这是我的运行命令

model_path=./vicuna-7b-v1.5-16k
instruct_ds=./data/stage_1/graph_matching.json
graph_data_path=./graph_data/all_graph_data.pt
pretra_gnn=./graph_transformer/clip_gt_arxiv_pub.pkl
output_model=./checkpoints/stage_1

wandb offline
CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.run --nnodes=1 --nproc_per_node=2 --master_port=20001 \
    graphgpt/train/train_mem.py \
    --model_name_or_path ${model_path} \
    --version v1 \
    --data_path ${instruct_ds} \
    --graph_content ./arxiv_ti_ab.json \
    --graph_data_path ${graph_data_path} \
    --graph_tower ${pretra_gnn} \
    --tune_graph_mlp_adapter True \
    --graph_select_layer -2 \
    --use_graph_start_end \
    --bf16 False \
    --output_dir ${output_model} \
    --num_train_epochs 3 \
    --per_device_train_batch_size 2 \
    --per_device_eval_batch_size 2 \
    --gradient_accumulation_steps 1 \
    --evaluation_strategy "no" \
    --save_strategy "steps" \
    --save_steps 2400 \
    --save_total_limit 1 \
    --learning_rate 2e-3 \
    --weight_decay 0. \
    --warmup_ratio 0.03 \
    --lr_scheduler_type "cosine" \
    --logging_steps 1 \
    --tf32 False \
    --model_max_length 2048 \
    --gradient_checkpointing True \
    --lazy_preprocess True \
    --report_to wandb

有人能帮帮我吗，谢谢

我解决了，作者写的的配置不对，应该改成pretra_gnn=clip_gt_arxiv

我现在修改为pretra_gnn=clip_gt_arxiv_pub，但还是报错了：

Traceback (most recent call last):
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_mem.py", line 13, in <module>
    train()
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_graph.py", line 866, in train
    model_graph_dict = model.get_model().initialize_graph_modules(
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/model/GraphLlama.py", line 145, in initialize_graph_modules
    clip_graph, args= load_model_pretrained(CLIP, self.config.pretrain_graph_model_path) 
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/model/GraphLlama.py", line 54, in load_model_pretrained
    assert osp.exists(osp.join(pretrain_model_path, 'config.json')), 'config.json missing'
AssertionError: config.json missing

我发现我的pretrain_model_path: ./graph_transformer/clip_gt_arxiv_pub，而我的GNN模型目录结构为： image

我发现train_graph.py的804行model.config.pretrain_graph_model_path = model.config.pretrain_graph_model_path + model_args.graph_tower 它似乎将pretrain_graph_model_path和pretra_gnn拼接在了一起，导致找不到config文件。 请问您知道应该如何解决吗，不胜感激！

Traceback (most recent call last):
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_mem.py", line 15, in <module>
    train()
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_graph.py", line 863, in train
    model_graph_dict = model.get_model().initialize_graph_modules(
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/model/GraphLlama.py", line 148, in initialize_graph_modules
    graph_tower.requires_grad_(False)
AttributeError: 'str' object has no attribute 'requires_grad_'

我认为可能是vicuna的config.json中的问题：

"graph_hidden_size": 128, 
"pretrain_graph_model_path": "/home/shaozhihui/szh/GraphGPT/graph_transformer/"

我的目录结构如下： image 这是我的运行命令

model_path=./vicuna-7b-v1.5-16k
instruct_ds=./data/stage_1/graph_matching.json
graph_data_path=./graph_data/all_graph_data.pt
pretra_gnn=./graph_transformer/clip_gt_arxiv_pub.pkl
output_model=./checkpoints/stage_1

wandb offline
CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.run --nnodes=1 --nproc_per_node=2 --master_port=20001 \
    graphgpt/train/train_mem.py \
    --model_name_or_path ${model_path} \
    --version v1 \
    --data_path ${instruct_ds} \
    --graph_content ./arxiv_ti_ab.json \
    --graph_data_path ${graph_data_path} \
    --graph_tower ${pretra_gnn} \
    --tune_graph_mlp_adapter True \
    --graph_select_layer -2 \
    --use_graph_start_end \
    --bf16 False \
    --output_dir ${output_model} \
    --num_train_epochs 3 \
    --per_device_train_batch_size 2 \
    --per_device_eval_batch_size 2 \
    --gradient_accumulation_steps 1 \
    --evaluation_strategy "no" \
    --save_strategy "steps" \
    --save_steps 2400 \
    --save_total_limit 1 \
    --learning_rate 2e-3 \
    --weight_decay 0. \
    --warmup_ratio 0.03 \
    --lr_scheduler_type "cosine" \
    --logging_steps 1 \
    --tf32 False \
    --model_max_length 2048 \
    --gradient_checkpointing True \
    --lazy_preprocess True \
    --report_to wandb

有人能帮帮我吗，谢谢

我解决了，作者写的的配置不对，应该改成pretra_gnn=clip_gt_arxiv

我现在修改为pretra_gnn=clip_gt_arxiv_pub，但还是报错了：

Traceback (most recent call last):
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_mem.py", line 13, in <module>
    train()
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_graph.py", line 866, in train
    model_graph_dict = model.get_model().initialize_graph_modules(
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/model/GraphLlama.py", line 145, in initialize_graph_modules
    clip_graph, args= load_model_pretrained(CLIP, self.config.pretrain_graph_model_path) 
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/model/GraphLlama.py", line 54, in load_model_pretrained
    assert osp.exists(osp.join(pretrain_model_path, 'config.json')), 'config.json missing'
AssertionError: config.json missing

我发现我的pretrain_model_path: ./graph_transformer/clip_gt_arxiv_pub，而我的GNN模型目录结构为： image

我发现train_graph.py的804行model.config.pretrain_graph_model_path = model.config.pretrain_graph_model_path + model_args.graph_tower 它似乎将pretrain_graph_model_path和pretra_gnn拼接在了一起，导致找不到config文件。 请问您知道应该如何解决吗，不胜感激！

我查阅了代码，这里不建议使用文件名，而是直接写为“pretra_gnn=clip_gt_arxiv”，它是一个目录的需求同时又固定了判断选项，同时我记得后续读取模型时会将这里的路径作为模型的目录，所以建议在设置graph_transformer作为模型目录时，新建一个命名为clip_gt_arxiv的子目录，将模型和配置拷贝一份进去。这个问题来自于作者设置的环境目录以及判断配置的代码导致的，必须是clip_gt_arxiv以及其他几个固定选项，否则无法读目录，同时目录设置下需要与这个选项关联作为最终模型目录。（细节可自行查阅代码）

tjb-tech commented 6 months ago

作为

是的，如果需要加入自己的pre train模型，可以通过修改代码来实现。我们后续会加强代码通用逻辑。感谢您的解答！

Melo-1017 commented 3 months ago

@tjb-tech @linwhitehat 两位好，我也遇到这个问题，我想了解一下vicuna的json中添加的pretrain_graph_model_path和graphgpt_stage1中的pretra_gnn是不是指同一个路径？也就是clip_gt_arxiv的路径？按照这个issue提到的做法，我在GraphGPT下直接新建了clip_gt_arxiv文件夹存放模型。 image 之后运行graphgpt_stage1，但还是提示 AttributeError: 'str' object has no attribute 'requires_grad_'，此时我的命令是：

model_path=/root/nas/models_hf/vicuna-7b-v1.5
instruct_ds=/root/nas/GraphGPT/train_instruct_graphmatch.json
graph_data_path=/root/nas/GraphGPT/graphgpt/graph_data/graph_data_all.pt
pretra_gnn=/root/nas/GraphGPT/clip_gt_arxiv
output_model=/root/nas/GraphGPT/checkpoints/stage_1

vicuna中则是："pretrain_graph_model_path": "/root/nas/GraphGPT/clip_gt_arxiv" 我若修改为pretra_gnn=clip_gt_arxiv，则报错config.json missing 请问我需要怎样配置路径来解决这个问题？

tigerbunny2023 commented 3 months ago

@tjb-tech @linwhitehat 两位好，我也遇到这个问题，我想了解一下vicuna的json中添加的pretrain_graph_model_path和graphgpt_stage1中的pretra_gnn是不是指同一个路径？也就是clip_gt_arxiv的路径？按照这个issue提到的做法，我在GraphGPT下直接新建了clip_gt_arxiv文件夹存放模型。 image 之后运行graphgpt_stage1，但还是提示 AttributeError: 'str' object has no attribute 'requires_grad_'，此时我的命令是：

model_path=/root/nas/models_hf/vicuna-7b-v1.5
instruct_ds=/root/nas/GraphGPT/train_instruct_graphmatch.json
graph_data_path=/root/nas/GraphGPT/graphgpt/graph_data/graph_data_all.pt
pretra_gnn=/root/nas/GraphGPT/clip_gt_arxiv
output_model=/root/nas/GraphGPT/checkpoints/stage_1

vicuna中则是："pretrain_graph_model_path": "/root/nas/GraphGPT/clip_gt_arxiv" 我若修改为pretra_gnn=clip_gt_arxiv，则报错config.json missing 请问我需要怎样配置路径来解决这个问题？

我遇到了一样的问题，请问你解决了吗？

Melo-1017 commented 3 months ago

@tjb-tech @linwhitehat 两位好，我也遇到这个问题，我想了解一下vicuna的json中添加的pretrain_graph_model_path和graphgpt_stage1中的pretra_gnn是不是指同一个路径？也就是clip_gt_arxiv的路径？按照这个issue提到的做法，我在GraphGPT下直接新建了clip_gt_arxiv文件夹存放模型。 image 之后运行graphgpt_stage1，但还是提示 AttributeError: 'str' object has no attribute 'requires_grad_'，此时我的命令是：

model_path=/root/nas/models_hf/vicuna-7b-v1.5
instruct_ds=/root/nas/GraphGPT/train_instruct_graphmatch.json
graph_data_path=/root/nas/GraphGPT/graphgpt/graph_data/graph_data_all.pt
pretra_gnn=/root/nas/GraphGPT/clip_gt_arxiv
output_model=/root/nas/GraphGPT/checkpoints/stage_1

vicuna中则是："pretrain_graph_model_path": "/root/nas/GraphGPT/clip_gt_arxiv" 我若修改为pretra_gnn=clip_gt_arxiv，则报错config.json missing 请问我需要怎样配置路径来解决这个问题？

我遇到了一样的问题，请问你解决了吗？

没有，我已经提了一个新的issue #61