renli1024 / SE-GNN

Semantic Evidence aware Graph Neural Network (SE-GNN) for Knowledge Graph Embedding task (AAAI'22).
59 stars 11 forks source link

您好,有关代码运行内存溢出的问题 #1

Closed dbbice closed 2 years ago

dbbice commented 2 years ago

D:\anaconda\envs\py38\python.exe C:/Users/wsco28/Desktop/SE-GNN-main/code/run.py C:/Users/wsco28/Desktop/SE-GNN-main/code/run.py:78: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_path=join('..', 'config'), config_name="config") D:\anaconda\envs\py38\lib\site-packages\hydra_internal\hydra.py:119: UserWarning: Future Hydra versions will no longer change working directory at job runtime by default. See https://hydra.cc/docs/next/upgrades/1.1_to_1.2/changes_to_job_working_dir/ for more information. ret = run_job( [2022-05-19 10:21:29,312][root][INFO] - ------Config------ {'dataset': 'FB15k_237', 'epoch': 600, 'batch_size': 128, 'learning_rate': 0.00035, 'h_dim': 450, 'pred_rel_w': False, 'label_smooth': 0.1, 'kg_layer': 2, 'rm_rate': 0.5, 'ent_drop': 0.3, 'rel_drop': 0.1, 'comp_op': 'mul', 'bn': False, 'k_h': 15, 'k_w': 30, 'conv_drop': 0.3, 'ent_drop_pred': 0.3, 'fc_drop': 0.5, 'ker_sz': 8, 'out_channel': 200, 'device': 0, 'max_no_improve': 50, 'cpu_worker_num': 10, 'warmup_epoch': 5} [2022-05-19 10:21:29,312][root][INFO] - Code dir path: C:/Users/wsco28/Desktop/SE-GNN-main/code [2022-05-19 10:21:29,312][root][INFO] - Config dir path: C:/Users/wsco28/Desktop/SE-GNN-main\config [2022-05-19 10:21:29,312][root][INFO] - Model save path: C:\Users\wsco28\Desktop\SE-GNN-main\data\output\FB15k_237\2022-05-19\10-21-29 'cp' �����ڲ����ⲿ���Ҳ���ǿ����еij��� ���������ļ��� 'cp' �����ڲ����ⲿ���Ҳ���ǿ����еij��� ���������ļ��� [2022-05-19 10:21:33,070][root][INFO] - kg # node: 14541 [2022-05-19 10:21:33,070][root][INFO] - kg # edge: 544230 [2022-05-19 10:21:33,071][root][INFO] - kg # zero deg node: 36 [2022-05-19 10:21:33,071][root][INFO] - ---Load Train Data--- [2022-05-19 10:21:35,245][root][INFO] - -----Model Parameter Configuration----- [2022-05-19 10:21:35,245][root][INFO] - Parameter ent_emb: torch.Size([14541, 450]), require_grad = True [2022-05-19 10:21:35,245][root][INFO] - Parameter pred_rel_emb: torch.Size([474, 450]), require_grad = True [2022-05-19 10:21:35,245][root][INFO] - Parameter edge_layers.0.neigh_w: torch.Size([450, 450]), require_grad = True [2022-05-19 10:21:35,245][root][INFO] - Parameter edge_layers.1.neigh_w: torch.Size([450, 450]), require_grad = True [2022-05-19 10:21:35,246][root][INFO] - Parameter node_layers.0.neigh_w: torch.Size([450, 450]), require_grad = True [2022-05-19 10:21:35,246][root][INFO] - Parameter node_layers.1.neigh_w: torch.Size([450, 450]), require_grad = True [2022-05-19 10:21:35,246][root][INFO] - Parameter comp_layers.0.neigh_w: torch.Size([450, 450]), require_grad = True [2022-05-19 10:21:35,246][root][INFO] - Parameter comp_layers.1.neigh_w: torch.Size([450, 450]), require_grad = True [2022-05-19 10:21:35,246][root][INFO] - Parameter rel_embs.0: torch.Size([474, 450]), require_grad = True [2022-05-19 10:21:35,246][root][INFO] - Parameter rel_embs.1: torch.Size([474, 450]), require_grad = True [2022-05-19 10:21:35,246][root][INFO] - Parameter predictor.bn0.weight: torch.Size([1]), require_grad = True [2022-05-19 10:21:35,246][root][INFO] - Parameter predictor.bn0.bias: torch.Size([1]), require_grad = True [2022-05-19 10:21:35,246][root][INFO] - Parameter predictor.bn1.weight: torch.Size([200]), require_grad = True [2022-05-19 10:21:35,246][root][INFO] - Parameter predictor.bn1.bias: torch.Size([200]), require_grad = True [2022-05-19 10:21:35,246][root][INFO] - Parameter predictor.bn2.weight: torch.Size([450]), require_grad = True [2022-05-19 10:21:35,246][root][INFO] - Parameter predictor.bn2.bias: torch.Size([450]), require_grad = True [2022-05-19 10:21:35,246][root][INFO] - Parameter predictor.conv.weight: torch.Size([200, 1, 8, 8]), require_grad = True [2022-05-19 10:21:35,246][root][INFO] - Parameter predictor.fc.weight: torch.Size([450, 105800]), require_grad = True [2022-05-19 10:21:35,247][root][INFO] - Training... total epoch: 600, step: 702000 C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_path=join('..', 'config'), config_name="config") C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_path=join('..', 'config'), config_name="config") C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_path=join('..', 'config'), config_name="config") C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_path=join('..', 'config'), config_name="config") C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_path=join('..', 'config'), config_name="config") C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_path=join('..', 'config'), config_name="config") C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_path=join('..', 'config'), config_name="config") C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_path=join('..', 'config'), config_name="config") C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_path=join('..', 'config'), config_name="config") C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_path=join('..', 'config'), config_name="config") Error executing job with overrides: [] Traceback (most recent call last): File "C:/Users/wsco28/Desktop/SE-GNN-main/code/run.py", line 146, in main train_log = train_step(model, batch_data, kg, optimizer, scheduler) File "C:\Users\wsco28\Desktop\SE-GNN-main\code\model_helper.py", line 30, in train_step loss.backward() File "D:\anaconda\envs\py38\lib\site-packages\torch\tensor.py", line 245, in backward torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs) File "D:\anaconda\envs\py38\lib\site-packages\torch\autograd__init__.py", line 145, in backward Variable._execution_engine.run_backward( RuntimeError: CUDA out of memory. Tried to allocate 934.00 MiB (GPU 0; 12.00 GiB total capacity; 8.53 GiB already allocated; 88.81 MiB free; 9.52 GiB reserved in total by PyTorch)

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

Process finished with exit code 1

您好,请问一下这个内存溢出应该如何解决,我调了237.yaml的bacth_size不起效果~

renli1024 commented 2 years ago

您好, 可以尝试调小h_dim参数, 该参数对显存占用影响比较大, 不过调小后性能可能会下降, 请留意下.

MiaoHu-Pro commented 2 years ago

D:\anaconda\envs\py38\python.exe C:/Users/wsco28/Desktop/SE-GNN-main/code/run.py C:/Users/wsco28/Desktop/SE-GNN-main/code/run.py:78: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_path=join('..', 'config'), config_name="config") D:\anaconda\envs\py38\lib\site-packages\hydra_internal\hydra.py:119: UserWarning: Future Hydra versions will no longer change working directory at job runtime by default. See https://hydra.cc/docs/next/upgrades/1.1_to_1.2/changes_to_job_working_dir/ for more information. ret = run_job( [2022-05-19 10:21:29,312][root][INFO] - ------Config------ {'dataset': 'FB15k_237', 'epoch': 600, 'batch_size': 128, 'learning_rate': 0.00035, 'h_dim': 450, 'pred_rel_w': False, 'label_smooth': 0.1, 'kg_layer': 2, 'rm_rate': 0.5, 'ent_drop': 0.3, 'rel_drop': 0.1, 'comp_op': 'mul', 'bn': False, 'k_h': 15, 'k_w': 30, 'conv_drop': 0.3, 'ent_drop_pred': 0.3, 'fc_drop': 0.5, 'ker_sz': 8, 'out_channel': 200, 'device': 0, 'max_no_improve': 50, 'cpu_worker_num': 10, 'warmup_epoch': 5} [2022-05-19 10:21:29,312][root][INFO] - Code dir path: C:/Users/wsco28/Desktop/SE-GNN-main/code [2022-05-19 10:21:29,312][root][INFO] - Config dir path: C:/Users/wsco28/Desktop/SE-GNN-main\config [2022-05-19 10:21:29,312][root][INFO] - Model save path: C:\Users\wsco28\Desktop\SE-GNN-main\data\output\FB15k_237\2022-05-19\10-21-29 'cp' �����ڲ����ⲿ���Ҳ���ǿ����еij��� ���������ļ��� 'cp' �����ڲ����ⲿ���Ҳ���ǿ����еij��� ���������ļ��� [2022-05-19 10:21:33,070][root][INFO] - kg # node: 14541 [2022-05-19 10:21:33,070][root][INFO] - kg # edge: 544230 [2022-05-19 10:21:33,071][root][INFO] - kg # zero deg node: 36 [2022-05-19 10:21:33,071][root][INFO] - ---Load Train Data--- [2022-05-19 10:21:35,245][root][INFO] - -----Model Parameter Configuration----- [2022-05-19 10:21:35,245][root][INFO] - Parameter ent_emb: torch.Size([14541, 450]), require_grad = True [2022-05-19 10:21:35,245][root][INFO] - Parameter pred_rel_emb: torch.Size([474, 450]), require_grad = True [2022-05-19 10:21:35,245][root][INFO] - Parameter edge_layers.0.neigh_w: torch.Size([450, 450]), require_grad = True [2022-05-19 10:21:35,245][root][INFO] - Parameter edge_layers.1.neigh_w: torch.Size([450, 450]), require_grad = True [2022-05-19 10:21:35,246][root][INFO] - Parameter node_layers.0.neigh_w: torch.Size([450, 450]), require_grad = True [2022-05-19 10:21:35,246][root][INFO] - Parameter node_layers.1.neigh_w: torch.Size([450, 450]), require_grad = True [2022-05-19 10:21:35,246][root][INFO] - Parameter comp_layers.0.neigh_w: torch.Size([450, 450]), require_grad = True [2022-05-19 10:21:35,246][root][INFO] - Parameter comp_layers.1.neigh_w: torch.Size([450, 450]), require_grad = True [2022-05-19 10:21:35,246][root][INFO] - Parameter rel_embs.0: torch.Size([474, 450]), require_grad = True [2022-05-19 10:21:35,246][root][INFO] - Parameter rel_embs.1: torch.Size([474, 450]), require_grad = True [2022-05-19 10:21:35,246][root][INFO] - Parameter predictor.bn0.weight: torch.Size([1]), require_grad = True [2022-05-19 10:21:35,246][root][INFO] - Parameter predictor.bn0.bias: torch.Size([1]), require_grad = True [2022-05-19 10:21:35,246][root][INFO] - Parameter predictor.bn1.weight: torch.Size([200]), require_grad = True [2022-05-19 10:21:35,246][root][INFO] - Parameter predictor.bn1.bias: torch.Size([200]), require_grad = True [2022-05-19 10:21:35,246][root][INFO] - Parameter predictor.bn2.weight: torch.Size([450]), require_grad = True [2022-05-19 10:21:35,246][root][INFO] - Parameter predictor.bn2.bias: torch.Size([450]), require_grad = True [2022-05-19 10:21:35,246][root][INFO] - Parameter predictor.conv.weight: torch.Size([200, 1, 8, 8]), require_grad = True [2022-05-19 10:21:35,246][root][INFO] - Parameter predictor.fc.weight: torch.Size([450, 105800]), require_grad = True [2022-05-19 10:21:35,247][root][INFO] - Training... total epoch: 600, step: 702000 C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_path=join('..', 'config'), config_name="config") C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_path=join('..', 'config'), config_name="config") C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_path=join('..', 'config'), config_name="config") C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_path=join('..', 'config'), config_name="config") C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_path=join('..', 'config'), config_name="config") C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_path=join('..', 'config'), config_name="config") C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_path=join('..', 'config'), config_name="config") C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_path=join('..', 'config'), config_name="config") C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_path=join('..', 'config'), config_name="config") C:\Users\wsco28\Desktop\SE-GNN-main\code\run.py:78: UserWarning: The version_base parameter is not specified. Please specify a compatability version level, or None. Will assume defaults for version 1.1 @hydra.main(config_path=join('..', 'config'), config_name="config") Error executing job with overrides: [] Traceback (most recent call last): File "C:/Users/wsco28/Desktop/SE-GNN-main/code/run.py", line 146, in main train_log = train_step(model, batch_data, kg, optimizer, scheduler) File "C:\Users\wsco28\Desktop\SE-GNN-main\code\model_helper.py", line 30, in train_step loss.backward() File "D:\anaconda\envs\py38\lib\site-packages\torch\tensor.py", line 245, in backward torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs) File "D:\anaconda\envs\py38\lib\site-packages\torch\autogradinit.py", line 145, in backward Variable._execution_engine.run_backward( RuntimeError: CUDA out of memory. Tried to allocate 934.00 MiB (GPU 0; 12.00 GiB total capacity; 8.53 GiB already allocated; 88.81 MiB free; 9.52 GiB reserved in total by PyTorch)

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

Process finished with exit code 1

您好,请问一下这个内存溢出应该如何解决,我调了237.yaml的bacth_size不起效果~

@dbbice 同学您好,请问你的运行环境是什么样的,能提供一下 requirements 文件吗?我这一直因为环境问题报错。