search

Alvin-Zeng / PGCN

Graph Convolutional Networks for Temporal Action Localization (ICCV2019)
319 stars 66 forks source link

Some error during training, please help #26

Closed gauravcr7rm closed 4 years ago

gauravcr7rm commented 4 years ago

pickle.dump([self.act_iou_dict, self.act_dis_dict, self.prop_dict], open(self.prop_dict_path, "wb")) MemoryError

Alvin-Zeng commented 4 years ago

Could you provide more details about the error?

gauravcr7rm commented 4 years ago

(learnpytorch) C:\Users\gaura\OneDrive\Documents\GitHub\pgcn>python pgcn_train.py thumos14 --snapshot_pre /Users/gaura/OneDrive/Desktop/BHU_Intern 2020-05-15 19:33:15,384 INFO creating folder: /Users/gaura/OneDrive/Desktop/BHU_Intern |===>Backups using time: 2.321 s |===>Backups will be saved at /Users/gaura/OneDrive/Desktop/BHU_Intern 2020-05-15 19:33:17,723 INFO runtime args

Namespace(batch_size=32, clip_gradient=None, comp_loss_weight=0.5, dataset='thumos14', dataset_configs=ordereddict([('train_ft_path', '/Users/gaura/OneDrive/Desktop/BHU_Intern/Flow_Train_All'), ('test_ft_path', '/Users/gaura/OneDrive/Desktop/BHU_Intern/Flow_Test_All'), ('train_dict_path', 'data/thumos14_train_prop_dict.pkl'), ('val_dict_path', 'data/thumos14_val_prop_dict.pkl'), ('test_dict_path', 'data/thumos14_test_prop_dict.pkl'), ('train_prop_file', 'data/bsn_train_proposal_list.txt'), ('test_prop_file', 'data/bsn_test_proposal_list.txt'), ('training_epoch_multiplier', 20), ('testing_epoch_multiplier', 1), ('fg_iou_thresh', 0.7), ('bg_iou_thresh', 0.01), ('incomplete_iou_thresh', 0.3), ('bg_coverage_thresh', 0.02), ('incomplete_overlap_thresh', 0.01), ('prop_per_video', 8), ('fg_ratio', 1), ('bg_ratio', 1), ('incomplete_ratio', 6), ('iou_threshold', 0.7), ('dis_threshold', 0), ('starting_ratio', 0.5), ('ending_ratio', 0.5)]), dropout=0.8, epochs=70, eval_freq=5, evaluate=False, evaluation=ordereddict([('top_k', 2000), ('nms_threshold', 0.2)]), gpus=None, graph_configs=ordereddict([('adj_num', 21), ('child_num', 4), ('iou_num', 8), ('dis_num', 2)]), iter_size=1, lr=0.01, lr_steps=[15, 30, 45], model_configs=ordereddict([('num_class', 20), ('act_feat_dim', 1024), ('comp_feat_dim', 3072), ('dropout', 0.8), ('gcn_dropout', 0.7)]), momentum=0.9, print_freq=20, reg_loss_weight=0.5, resume='', snapshot_pref='/Users/gaura/OneDrive/Desktop/BHU_Intern', start_epoch=0, training_epoch_multiplier=10, weight_decay=0.0005, workers=4)

config

ordereddict([('train_ft_path', '/Users/gaura/OneDrive/Desktop/BHU_Intern/Flow_Train_All'), ('test_ft_path', '/Users/gaura/OneDrive/Desktop/BHU_Intern/Flow_Test_All'), ('train_dict_path', 'data/thumos14_train_prop_dict.pkl'), ('val_dict_path', 'data/thumos14_val_prop_dict.pkl'), ('test_dict_path', 'data/thumos14_test_prop_dict.pkl'), ('train_prop_file', 'data/bsn_train_proposal_list.txt'), ('test_prop_file', 'data/bsn_test_proposal_list.txt'), ('training_epoch_multiplier', 20), ('testing_epoch_multiplier', 1), ('fg_iou_thresh', 0.7), ('bg_iou_thresh', 0.01), ('incomplete_iou_thresh', 0.3), ('bg_coverage_thresh', 0.02), ('incomplete_overlap_thresh', 0.01), ('prop_per_video', 8), ('fg_ratio', 1), ('bg_ratio', 1), ('incomplete_ratio', 6), ('iou_threshold', 0.7), ('dis_threshold', 0), ('starting_ratio', 0.5), ('ending_ratio', 0.5)]) File parsed. Time:12.89 100%|################################################################################| 199/199 [08:36<00:00, 2.60s/it] Dict constructed. Time:519.94 Traceback (most recent call last): File "pgcn_train.py", line 369, in main() File "pgcn_train.py", line 84, in main test_mode=False), File "C:\Users\gaura\OneDrive\Documents\GitHub\pgcn\pgcn_dataset.py", line 201, in init pickle.dump([self.act_iou_dict, self.act_dis_dict, self.prop_dict], open(self.prop_dict_path, "wb")) MemoryError

Alvin-Zeng commented 4 years ago

You can monitor the usage of CPU memory. This error seems to be related to the usage of memory. Make sure that you have enough memory to save the dictionary.