amirmk89 / gepc

Graph Embedded Pose Clustering for Anomaly Detection
Other
76 stars 18 forks source link

Some question about the experimental results #5

Open pangwenfeng opened 3 years ago

pangwenfeng commented 3 years ago

Thanks for your nice work first! I run the code without any change, but I cannot get the similar result on ShanghaiTech as the paper post, which is lower than the results shown in the paper. The related parameters are as follows (all parameters are set as default): Namespace(act='relu', ae_batch_size=512, ae_epochs=10, ae_fn=None, ae_lr=0.0001, ae_lr_decay=0.99, ae_optimizer='adam', ae_sched='tri', ae_test_every=20, ae_weight_decay=1e-05, alpha=0.001, ckpt_dir='data/exp_dir/stc/Dec14_1059/checkpoints/', conv_oper='sagc', data_dir='data/', dcec_batch_size=512, dcec_epochs=25, dcec_fn=None, dcec_lr=0.0008, dcec_lr_decay=0.98, dcec_optimizer='adam', dcec_sched='tri', dcec_weight_decay=1e-05, debug=False, device='cuda:0', dpmm_fn=None, dropout=0.3, exp_dir='data/exp_dir', gamma=0.6, headless=False, k_init_batch=4, k_init_downsample=1, n_clusters=10, norm_scale=0, num_transform=5, num_workers=8, optimizer='adam', patch_db=False, patch_features=False, patch_size=16, pose_path={'train': 'data/pose/training/tracked_person/', 'test': 'data/pose/testing/tracked_person/'}, pretrain_epochs=0, prop_norm_scale=0, res_batch_size=256, save_results=1, seed=14061439651021546021, seg_len=12, seg_stride=8, train_seg_conf_th=0.0, update_interval=2.0, verbose=1, vid_path={'train': 'data/training/videos/', 'test': 'data/testing/frames/'})

I run the code twice and got the AUC of 73.29%, 72.84%, respectively.

img1 img2

I cannot find the experiment setting details such as learning rate, batch size and sampling method in the paper, so If I want to get the results shown in the paper, what should I change to the current code? or did I make any mistakes in the reimplementation? Thank you very much!

amirmk89 commented 3 years ago

Hi, Thank you for your question. It may depend on the environment settings, as the two results you got are also very different from one another, and in my experience results from several runs are often closer. You can also try increasing the 'train_seg_conf_th' parameter.

On Mon, Dec 14, 2020, 15:06 庞 notifications@github.com wrote:

Thanks for your nice work first! I run the code without any change, but I cannot get the similar result on ShanghaiTech as the paper post, which is lower than the results shown in the paper. The related parameters are as follows (all parameters are set as default): Namespace(act='relu', ae_batch_size=512, ae_epochs=10, ae_fn=None, ae_lr=0.0001, ae_lr_decay=0.99, ae_optimizer='adam', ae_sched='tri', ae_test_every=20, ae_weight_decay=1e-05, alpha=0.001, ckpt_dir='data/exp_dir/stc/Dec14_1059/checkpoints/', conv_oper='sagc', data_dir='data/', dcec_batch_size=512, dcec_epochs=25, dcec_fn=None, dcec_lr=0.0008, dcec_lr_decay=0.98, dcec_optimizer='adam', dcec_sched='tri', dcec_weight_decay=1e-05, debug=False, device='cuda:0', dpmm_fn=None, dropout=0.3, exp_dir='data/exp_dir', gamma=0.6, headless=False, k_init_batch=4, k_init_downsample=1, n_clusters=10, norm_scale=0, num_transform=5, num_workers=8, optimizer='adam', patch_db=False, patch_features=False, patch_size=16, pose_path={'train': 'data/pose/training/tracked_person/', 'test': 'data/pose/testing/tracked_person/'}, pretrain_epochs=0, prop_norm_scale=0, res_batch_size=256, save_results=1, seed=14061439651021546021, seg_len=12, seg_stride=8, train_seg_conf_th=0.0, update_interval=2.0, verbose=1, vid_path={'train': 'data/training/videos/', 'test': 'data/testing/frames/'})

I run the code twice and got the AUC of 73.29%, 72.84%, respectively. I cannot find the experiment setting details such as learning rate, batch size and sampling method in the paper, so If I want to get the results shown in the paper, what should I change to the current code? Thank you very much!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/amirmk89/gepc/issues/5, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFQGYUM5G55FTM3F3JNOI3SUYETTANCNFSM4U2W2FRQ .

pangwenfeng commented 3 years ago

Thanks for your reply! To ensure that I have same environment as yours, I created a new environment with the environment.txt file by conda (By the way, the command to create environment is incorrect, and correct command should be: conda create --name gpec --file environment.txt). However, I got the AUC of 73.52%, which is lower than the result in the paper. I also tried to increase the 'train_seg_conf_th' parameter, and get an AUC of 74.28%. So what else should I do? Should I set a specific value for random seed or any other parameter? Thank you very much.