Thanks for your great work on online anomalous trajectory detection. I am very interested in it. I also noticed that you updated your latest code in this repository. Thanks again for your effort in this community.
I have some points that I do not understand very well. It would be great if you can give me a guide.
First, there are three steps in the repository, which are pretrain, train and evaluation. I went through your code, and found that outliers are only injected in the evaluation step (sampler.inject_outlers in run_evaluate method in run_loop.py, line 163). I want to confirm if there is any misunderstanding from me to your paper since the paper mentions that 'Note that the proportion of the injected anomalies in the training data would not largely affect the performance of our methods.' . Do you also inject outliers in pretrain and train steps?
Second, I want to reproduce the results in table IV in your paper. I tried with d=3,alpha=0.1,observed ratio p=1.0 for GM-VSAE10, the PR-AUC is 0.976 in my running. I also tired with d=3,alpha=0.1,observed ratio p=0.1 for GM-VSAE10, the PR-AUC is still around 0.97 in my trying. The results are quite different from your results in Table IV. So I want to seek your help to check if there is anything wrong with the way that I reproduce your code. My way of reproducing d=3,alpha=0.1,observed ratio p=1.0 for GM-VSAE10 is to change the parameters as sampler.inject_outliers(otype='random', data_type=eval_data, ratio=0.05,
level=3, point_prob=0.1, vary=False) in line 163 of run_loop.py and I also checked that observed ratio p is 1.0 in line 67 and cluster_num is 10 in line 47, then run pretrain.sh, train.sh, eval.sh one by one. But the results from my reproduction are quite different from your paper. Could you please point out the steps that I am wrong in the reproducing way? Many thanks for your kind help.
BruceLuke
commented
1 year ago
Dear Yiding,
Thanks for your great work on online anomalous trajectory detection. I am very interested in it. I also noticed that you updated your latest code in this repository. Thanks again for your effort in this community.
I have some points that I do not understand very well. It would be great if you can give me a guide.
First, there are three steps in the repository, which are pretrain, train and evaluation. I went through your code, and found that outliers are only injected in the evaluation step (sampler.inject_outlers in run_evaluate method in run_loop.py, line 163). I want to confirm if there is any misunderstanding from me to your paper since the paper mentions that 'Note that the proportion of the injected anomalies in the training data would not largely affect the performance of our methods.' . Do you also inject outliers in pretrain and train steps?
Second, I want to reproduce the results in table IV in your paper. I tried with d=3,alpha=0.1,observed ratio p=1.0 for GM-VSAE10, the PR-AUC is 0.976 in my running. I also tired with d=3,alpha=0.1,observed ratio p=0.1 for GM-VSAE10, the PR-AUC is still around 0.97 in my trying. The results are quite different from your results in Table IV. So I want to seek your help to check if there is anything wrong with the way that I reproduce your code. My way of reproducing d=3,alpha=0.1,observed ratio p=1.0 for GM-VSAE10 is to change the parameters as sampler.inject_outliers(otype='random', data_type=eval_data, ratio=0.05, level=3, point_prob=0.1, vary=False) in line 163 of run_loop.py and I also checked that observed ratio p is 1.0 in line 67 and cluster_num is 10 in line 47, then run pretrain.sh, train.sh, eval.sh one by one. But the results from my reproduction are quite different from your paper. Could you please point out the steps that I am wrong in the reproducing way? Many thanks for your kind help.
Best, Xiaolin