search

ruiking04 / COCA

Deep Contrastive One-Class Time Series Anomaly Detection
29 stars 9 forks source link

COCA - NoCL #27

Open marciahon29 opened 7 months ago

marciahon29 commented 7 months ago

Please could you let me know wht is the formula for COCA-NoCL. To me, I understand it as being equation (3.3) without Lsim.

In your paper you state: "NoCL removes the contastive learning of COCA to optimize the similarity of representations and one-class center." Please could you expand this? I do not fully understand.

ruiking04 commented 7 months ago

COCA-NoCL means only One-class. Its optimization goal is to minimize the Euclidean distance between representation Q and center O. In the case of vector normalization, it is equivalent to making the cos similarity of Q and O large.

image

code

marciahon29 commented 7 months ago

Thank you.

The reason I am asking is that for my dataset, COCA-NoCL performed the best. I was expecting COCA. I don't understand why I get such a result. Would you have any ideas?

Do you z-normalize the datasets? Or is this already done by UCR?

ruiking04 commented 7 months ago

What metric is used? Is it RPA F1?

Are the hyperparameters of COCA and COCA-NoCL the same, for example:

        self.kernel_size = 8
        self.stride = 1
        self.final_out_channels = 64
        self.hidden_size = 128
        self.num_layers = 3
        self.project_channels = 32
        self.num_epoch = 50
        self.freeze_length_epoch = 10
        self.change_center_epoch = 10
        self.center_eps = 0.1

I think COCA and COCA-NoCL can be set to the same hyperparameters. If that doesn't work, you can only look for some new hyperparameters. Deep model hyperparameter tuning is indeed a very hard task.

marciahon29 commented 7 months ago

All the hyperparameters are the same because I used : UCR_Configs.py

On Thu, Feb 1, 2024 at 10:13 PM RuiKing @.***> wrote:

What metric is used? Is it RPA F1?

Are the hyperparameters of COCA and COCA-NoCL the same, for example:

    self.kernel_size = 8
    self.stride = 1
    self.final_out_channels = 64
    self.hidden_size = 128
    self.num_layers = 3
    self.project_channels = 32
    self.num_epoch = 50
    self.freeze_length_epoch = 10
    self.change_center_epoch = 10
    self.center_eps = 0.1

I think COCA and COCA-NoCL can be set to the same hyperparameters. If that doesn't work, you can only look for some new hyperparameters. Deep model parameter tuning is indeed a very hard task.

— Reply to this email directly, view it on GitHub https://github.com/ruiking04/COCA/issues/27#issuecomment-1922730948, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFOIJPSP3SXNC735BUOZVFDYRRKW3AVCNFSM6AAAAABCPXOPWKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRSG4ZTAOJUHA . You are receiving this because you authored the thread.Message ID: @.***>

marciahon29 commented 7 months ago

I would like to again do a Randomized Grid Search to find the optimum hyperparameters.

Please could you let me know what variables are acceptable for the following that I should test:

    self.input_channels = 1
    self.kernel_size = 8
    self.stride = 1
    self.final_out_channels = 64
    self.hidden_size = 128
    self.num_layers = 3
    self.project_channels = 32
    self.dropout = 0.45
    self.features_len = 18
    self.time_step = 4 #random.choice([1,2,4,8,16,32]) #4
    self.window_size = 64 #random.choice([16,32,64,128]) #64
    self.num_epoch = 50
    self.freeze_length_epoch = 10
    self.change_center_epoch = 10
    self.center_eps = 0.1
    self.omega1 = 1
    self.omega2 = 0.1
    self.beta1 = 0.9
    self.beta2 = 0.99
    self.lr = 3e-4
    self.drop_last = False
    self.nu = 0.01
    self.detect_nu = 0.0005
    self.threshold_determine = 'one-anomaly'
    self.loss_type = 'distance'
    self.scale_ratio = 0.7

random.choice([0.6,0.7,0.8,0.9,1.0,1.1,1.2,1.3]) #0.8

    self.jitter_ratio = 0.1

random.choice([0.01,0.05,0.1,0.2,0.3,0.4,0.5]) #0.2

For times_step I have tried : 1,2,4,5,16,32. window_size : 16, 32, 64, 128. batch_size : 256, 512, 512. scale_ratio .6/7/8/9/10/11/12/13. jitter_ratio : 001/005/01/02/03/04/05

Thanks,

Marcia

On Fri, Feb 2, 2024 at 8:09 AM Marcia Hon @.***> wrote:

All the hyperparameters are the same because I used : UCR_Configs.py

On Thu, Feb 1, 2024 at 10:13 PM RuiKing @.***> wrote:

What metric is used? Is it RPA F1?

Are the hyperparameters of COCA and COCA-NoCL the same, for example:

    self.kernel_size = 8
    self.stride = 1
    self.final_out_channels = 64
    self.hidden_size = 128
    self.num_layers = 3
    self.project_channels = 32
    self.num_epoch = 50
    self.freeze_length_epoch = 10
    self.change_center_epoch = 10
    self.center_eps = 0.1

I think COCA and COCA-NoCL can be set to the same hyperparameters. If that doesn't work, you can only look for some new hyperparameters. Deep model parameter tuning is indeed a very hard task.

— Reply to this email directly, view it on GitHub https://github.com/ruiking04/COCA/issues/27#issuecomment-1922730948, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFOIJPSP3SXNC735BUOZVFDYRRKW3AVCNFSM6AAAAABCPXOPWKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRSG4ZTAOJUHA . You are receiving this because you authored the thread.Message ID: @.***>

marciahon29 commented 7 months ago

Hello,

I have solved the issue by doing a Randomized Grid Search based on the parameters: time_step, window_size, num_epoch, batch_size, scale_ratio, and jitter_ratio. According to the literature review, higher num_epoch and batch_size give more accuracy.

I used the following values:

time_step: 1,2,4,8,16,32 window_size: 16,32,64,128 num_epoch: 100, 200, 300, ..., 800, 900, 1000 batch_size: 256,512,1024,2048,096,8192 scale_ratio: 0.6,0.7,0.8,0.9,1.0,1.1,1.2,1.3 jitter_ratio: 0.001,0.05,0.1,0.2,0.3,0.4,.05

My accuracy COCA accuracy jumped from ~65% to 95%

Thanks, Marcia