search

zcakhaa / DeepLOB-Deep-Convolutional-Neural-Networks-for-Limit-Order-Books

This jupyter notebook is used to demonstrate our recent work, "DeepLOB: Deep Convolutional Neural Networks for Limit Order Books", published in IEEE Transactions on Singal Processing. We use FI-2010 dataset and present how model architecture is constructed here. The FI-2010 is publicly avilable and interested readers can check out their paper.
374 stars 205 forks source link

The label Extraction #11

Closed HaishuoFang closed 3 years ago

HaishuoFang commented 3 years ago

Hello,

I have a question about the labelling? When you label data, you use the next k timesteps avg price / current price. Are k timesteps here natural timesteps or the next k events(some changes happening in LOB)?

Thank you

HaishuoFang commented 3 years ago

Another question is about input, What is the meaning of the most 100 updates? Does it mean the most 100 events

zcakhaa commented 3 years ago

It is the tick time (event) not clock time. Yes, the most 100 events.

From: Haishuo @.> Sent: Thursday, April 22, 2021 9:40 AM To: zcakhaa/DeepLOB-Deep-Convolutional-Neural-Networks-for-Limit-Order-Books @.> Cc: Subscribed @.***> Subject: Re: [zcakhaa/DeepLOB-Deep-Convolutional-Neural-Networks-for-Limit-Order-Books] The label Extraction (#11)

Another question is about input, What is the meaning of the most 100 updates? Does it mean the most 100 events

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/zcakhaa/DeepLOB-Deep-Convolutional-Neural-Networks-for-Limit-Order-Books/issues/11#issuecomment-824694337, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFN66THG4NKX2LE5OXN5APDTJ7VH7ANCNFSM43MCQGOA.

HaishuoFang commented 3 years ago

Thank you for your reply! I am adapting your model to my dataset. I have 800,000 events per day and one-month data, which is a huge dataset. It is really time-consuming to run one epoch. The weird thing is f1 of class "down" is zero, which means it cannot learn anything information about class "down" and loss don't coverage. I'll inspect it.

I just wonder if you have similar experience before? Thank you

zcakhaa commented 3 years ago

I haven't experienced it. If the loss does not improve on training data, there must be anything wrong with the setup.

From: Haishuo @.> Sent: Friday, April 23, 2021 11:00 AM To: zcakhaa/DeepLOB-Deep-Convolutional-Neural-Networks-for-Limit-Order-Books @.> Cc: Zihao Zhang @.>; Comment @.> Subject: Re: [zcakhaa/DeepLOB-Deep-Convolutional-Neural-Networks-for-Limit-Order-Books] The label Extraction (#11)

Thank you for your reply! I am adapting your model to my dataset. I have 800,000 events per day and one-month data, which is a huge dataset. It is really time-consuming to run one epoch. The weird thing is f1 of class "down" is zero, which means it cannot learn anything information about class "down" and loss don't coverage. I'll inspect it.

I just wonder if you have similar experience before? Thank you

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/zcakhaa/DeepLOB-Deep-Convolutional-Neural-Networks-for-Limit-Order-Books/issues/11#issuecomment-825579474, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFN66TACLV6ZJEAHTPB2W3LTKFHNJANCNFSM43MCQGOA.

HaishuoFang commented 3 years ago

Thanks for your reply. I solved this problem by adding the BatchNorm layer. It makes the model more robust and easy to train

zcakhaa commented 3 years ago

Great. You can also try different activation functions. Sometimes, ReLU or Leaky-ReLU would lead to unstable training for time-series.

From: Haishuo @.> Sent: Wednesday, April 28, 2021 7:56 PM To: zcakhaa/DeepLOB-Deep-Convolutional-Neural-Networks-for-Limit-Order-Books @.> Cc: Zihao Zhang @.>; Comment @.> Subject: Re: [zcakhaa/DeepLOB-Deep-Convolutional-Neural-Networks-for-Limit-Order-Books] The label Extraction (#11)

Thanks for your reply. I solved this problem by adding the BatchNorm layer. It makes the model more robust and easy to train

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/zcakhaa/DeepLOB-Deep-Convolutional-Neural-Networks-for-Limit-Order-Books/issues/11#issuecomment-828736720, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFN66TD3FJQDQZF75TYZHZ3TLBSALANCNFSM43MCQGOA.

HaishuoFang commented 3 years ago

The final question: what is the frequency of high-frequency trading? I got 0.017 seconds lob data, which is quite large. I downsampled it on half-minute. I wonder if the frequency of half-minute is too low? Do you know what is the suitable time interval in high-frequency trading? Thank you

zcakhaa commented 3 years ago

It depends on your setup. For doing market-making, you probably need to work with the raw tick time without downsampling. If you are paying for the spread, you can downsample it to half-minute or minute.

From: Haishuo @.> Sent: Wednesday, April 28, 2021 9:55 PM To: zcakhaa/DeepLOB-Deep-Convolutional-Neural-Networks-for-Limit-Order-Books @.> Cc: Zihao Zhang @.>; Comment @.> Subject: Re: [zcakhaa/DeepLOB-Deep-Convolutional-Neural-Networks-for-Limit-Order-Books] The label Extraction (#11)

The final question: what is the frequency of high-frequency trading? I got 0.017 seconds lob data, which is quite large. I downsampled it on half-minute. I wonder if the frequency of half-minute is too low? Do you know what is the suitable time interval in high-frequency trading? Thank you

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/zcakhaa/DeepLOB-Deep-Convolutional-Neural-Networks-for-Limit-Order-Books/issues/11#issuecomment-828806440, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFN66TEVROPZD3EZCXVFVPDTLB76HANCNFSM43MCQGOA.

HaishuoFang commented 3 years ago

Thanks for your reply. Does it make sense to compare the performance of different sample frequency? It seems the only difference between them is long-term or short-term price movements

zcakhaa commented 3 years ago

As I mentioned that it depends on your setup. If you do market-making, you are earning spread instead of paying for it. If you need to pay for the spread, you probably need a lower frequency so the price movement is big enough to cover the costs.

From: Haishuo @.> Sent: Thursday, April 29, 2021 8:35 AM To: zcakhaa/DeepLOB-Deep-Convolutional-Neural-Networks-for-Limit-Order-Books @.> Cc: Zihao Zhang @.>; Comment @.> Subject: Re: [zcakhaa/DeepLOB-Deep-Convolutional-Neural-Networks-for-Limit-Order-Books] The label Extraction (#11)

Thanks for your reply. Does it make sense to compare the performance of different sample frequency? It seems the only difference between them is long-term or short-term price movements

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/zcakhaa/DeepLOB-Deep-Convolutional-Neural-Networks-for-Limit-Order-Books/issues/11#issuecomment-829045945, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFN66TBH3WJPH6ND3LMVHTTTLEK37ANCNFSM43MCQGOA.

HaishuoFang commented 3 years ago

Thanks for your patience. Got it!