AI4Finance-Foundation / FinRL-Meta

FinRL­-Meta: Dynamic datasets and market environments for FinRL.
https://ai4finance.org
MIT License
1.21k stars 567 forks source link

[Suggestion] Feature helper / ideas #27

Open cryptocoinserver opened 2 years ago

cryptocoinserver commented 2 years ago

Candle information (might help the model learning candlestick patterns):

def lower_shadow(df): return np.minimum(df['Close'], df['Open']) - df['Low']

def real_body(df):

maybe np.abs()

return df['Close'] - df['Open']

def full_length(df):

maybe np.abs()

return df['High'] - df['Low']

**Seasonality information** (See this great video, where this was recommended - [https://www.youtube.com/watch?v=c0gpgCyjTM8](https://www.youtube.com/watch?v=c0gpgCyjTM8)):
For example...
- day of week
- day of month
- month
- year
- week of year
- hour

**Volatility Features:**
- Realized 
- Parkinson 
- Garman-Klass 
- Roger-Satchell 
- Garman-Klass-Yang-Zhang
- Yang-Zhang

Source: [https://www.kaggle.com/yamqwe/crypto-prediction-volatility-features#Feature-Engineering-%F0%9F%94%AC](https://www.kaggle.com/yamqwe/crypto-prediction-volatility-features#Feature-Engineering-%F0%9F%94%AC) - this kaggle competition is a good source for more ideas. 

**TSfresh**
[https://tsfresh.readthedocs.io/en/latest/text/list_of_features.html](https://tsfresh.readthedocs.io/en/latest/text/list_of_features.html) Great tool to easily add (and filter) a lot more valuable features.
rayrui312 commented 2 years ago

Thanks for your valuable suggestions. We will try to add support for these features in the future version.

YangletLiu commented 2 years ago

@cryptocoinserver Hi, thanks. Would you be able to help us add some of them?

cryptocoinserver commented 2 years ago

Sadly, my time is pretty limited currently due to university and other project(s). If I'm not mistaken, you can just use the Volatility Features code from the kaggle notebook I linked: "This Notebook has been released under the Apache 2.0 open source license.".

And the seasonality stuff is easy, as pandas has those functions built in: https://pandas.pydata.org/docs/reference/api/pandas.Series.dt.dayofweek.html?highlight=dayofweek#pandas.Series.dt.dayofweek

I saw some people asking how to help on Slack. Maybe you could add some tasks to https://github.com/AI4Finance-Foundation/FinRL-Meta/projects for (those) contributors to work through?

cryptocoinserver commented 2 years ago

Might be an interesting feature to test: https://www.kaggle.com/wptouxx/g-research-cointegration-analysis Cointegration tests for all asset pairs and obtaining their p-values.