search

scrtlabs / catalyst

An Algorithmic Trading Library for Crypto-Assets in Python
http://enigma.co
Apache License 2.0
2.49k stars 725 forks source link

Bitfinex btc_usd data corrupt #524

Open brandsimon opened 5 years ago

brandsimon commented 5 years ago

Hi,

the bitfinex data is corrupt for symbol btc_usd. On the 30.11.2018 volume is always 0, low, high, open and close is always 4321.9. On the 31.10.2018 volume is always 0, low, high, open and close are always 6330.1. On the 31.9.2018 the data varies, but it does not seam real data to me, because a lot of values are duplicates. Binance for example seams ok to me.

lenak25 commented 5 years ago

Hi @brandsimon , thanks for reporting. We will investigate this.

brandsimon commented 5 years ago

@lenak25 Is there anything new? Are other exchanges effected? I am not sure anymore that binance is ok.

brandsimon commented 5 years ago

Here is something to reproduce it:

import pandas
from catalyst import run_algorithm
from catalyst.api import symbol

def initialize(context):
    pass

def handle_data(context, data):
    print(data.history(assets=symbol('btc_usdt'),
                       fields=['high', 'low', 'close', 'volume'],
                       bar_count=26,
                       frequency='1h'))

def main():
    date = pandas.to_datetime('2018-12-01', utc=True)
    run_algorithm(
        capital_base=1000,
        data_frequency='daily',
        initialize=initialize,
        handle_data=handle_data,
        exchange_name='bitfinex',
        algo_namespace='test',
        quote_currency='usd',
        start=date,
        end=date,
        simulate_orders=True,
        live=False,
    )

if __name__ == "__main__":
    main()

Result:

[2019-01-09 23:45:57.946975] INFO: run_algo: Catalyst version 0.5.21
[2019-01-09 23:45:57.947103] WARNING: run_algo: Catalyst is currently in ALPHA. It is going through rapid development and it is subject to errors. Please use carefully. We encourage you to report any issue on GitHub: https://github.com/enigmampc/catalyst/issues
[2019-01-09 23:46:00.950276] INFO: run_algo: running algo in backtest mode
[2019-01-09 23:46:01.324120] INFO: exchange_algorithm: initialized trading algorithm in backtest mode
                            close    high     low       volume
2018-11-29 23:00:00+00:00  4321.9  4334.1  4289.1  2148.680544
2018-11-30 00:00:00+00:00  4321.9  4321.9  4321.9     0.000000
2018-11-30 01:00:00+00:00  4321.9  4321.9  4321.9     0.000000
2018-11-30 02:00:00+00:00  4321.9  4321.9  4321.9     0.000000
2018-11-30 03:00:00+00:00  4321.9  4321.9  4321.9     0.000000
2018-11-30 04:00:00+00:00  4321.9  4321.9  4321.9     0.000000
2018-11-30 05:00:00+00:00  4321.9  4321.9  4321.9     0.000000
2018-11-30 06:00:00+00:00  4321.9  4321.9  4321.9     0.000000
2018-11-30 07:00:00+00:00  4321.9  4321.9  4321.9     0.000000
2018-11-30 08:00:00+00:00  4321.9  4321.9  4321.9     0.000000
2018-11-30 09:00:00+00:00  4321.9  4321.9  4321.9     0.000000
2018-11-30 10:00:00+00:00  4321.9  4321.9  4321.9     0.000000
2018-11-30 11:00:00+00:00  4321.9  4321.9  4321.9     0.000000
2018-11-30 12:00:00+00:00  4321.9  4321.9  4321.9     0.000000
2018-11-30 13:00:00+00:00  4321.9  4321.9  4321.9     0.000000
2018-11-30 14:00:00+00:00  4321.9  4321.9  4321.9     0.000000
2018-11-30 15:00:00+00:00  4321.9  4321.9  4321.9     0.000000
2018-11-30 16:00:00+00:00  4321.9  4321.9  4321.9     0.000000
2018-11-30 17:00:00+00:00  4321.9  4321.9  4321.9     0.000000
2018-11-30 18:00:00+00:00  4321.9  4321.9  4321.9     0.000000
2018-11-30 19:00:00+00:00  4321.9  4321.9  4321.9     0.000000
2018-11-30 20:00:00+00:00  4321.9  4321.9  4321.9     0.000000
2018-11-30 21:00:00+00:00  4321.9  4321.9  4321.9     0.000000
2018-11-30 22:00:00+00:00  4321.9  4321.9  4321.9     0.000000
2018-11-30 23:00:00+00:00  4321.9  4321.9  4321.9     0.000000
2018-12-01 00:00:00+00:00  4044.6  4046.8  4038.5    11.947767
[2019-01-09 23:46:01.460757] INFO: Performance: Simulated 1 trading days out of 1.
[2019-01-09 23:46:01.460893] INFO: Performance: first open: 2018-12-01 00:00:00+00:00
[2019-01-09 23:46:01.460950] INFO: Performance: last close: 2018-12-01 23:59:00+00:00

For 2018-11-01:

[2019-01-09 23:48:10.534678] INFO: run_algo: Catalyst version 0.5.21
[2019-01-09 23:48:10.534828] WARNING: run_algo: Catalyst is currently in ALPHA. It is going through rapid development and it is subject to errors. Please use carefully. We encourage you to report any issue on GitHub: https://github.com/enigmampc/catalyst/issues
[2019-01-09 23:48:13.535153] INFO: run_algo: running algo in backtest mode
[2019-01-09 23:48:13.886231] INFO: exchange_algorithm: initialized trading algorithm in backtest mode
                                 close    high     low      volume
2018-10-30 23:00:00+00:00  6330.100000  6330.1  6315.5  143.752370
2018-10-31 00:00:00+00:00  6330.100000  6330.1  6330.1    0.000000
2018-10-31 01:00:00+00:00  6330.100000  6330.1  6330.1    0.000000
2018-10-31 02:00:00+00:00  6330.100000  6330.1  6330.1    0.000000
2018-10-31 03:00:00+00:00  6330.100000  6330.1  6330.1    0.000000
2018-10-31 04:00:00+00:00  6330.100000  6330.1  6330.1    0.000000
2018-10-31 05:00:00+00:00  6330.100000  6330.1  6330.1    0.000000
2018-10-31 06:00:00+00:00  6330.100000  6330.1  6330.1    0.000000
2018-10-31 07:00:00+00:00  6330.100000  6330.1  6330.1    0.000000
2018-10-31 08:00:00+00:00  6330.100000  6330.1  6330.1    0.000000
2018-10-31 09:00:00+00:00  6330.100000  6330.1  6330.1    0.000000
2018-10-31 10:00:00+00:00  6330.100000  6330.1  6330.1    0.000000
2018-10-31 11:00:00+00:00  6330.100000  6330.1  6330.1    0.000000
2018-10-31 12:00:00+00:00  6330.100000  6330.1  6330.1    0.000000
2018-10-31 13:00:00+00:00  6330.100000  6330.1  6330.1    0.000000
2018-10-31 14:00:00+00:00  6330.100000  6330.1  6330.1    0.000000
2018-10-31 15:00:00+00:00  6330.100000  6330.1  6330.1    0.000000
2018-10-31 16:00:00+00:00  6330.100000  6330.1  6330.1    0.000000
2018-10-31 17:00:00+00:00  6330.100000  6330.1  6330.1    0.000000
2018-10-31 18:00:00+00:00  6330.100000  6330.1  6330.1    0.000000
2018-10-31 19:00:00+00:00  6330.100000  6330.1  6330.1    0.000000
2018-10-31 20:00:00+00:00  6330.100000  6330.1  6330.1    0.000000
2018-10-31 21:00:00+00:00  6330.100000  6330.1  6330.1    0.000000
2018-10-31 22:00:00+00:00  6330.100000  6330.1  6330.1    0.000000
2018-10-31 23:00:00+00:00  6330.100000  6330.1  6330.1    0.000000
2018-11-01 00:00:00+00:00  6376.568059  6378.0  6368.4   46.470195
[2019-01-09 23:48:14.027564] INFO: Performance: Simulated 1 trading days out of 1.
[2019-01-09 23:48:14.027700] INFO: Performance: first open: 2018-11-01 00:00:00+00:00
[2019-01-09 23:48:14.027814] INFO: Performance: last close: 2018-11-01 23:59:00+00:00
lenak25 commented 5 years ago

Thanks for the information @brandsimon . Have you witnessed any issues with Binance as well? There should not be any influence between the two exchanges' data.

brandsimon commented 5 years ago

@lenak25 I am not sure there is a issue with binance. I need to look into that in detail when I have time for that.