search

warren-oneill / powerline

Extension to the zipline library for the German EPEX and EEX energy markets
Other
7 stars 1 forks source link

Wrong data in handle_data #57

Open dhexus opened 8 years ago

dhexus commented 8 years ago

EexExchange gives wrong data. For instance, when self.datetime is Timestamp('2015-06-02 14:00:00+0000', tz='UTC'), I get

BarData({0: SIDData({'_freqstr': None, 'sid': 0, 'type': 4, '_initial_len': 3, 'price': 27.5, 'dt': Timestamp('2013-01-07 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 0}), 1: SIDData({'_freqstr': None, 'sid': 1, 'type': 4, '_initial_len': 3, 'price': 49.1, 'dt': Timestamp('2013-01-14 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 1}), 2: SIDData({'_freqstr': None, 'sid': 2, 'type': 4, '_initial_len': 3, 'price': 48.77, 'dt': Timestamp('2013-01-21 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 2}), 3: SIDData({'_freqstr': None, 'sid': 3, 'type': 4, '_initial_len': 3, 'price': 48.68, 'dt': Timestamp('2013-01-28 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 3}), 4: SIDData({'_freqstr': None, 'sid': 4, 'type': 4, '_initial_len': 3, 'price': 33.13, 'dt': Timestamp('2013-02-04 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 4}), 5: SIDData({'_freqstr': None, 'sid': 5, 'type': 4, '_initial_len': 3, 'price': 41.08, 'dt': Timestamp('2013-02-11 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 5}), 6: SIDData({'_freqstr': None, 'sid': 6, 'type': 4, '_initial_len': 3, 'price': 45.78, 'dt': Timestamp('2013-02-18 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 6}), 7: SIDData({'_freqstr': None, 'sid': 7, 'type': 4, '_initial_len': 3, 'price': 45.73, 'dt': Timestamp('2013-02-25 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 7}), 8: SIDData({'_freqstr': None, 'sid': 8, 'type': 4, '_initial_len': 3, 'price': 50.5, 'dt': Timestamp('2013-03-04 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 8}), 9: SIDData({'_freqstr': None, 'sid': 9, 'type': 4, '_initial_len': 3, 'price': 38.03, 'dt': Timestamp('2013-03-11 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 9}), 10: SIDData({'_freqstr': None, 'sid': 10, 'type': 4, '_initial_len': 3, 'price': 39.44, 'dt': Timestamp('2013-03-18 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 10}), 11: SIDData({'_freqstr': None, 'sid': 11, 'type': 4, '_initial_len': 3, 'price': 37.01, 'dt': Timestamp('2013-03-25 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 11}), 12: SIDData({'_freqstr': None, 'sid': 12, 'type': 4, '_initial_len': 3, 'price': 39.88, 'dt': Timestamp('2013-04-02 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 12}), 13: SIDData({'_freqstr': None, 'sid': 13, 'type': 4, '_initial_len': 3, 'price': 42.5, 'dt': Timestamp('2013-04-08 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 13}), 14: SIDData({'_freqstr': None, 'sid': 14, 'type': 4, '_initial_len': 3, 'price': 45.02, 'dt': Timestamp('2013-04-15 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 14}), 15: SIDData({'_freqstr': None, 'sid': 15, 'type': 4, '_initial_len': 3, 'price': 32.17, 'dt': Timestamp('2013-04-22 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 15}), 16: SIDData({'_freqstr': None, 'sid': 16, 'type': 4, '_initial_len': 3, 'price': 32.48, 'dt': Timestamp('2013-04-29 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B5', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 16}), 17: SIDData({'_freqstr': None, 'sid': 17, 'type': 4, '_initial_len': 3, 'price': 29.05, 'dt': Timestamp('2013-05-06 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 17}), 18: SIDData({'_freqstr': None, 'sid': 18, 'type': 4, '_initial_len': 3, 'price': 27.91, 'dt': Timestamp('2013-05-13 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 18}), 19: SIDData({'_freqstr': None, 'sid': 19, 'type': 4, '_initial_len': 3, 'price': 30.28, 'dt': Timestamp('2013-05-21 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 19}), 20: SIDData({'_freqstr': None, 'sid': 20, 'type': 4, '_initial_len': 3, 'price': 35.92, 'dt': Timestamp('2013-05-27 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 20}), 21: SIDData({'_freqstr': None, 'sid': 21, 'type': 4, '_initial_len': 3, 'price': 34.7, 'dt': Timestamp('2013-06-03 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 21}), 22: SIDData({'_freqstr': None, 'sid': 22, 'type': 4, '_initial_len': 3, 'price': 31.19, 'dt': Timestamp('2013-06-10 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 22}), 23: SIDData({'_freqstr': None, 'sid': 23, 'type': 4, '_initial_len': 3, 'price': 23.37, 'dt': Timestamp('2013-06-17 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 23}), 24: SIDData({'_freqstr': None, 'sid': 24, 'type': 4, '_initial_len': 3, 'price': 28.67, 'dt': Timestamp('2013-06-24 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 24}), 25: SIDData({'_freqstr': None, 'sid': 25, 'type': 4, '_initial_len': 3, 'price': 30.48, 'dt': Timestamp('2013-07-01 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 25}), 26: SIDData({'_freqstr': None, 'sid': 26, 'type': 4, '_initial_len': 3, 'price': 32.87, 'dt': Timestamp('2013-07-08 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 26}), 27: SIDData({'_freqstr': None, 'sid': 27, 'type': 4, '_initial_len': 3, 'price': 32.45, 'dt': Timestamp('2013-07-15 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 27}), 28: SIDData({'_freqstr': None, 'sid': 28, 'type': 4, '_initial_len': 3, 'price': 37.94, 'dt': Timestamp('2013-07-22 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 28}), 29: SIDData({'_freqstr': None, 'sid': 29, 'type': 4, '_initial_len': 3, 'price': 41.37, 'dt': Timestamp('2013-07-29 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B5', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 29}), 30: SIDData({'_freqstr': None, 'sid': 30, 'type': 4, '_initial_len': 3, 'price': 36.38, 'dt': Timestamp('2013-08-05 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 30}), 31: SIDData({'_freqstr': None, 'sid': 31, 'type': 4, '_initial_len': 3, 'price': 35.07, 'dt': Timestamp('2013-08-12 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 31}), 32: SIDData({'_freqstr': None, 'sid': 32, 'type': 4, '_initial_len': 3, 'price': 32.99, 'dt': Timestamp('2013-08-19 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 32}), 33: SIDData({'_freqstr': None, 'sid': 33, 'type': 4, '_initial_len': 3, 'price': 41.88, 'dt': Timestamp('2013-08-26 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 33}), 34: SIDData({'_freqstr': None, 'sid': 34, 'type': 4, '_initial_len': 3, 'price': 43.58, 'dt': Timestamp('2013-09-02 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 34}), 35: SIDData({'_freqstr': None, 'sid': 35, 'type': 4, '_initial_len': 3, 'price': 41.15, 'dt': Timestamp('2013-09-09 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 35}), 36: SIDData({'_freqstr': None, 'sid': 36, 'type': 4, '_initial_len': 3, 'price': 43.55, 'dt': Timestamp('2013-09-16 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 36}), 37: SIDData({'_freqstr': None, 'sid': 37, 'type': 4, '_initial_len': 3, 'price': 40.69, 'dt': Timestamp('2013-09-23 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 37}), 38: SIDData({'_freqstr': None, 'sid': 38, 'type': 4, '_initial_len': 3, 'price': 44.21, 'dt': Timestamp('2013-09-30 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B5', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 38}), 39: SIDData({'_freqstr': None, 'sid': 39, 'type': 4, '_initial_len': 3, 'price': 36.94, 'dt': Timestamp('2013-10-07 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 39}), 40: SIDData({'_freqstr': None, 'sid': 40, 'type': 4, '_initial_len': 3, 'price': 45.88, 'dt': Timestamp('2013-10-14 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 40}), 41: SIDData({'_freqstr': None, 'sid': 41, 'type': 4, '_initial_len': 3, 'price': 37.85, 'dt': Timestamp('2013-10-21 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 41}), 42: SIDData({'_freqstr': None, 'sid': 42, 'type': 4, '_initial_len': 3, 'price': 33.53, 'dt': Timestamp('2013-10-28 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 42}), 43: SIDData({'_freqstr': None, 'sid': 43, 'type': 4, '_initial_len': 3, 'price': 28.36, 'dt': Timestamp('2013-11-04 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 43}), 44: SIDData({'_freqstr': None, 'sid': 44, 'type': 4, '_initial_len': 3, 'price': 32.03, 'dt': Timestamp('2013-11-11 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 44}), 45: SIDData({'_freqstr': None, 'sid': 45, 'type': 4, '_initial_len': 3, 'price': 43.25, 'dt': Timestamp('2013-11-18 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 45}), 46: SIDData({'_freqstr': None, 'sid': 46, 'type': 4, '_initial_len': 3, 'price': 46.1, 'dt': Timestamp('2013-11-25 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 46}), 47: SIDData({'_freqstr': None, 'sid': 47, 'type': 4, '_initial_len': 3, 'price': 40.29, 'dt': Timestamp('2013-12-02 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 47}), 48: SIDData({'_freqstr': None, 'sid': 48, 'type': 4, '_initial_len': 3, 'price': 41.48, 'dt': Timestamp('2013-12-09 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 48}), 49: SIDData({'_freqstr': None, 'sid': 49, 'type': 4, '_initial_len': 3, 'price': 49.8, 'dt': Timestamp('2013-12-16 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 49}), 50: SIDData({'_freqstr': None, 'sid': 50, 'type': 4, '_initial_len': 3, 'price': 33.72, 'dt': Timestamp('2013-12-23 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 50}), 51: SIDData({'_freqstr': None, 'sid': 51, 'type': 4, '_initial_len': 3, 'price': 21.27, 'dt': Timestamp('2013-12-30 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B5', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 51}), 52: SIDData({'_freqstr': None, 'sid': 52, 'type': 4, '_initial_len': 3, 'price': 25.6, 'dt': Timestamp('2014-01-06 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 52}), 53: SIDData({'_freqstr': None, 'sid': 53, 'type': 4, '_initial_len': 3, 'price': 27.41, 'dt': Timestamp('2014-01-13 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 53}), 54: SIDData({'_freqstr': None, 'sid': 54, 'type': 4, '_initial_len': 3, 'price': 39.12, 'dt': Timestamp('2014-01-20 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 54}), 55: SIDData({'_freqstr': None, 'sid': 55, 'type': 4, '_initial_len': 3, 'price': 44.58, 'dt': Timestamp('2014-01-27 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 55}), 56: SIDData({'_freqstr': None, 'sid': 56, 'type': 4, '_initial_len': 3, 'price': 38.48, 'dt': Timestamp('2014-02-03 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 56}), 57: SIDData({'_freqstr': None, 'sid': 57, 'type': 4, '_initial_len': 3, 'price': 33.95, 'dt': Timestamp('2014-02-10 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 57}), 58: SIDData({'_freqstr': None, 'sid': 58, 'type': 4, '_initial_len': 3, 'price': 30.43, 'dt': Timestamp('2014-02-17 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 58}), 59: SIDData({'_freqstr': None, 'sid': 59, 'type': 4, '_initial_len': 3, 'price': 34.61, 'dt': Timestamp('2014-02-24 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 59}), 60: SIDData({'_freqstr': None, 'sid': 60, 'type': 4, '_initial_len': 3, 'price': 36.03, 'dt': Timestamp('2014-03-03 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 60}), 61: SIDData({'_freqstr': None, 'sid': 61, 'type': 4, '_initial_len': 3, 'price': 35.29, 'dt': Timestamp('2014-03-10 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 61}), 62: SIDData({'_freqstr': None, 'sid': 62, 'type': 4, '_initial_len': 3, 'price': 26.71, 'dt': Timestamp('2014-03-17 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 62}), 63: SIDData({'_freqstr': None, 'sid': 63, 'type': 4, '_initial_len': 3, 'price': 27.75, 'dt': Timestamp('2014-03-24 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 63}), 64: SIDData({'_freqstr': None, 'sid': 64, 'type': 4, '_initial_len': 3, 'price': 33.6, 'dt': Timestamp('2014-03-31 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B5', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 64}), 65: SIDData({'_freqstr': None, 'sid': 65, 'type': 4, '_initial_len': 3, 'price': 34.22, 'dt': Timestamp('2014-04-07 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 65}), 66: SIDData({'_freqstr': None, 'sid': 66, 'type': 4, '_initial_len': 3, 'price': 30.59, 'dt': Timestamp('2014-04-14 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 66}), 67: SIDData({'_freqstr': None, 'sid': 67, 'type': 4, '_initial_len': 3, 'price': 27.19, 'dt': Timestamp('2014-04-22 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 67}), 68: SIDData({'_freqstr': None, 'sid': 68, 'type': 4, '_initial_len': 3, 'price': 31.81, 'dt': Timestamp('2014-04-28 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 68}), 69: SIDData({'_freqstr': None, 'sid': 69, 'type': 4, '_initial_len': 3, 'price': 30.8, 'dt': Timestamp('2014-05-05 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 69}), 70: SIDData({'_freqstr': None, 'sid': 70, 'type': 4, '_initial_len': 3, 'price': 26.08, 'dt': Timestamp('2014-05-12 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 70}), 71: SIDData({'_freqstr': None, 'sid': 71, 'type': 4, '_initial_len': 3, 'price': 32.16, 'dt': Timestamp('2014-05-19 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 71}), 72: SIDData({'_freqstr': None, 'sid': 72, 'type': 4, '_initial_len': 3, 'price': 34.88, 'dt': Timestamp('2014-05-26 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 72}), 73: SIDData({'_freqstr': None, 'sid': 73, 'type': 4, '_initial_len': 3, 'price': 31.4, 'dt': Timestamp('2014-06-02 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 73}), 74: SIDData({'_freqstr': None, 'sid': 74, 'type': 4, '_initial_len': 3, 'price': 32.93, 'dt': Timestamp('2014-06-09 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 74}), 75: SIDData({'_freqstr': None, 'sid': 75, 'type': 4, '_initial_len': 3, 'price': 29.96, 'dt': Timestamp('2014-06-16 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 75}), 76: SIDData({'_freqstr': None, 'sid': 76, 'type': 4, '_initial_len': 3, 'price': 28.86, 'dt': Timestamp('2014-06-23 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 76}), 77: SIDData({'_freqstr': None, 'sid': 77, 'type': 4, '_initial_len': 3, 'price': 35.48, 'dt': Timestamp('2014-06-30 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B5', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 77}), 78: SIDData({'_freqstr': None, 'sid': 78, 'type': 4, '_initial_len': 3, 'price': 31.35, 'dt': Timestamp('2014-07-07 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 78}), 79: SIDData({'_freqstr': None, 'sid': 79, 'type': 4, '_initial_len': 3, 'price': 30.99, 'dt': Timestamp('2014-07-14 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 79}), 80: SIDData({'_freqstr': None, 'sid': 80, 'type': 4, '_initial_len': 3, 'price': 31.99, 'dt': Timestamp('2014-07-21 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 80}), 81: SIDData({'_freqstr': None, 'sid': 81, 'type': 4, '_initial_len': 3, 'price': 32.47, 'dt': Timestamp('2014-07-28 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 81}), 82: SIDData({'_freqstr': None, 'sid': 82, 'type': 4, '_initial_len': 3, 'price': 29.49, 'dt': Timestamp('2014-08-04 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 82}), 83: SIDData({'_freqstr': None, 'sid': 83, 'type': 4, '_initial_len': 3, 'price': 28.15, 'dt': Timestamp('2014-08-11 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 83}), 84: SIDData({'_freqstr': None, 'sid': 84, 'type': 4, '_initial_len': 3, 'price': 23.94, 'dt': Timestamp('2014-08-18 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 84}), 85: SIDData({'_freqstr': None, 'sid': 85, 'type': 4, '_initial_len': 3, 'price': 27.75, 'dt': Timestamp('2014-08-25 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 85}), 86: SIDData({'_freqstr': None, 'sid': 86, 'type': 4, '_initial_len': 3, 'price': 33.69, 'dt': Timestamp('2014-09-01 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 86}), 87: SIDData({'_freqstr': None, 'sid': 87, 'type': 4, '_initial_len': 3, 'price': 32.23, 'dt': Timestamp('2014-09-08 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 87}), 88: SIDData({'_freqstr': None, 'sid': 88, 'type': 4, '_initial_len': 3, 'price': 34.84, 'dt': Timestamp('2014-09-15 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 88}), 89: SIDData({'_freqstr': None, 'sid': 89, 'type': 4, '_initial_len': 3, 'price': 36.03, 'dt': Timestamp('2014-09-22 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 89}), 90: SIDData({'_freqstr': None, 'sid': 90, 'type': 4, '_initial_len': 3, 'price': 33.71, 'dt': Timestamp('2014-09-29 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B5', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 90}), 91: SIDData({'_freqstr': None, 'sid': 91, 'type': 4, '_initial_len': 3, 'price': 38.96, 'dt': Timestamp('2014-10-06 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 91}), 92: SIDData({'_freqstr': None, 'sid': 92, 'type': 4, '_initial_len': 3, 'price': 33.92, 'dt': Timestamp('2014-10-13 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 92}), 93: SIDData({'_freqstr': None, 'sid': 93, 'type': 4, '_initial_len': 3, 'price': 35.61, 'dt': Timestamp('2014-10-20 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 93}), 94: SIDData({'_freqstr': None, 'sid': 94, 'type': 4, '_initial_len': 3, 'price': 31.74, 'dt': Timestamp('2014-10-27 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 94}), 95: SIDData({'_freqstr': None, 'sid': 95, 'type': 4, '_initial_len': 3, 'price': 34.09, 'dt': Timestamp('2014-11-03 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 95}), 96: SIDData({'_freqstr': None, 'sid': 96, 'type': 4, '_initial_len': 3, 'price': 34.08, 'dt': Timestamp('2014-11-10 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 96}), 97: SIDData({'_freqstr': None, 'sid': 97, 'type': 4, '_initial_len': 3, 'price': 36.42, 'dt': Timestamp('2014-11-17 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 97}), 98: SIDData({'_freqstr': None, 'sid': 98, 'type': 4, '_initial_len': 3, 'price': 41.17, 'dt': Timestamp('2014-11-24 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 98}), 99: SIDData({'_freqstr': None, 'sid': 99, 'type': 4, '_initial_len': 3, 'price': 38.22, 'dt': Timestamp('2014-12-01 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 99}), 100: SIDData({'_freqstr': None, 'sid': 100, 'type': 4, '_initial_len': 3, 'price': 46.97, 'dt': Timestamp('2014-12-08 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 100}), 101: SIDData({'_freqstr': None, 'sid': 101, 'type': 4, '_initial_len': 3, 'price': 35.84, 'dt': Timestamp('2014-12-15 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 101}), 102: SIDData({'_freqstr': None, 'sid': 102, 'type': 4, '_initial_len': 3, 'price': 27.74, 'dt': Timestamp('2014-12-22 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 102}), 103: SIDData({'_freqstr': None, 'sid': 103, 'type': 4, '_initial_len': 3, 'price': 19.51, 'dt': Timestamp('2014-12-29 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B5', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 103}), 104: SIDData({'_freqstr': None, 'sid': 104, 'type': 4, '_initial_len': 3, 'price': 23.02, 'dt': Timestamp('2015-01-05 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 104}), 105: SIDData({'_freqstr': None, 'sid': 105, 'type': 4, '_initial_len': 3, 'price': 25.02, 'dt': Timestamp('2015-01-12 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 105}), 106: SIDData({'_freqstr': None, 'sid': 106, 'type': 4, '_initial_len': 3, 'price': 27.36, 'dt': Timestamp('2015-01-19 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 106}), 107: SIDData({'_freqstr': None, 'sid': 107, 'type': 4, '_initial_len': 3, 'price': 39.84, 'dt': Timestamp('2015-01-26 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 107}), 108: SIDData({'_freqstr': None, 'sid': 108, 'type': 4, '_initial_len': 3, 'price': 31.73, 'dt': Timestamp('2015-02-02 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 108}), 109: SIDData({'_freqstr': None, 'sid': 109, 'type': 4, '_initial_len': 3, 'price': 36.98, 'dt': Timestamp('2015-02-09 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 109}), 110: SIDData({'_freqstr': None, 'sid': 110, 'type': 4, '_initial_len': 3, 'price': 38.46, 'dt': Timestamp('2015-02-16 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 110}), 111: SIDData({'_freqstr': None, 'sid': 111, 'type': 4, '_initial_len': 3, 'price': 36.35, 'dt': Timestamp('2015-02-23 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 111}), 112: SIDData({'_freqstr': None, 'sid': 112, 'type': 4, '_initial_len': 3, 'price': 32.43, 'dt': Timestamp('2015-03-02 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 112}), 113: SIDData({'_freqstr': None, 'sid': 113, 'type': 4, '_initial_len': 3, 'price': 29.65, 'dt': Timestamp('2015-03-09 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 113}), 114: SIDData({'_freqstr': None, 'sid': 114, 'type': 4, '_initial_len': 3, 'price': 35.33, 'dt': Timestamp('2015-03-16 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 114}), 115: SIDData({'_freqstr': None, 'sid': 115, 'type': 4, '_initial_len': 3, 'price': 32.67, 'dt': Timestamp('2015-03-23 15:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 115}), 116: SIDData({'_freqstr': None, 'sid': 116, 'type': 4, '_initial_len': 3, 'price': 34.8, 'dt': Timestamp('2015-03-30 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B5', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 116}), 117: SIDData({'_freqstr': None, 'sid': 117, 'type': 4, '_initial_len': 3, 'price': 23.19, 'dt': Timestamp('2015-04-07 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 117}), 118: SIDData({'_freqstr': None, 'sid': 118, 'type': 4, '_initial_len': 3, 'price': 29.26, 'dt': Timestamp('2015-04-13 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 118}), 119: SIDData({'_freqstr': None, 'sid': 119, 'type': 4, '_initial_len': 3, 'price': 28.92, 'dt': Timestamp('2015-04-20 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 119}), 120: SIDData({'_freqstr': None, 'sid': 120, 'type': 4, '_initial_len': 3, 'price': 30.93, 'dt': Timestamp('2015-04-27 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 120}), 121: SIDData({'_freqstr': None, 'sid': 121, 'type': 4, '_initial_len': 3, 'price': 29.42, 'dt': Timestamp('2015-05-04 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 121}), 122: SIDData({'_freqstr': None, 'sid': 122, 'type': 4, '_initial_len': 3, 'price': 23.01, 'dt': Timestamp('2015-05-11 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 122}), 123: SIDData({'_freqstr': None, 'sid': 123, 'type': 4, '_initial_len': 3, 'price': 24.08, 'dt': Timestamp('2015-05-18 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 123}), 124: SIDData({'_freqstr': None, 'sid': 124, 'type': 4, '_initial_len': 3, 'price': 28.99, 'dt': Timestamp('2015-05-26 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 124}), 125: SIDData({'_freqstr': None, 'sid': 125, 'type': 4, '_initial_len': 3, 'price': 26.74, 'dt': Timestamp('2015-06-01 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 125}), 126: SIDData({'_freqstr': None, 'sid': 126, 'type': 4, '_initial_len': 3, 'price': 25.96, 'dt': Timestamp('2015-06-02 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B2', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 126}), 127: SIDData({'_freqstr': None, 'sid': 127, 'type': 4, '_initial_len': 3, 'price': 29.37, 'dt': Timestamp('2015-06-02 14:00:00+0000', tz='UTC'), 'volume': 15456, 'product': 'F1B3', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 127}), 128: SIDData({'_freqstr': None, 'sid': 128, 'type': 4, '_initial_len': 3, 'price': 29.25, 'dt': Timestamp('2015-06-02 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B4', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 128}), 129: SIDData({'_freqstr': None, 'sid': 129, 'type': 4, '_initial_len': 3, 'price': 29.25, 'dt': Timestamp('2015-06-02 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B5', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 129}), 130: SIDData({'_freqstr': None, 'sid': 130, 'type': 4, '_initial_len': 3, 'price': 29.13, 'dt': Timestamp('2015-06-02 14:00:00+0000', tz='UTC'), 'volume': 0, 'product': 'F1B1', 'source_id': 'EexSource-55a814932bf2aa0d4ded760a7522cab0', '_sid': 130})})
warren-oneill commented 8 years ago

Can you write the code that you are running please

dhexus commented 8 years ago

Implementation of handle_data could be anything. Setup looks like

    start=pd.Timestamp("2013-01-01 00:00:00+0000", tz="UTC")
    end=pd.Timestamp("2016-01-01 00:00:00+0000", tz="UTC")

    exchange = EexExchange(
        start=start,
        end=end,
        products=["F1B1", "F1B2", "F1B3", "F1B4", "F1B5"])

    exchange.env.write_data(futures_data=exchange.asset_metadata)

    data_source = exchange.source

    sim_params = create_simulation_parameters(
        start=data_source.start,
        end=data_source.end)

    algo = SomeAlgorithm(
        instant_fill=instant_fill,
        env=exchange.env,
        sim_params=sim_params,
        data_frequency="minute"
    )

    res = algo.run(data_source)
warren-oneill commented 8 years ago

ok i see it now. I saw it before but thought it was normal zipline behavior. I'll look into it next week.

dhexus commented 8 years ago

ok thanks

warren-oneill commented 8 years ago

I think I spoke to James about it and he said that all data gets delivered even the old ones but that doesn't really make sense.

dhexus commented 8 years ago

I think this is new and for some reason it breaks FIT. Delivering old prices for past products via handle_data really makes no sense. In case of EEX, data object should contain prices for currently delivered and future products. More generally, delivering prices for tradable products and products that just have traded makes sense, but price history should be managed in another way.