warren-oneill / powerline

Extension to the zipline library for the German EPEX and EEX energy markets
Other
7 stars 1 forks source link

Adding batches of data does not work #65

Closed maxkoe closed 8 years ago

maxkoe commented 8 years ago

Using container.update(BarData, TimeStamp) does not work properly with BarData, which contains more than one entry. All but the last entry for an individual 'market' are ignored.

Minimal example

import numpy as np
import pandas as pd

from zipline.history.history import HistorySpec
from zipline.protocol import BarData

from gg.powerline.history.history_container import EpexHistoryContainer
from gg.powerline.exchanges.epex_exchange import EpexExchange

start_date = pd.Timestamp('2015-06-25', tz='Europe/Berlin').tz_convert('UTC')
end_date = pd.Timestamp('2015-06-28', tz='Europe/Berlin').tz_convert('UTC')
source_start = start_date - pd.Timedelta(hours=2)

exchange = EpexExchange(start=start_date, end=end_date)
env = exchange.env

asset_metadata = {}
env.write_data(futures_data=asset_metadata)

history_spec = HistorySpec(bar_count=2, frequency='1m', field='price',
                           ffill=False, data_frequency='minute', env=env)
history_specs = {}
history_specs[history_spec.key_str] = history_spec

container = EpexHistoryContainer(history_specs, None, source_start, 'minute',
                                 env=env)

data = {}
data[1] = {'dt': pd.Timestamp('2015-06-28'),
           'price': np.random.uniform(0, 100),
           'market': 'epex_auction',
           'product': '00-01',
           'day': pd.Timestamp('2015-06-27'),
           'sid': 1}
data[2] = {'dt': pd.Timestamp('2015-06-28'),
           'price': np.random.uniform(0, 100),
           'market': 'epex_auction',
           'product': '01-02',
           'day': pd.Timestamp('2015-06-27'),
           'sid': 2}

bar = BarData(data)
container.update(bar, pd.Timestamp('2015-06-28'))

history = container.get_history()

print(history['epex_auction'])
maxkoe commented 8 years ago

A fix has been comitted to branch history_tests.

maxkoe commented 8 years ago

Improved styling in the bugfix via commit 10bd16d

maxkoe commented 8 years ago

@warren-oneill : please have a look at the fix and close the issue if appropriate.