search

quantopian / zipline

Zipline, a Pythonic Algorithmic Trading Library
https://www.zipline.io
Apache License 2.0
17.69k stars 4.73k forks source link

zipline NYSE calendar date issue - 2015-12-24 is missing #2227

Closed ayxemma closed 6 years ago

ayxemma commented 6 years ago

Dear Zipline Maintainers,

Before I tell you about my issue, let me describe my environment:

Environment

Now that you know a little about me, let me tell you about the issue I am having:

Description of Issue

it is supposed that the variable records each trading day under NYSE calendar (the calendar i set in TradingAlgorithm as:
perf = TradingAlgorithm(
    env=env,
    trading_calendar=get_calendar("NYSE"),
    sim_params=create_simulation_parameters(
        start=start,
        end=end,
        capital_base=CAPITAL_BASE,
        trading_calendar=get_calendar("NYSE"),
        data_frequency='daily'
    ),
    **{
        'initialize': initialize,
        'handle_data': handle_data,

'before_trading_start': before_trading,

        'analyze': analyze,
    }
).run(data, overwrite_sim_params=False)

* What happened instead?
but for date 2015-12-24   0:00:00, it returns 2015-12-23, which is not right:

  | tday
-- | --
2015-12-17   0:00:00 | 2015-12-17
2015-12-18   0:00:00 | 2015-12-18
2015-12-21   0:00:00 | 2015-12-21
2015-12-22   0:00:00 | 2015-12-22
2015-12-23   0:00:00 | 2015-12-23
2015-12-24   0:00:00 | 2015-12-23
2015-12-28   0:00:00 | 2015-12-28
2015-12-29   0:00:00 | 2015-12-29
2015-12-30   0:00:00 | 2015-12-30
2015-12-31   0:00:00 | 2015-12-31

--

--

Here is how you can reproduce this issue on your machine:

## Reproduction Steps

just create an algo where at each day you record the date as above.
...

## What steps have you taken to resolve this already?
i tried placing an order on 12/23/2015, since i use daily data, the order is executed on 12/24/2015, so it means the date does exsit i think, just somehow when using the function to get it `context.tday=get_datetime().date()` it does not reflect correctly.
...

...

Sincerely,
ccbttn
ssanderson commented 6 years ago

hi @ayxemma!

I can't reproduce this issue on master with the following script:

# repro.py

import pandas as pd

from zipline import run_algorithm
from zipline.api import get_datetime, record

def initialize(context):
    pass

def before_trading_start(context, data):
    record(tday=get_datetime())

result = run_algorithm(
    start=pd.Timestamp('2014-12-01', tz='UTC'),
    end=pd.Timestamp('2015-01-02', tz='UTC'),
    initialize=initialize,
    capital_base=10000,
    before_trading_start=before_trading_start,
    bundle='quandl',
)

print result.tday

(Note I ran this after running zipline ingest -b quandl with an API key exported.)

For me, running this script produces the following:

$ python repro.py
2014-12-01 21:00:00+00:00   2014-12-01 13:45:00+00:00
2014-12-02 21:00:00+00:00   2014-12-02 13:45:00+00:00
2014-12-03 21:00:00+00:00   2014-12-03 13:45:00+00:00
2014-12-04 21:00:00+00:00   2014-12-04 13:45:00+00:00
2014-12-05 21:00:00+00:00   2014-12-05 13:45:00+00:00
2014-12-08 21:00:00+00:00   2014-12-08 13:45:00+00:00
2014-12-09 21:00:00+00:00   2014-12-09 13:45:00+00:00
2014-12-10 21:00:00+00:00   2014-12-10 13:45:00+00:00
2014-12-11 21:00:00+00:00   2014-12-11 13:45:00+00:00
2014-12-12 21:00:00+00:00   2014-12-12 13:45:00+00:00
2014-12-15 21:00:00+00:00   2014-12-15 13:45:00+00:00
2014-12-16 21:00:00+00:00   2014-12-16 13:45:00+00:00
2014-12-17 21:00:00+00:00   2014-12-17 13:45:00+00:00
2014-12-18 21:00:00+00:00   2014-12-18 13:45:00+00:00
2014-12-19 21:00:00+00:00   2014-12-19 13:45:00+00:00
2014-12-22 21:00:00+00:00   2014-12-22 13:45:00+00:00
2014-12-23 21:00:00+00:00   2014-12-23 13:45:00+00:00
2014-12-24 18:00:00+00:00   2014-12-24 13:45:00+00:00
2014-12-26 21:00:00+00:00   2014-12-26 13:45:00+00:00
2014-12-29 21:00:00+00:00   2014-12-29 13:45:00+00:00
2014-12-30 21:00:00+00:00   2014-12-30 13:45:00+00:00
2014-12-31 21:00:00+00:00   2014-12-31 13:45:00+00:00
2015-01-02 21:00:00+00:00   2015-01-02 13:45:00+00:00
Name: tday, dtype: datetime64[ns, UTC]

You'll notice that the recorded variable entry for 2014-12-24 shows up with a timestamp that's 3 hours earlier than the other days, because the day before christmas is a half day, but otherwise that day works just like a normal trading day.

Are you by any chance using schedule_function to schedule your record calls? If so, are you passing half_days=False? That would cause your function to not run on the 24th, which would cause us to forward-fill the recorded values from the previous day when reporting results.

ayxemma commented 6 years ago

Thanks for the reply. i didn't put half_days=False in my code, but i figured out that the problem is that in the if __name__ == '__main__': function previously i comment out 'before_trading_start': before_trading,, it still runs before_trading, but after uncommenting it, the calendar does not have any problem.

    perf = TradingAlgorithm(
        env=env,
        trading_calendar=get_calendar("NYSE"),
        sim_params=create_simulation_parameters(
            start=start,
            end=end,
            capital_base=CAPITAL_BASE,
            trading_calendar=get_calendar("NYSE"),
            data_frequency='daily'
        ),
        **{
            'initialize': initialize,
            'handle_data': handle_data,
            'before_trading_start': before_trading,
            'analyze': analyze,
        }
    ).run(data, overwrite_sim_params=False)