Dear Zipline Maintainers,

Before I tell you about my issue, let me describe my environment: I am a beginner on Zipline and need help on non US stock data ingestion.

MacOs Zipline 1.3.0 Custom CSV Bundle Non US Equities Custom Trading Calendar Conda Environment installation

Now that you know a little about me, let me tell you about the issue I am having: I am a beginner on Zipline and need help on non US stock data ingestion.

Description

I need to make a single Sunday(30-10-2016) as a Trading day. I have registered this trading Calendar as in calendar_utils.py and have no problem with it. I have even got the adhoc_holidays() working.

This is the Trading calendar for Indian Stock market. India has few regualr holidays which fall on the same day every year and many holidays are irregular. Based on the same list of irregular holidays since Oct 2016 i have made my custom trading calendar. Yes, i am now dealing with data after October 2016 to current date only.

I want to point to special_opens where the issue is. I have a data in my csv for 30-10-2016 as it was a special trading day on Sunday.

Below is my NSEExchange trading calendar.

from datetime import time

from pandas._libs.tslib import Timestamp
from pandas.tseries.holiday import Holiday
from pytz import timezone
from trading_calendars.trading_calendar import TradingCalendar, HolidayCalendar
from itertools import chain

Chritmas = Holiday(
        "Christmas",
        month=12,
        day=25
)

Ambedkar_Jayanthi = Holiday(
    "Ambedkar_Jayanthi",
    month=4,
    day=14
)

Gandhi_Jayanthi = Holiday(
    "Gandhi_Jayanthi",
    month=10,
    day=2
)

India_Independence = Holiday(
    "India_Independence",
    month=8,
    day=15
)

May_Day = Holiday(
    "May_Day",
    month=5,
    day=1
)

India_Republic = Holiday(
    "India_Republic",
    month=1,
    day=26
)

OtherHolidays = [
    Timestamp('2016-03-07', tz='Asia/Kolkata'),
    Timestamp('2016-03-24', tz='Asia/Kolkata'),
    Timestamp('2016-03-25', tz='Asia/Kolkata'),
    Timestamp('2016-04-15', tz='Asia/Kolkata'),
    Timestamp('2016-04-19', tz='Asia/Kolkata'),
    Timestamp('2016-07-06', tz='Asia/Kolkata'),
    Timestamp('2016-09-05', tz='Asia/Kolkata'),
    Timestamp('2016-09-13', tz='Asia/Kolkata'),
    Timestamp('2016-10-11', tz='Asia/Kolkata'),
    Timestamp('2016-10-12', tz='Asia/Kolkata'),
    Timestamp('2016-10-31', tz='Asia/Kolkata'),
    Timestamp('2016-11-14', tz='Asia/Kolkata'),
    Timestamp('2017-02-24', tz='Asia/Kolkata'),
    Timestamp('2017-03-13', tz='Asia/Kolkata'),
    Timestamp('2017-04-04', tz='Asia/Kolkata'),
    Timestamp('2017-06-26', tz='Asia/Kolkata'),
    Timestamp('2017-08-25', tz='Asia/Kolkata'),
    Timestamp('2017-10-20', tz='Asia/Kolkata'),
    Timestamp('2018-02-13', tz='Asia/Kolkata'),
    Timestamp('2018-03-02', tz='Asia/Kolkata'),
    Timestamp('2018-03-29', tz='Asia/Kolkata'),
    Timestamp('2018-03-30', tz='Asia/Kolkata'),
    Timestamp('2018-08-22', tz='Asia/Kolkata'),
    Timestamp('2018-09-13', tz='Asia/Kolkata'),
    Timestamp('2018-09-20', tz='Asia/Kolkata'),
    Timestamp('2018-10-18', tz='Asia/Kolkata'),
    Timestamp('2018-11-07', tz='Asia/Kolkata'),
    Timestamp('2018-11-08', tz='Asia/Kolkata'),
    Timestamp('2018-11-23', tz='Asia/Kolkata')
]

SpecialOpen = [
    Timestamp('2016-10-30', tz='Asia/Kolkata')
]

class NSEExchangeCalendar(TradingCalendar):
    """
    Exchange calendar for NSE

    Open Time: 9:15 AM, Asia/Kolkata
    Close Time: 3:30 PM, Asia/Kolkata

    """

    regular_early_close = time(15, 30)

    @property
    def name(self):
        return "NSE"

    @property
    def tz(self):
        return timezone('Asia/Kolkata')

    @property
    def open_time(self):
        return time(9, 15)

    @property
    def close_time(self):
        return time(15,30)

    @property
    def regular_holidays(self):
        return HolidayCalendar([
            Chritmas,
            Ambedkar_Jayanthi,
            Gandhi_Jayanthi,
            India_Independence,
            May_Day,
            India_Republic
        ])

    @property
    def adhoc_holidays(self):
        return list(chain(
            OtherHolidays
        ))

    @property
    def special_opens(self):
        return list(chain(
            SpecialOpen
        ))

This is the valid exception which i get. #helpwanted.

(/Users/praj3/anaconda3/envs/ZipPallavi) bash-3.2$ zipline ingest -b csvdir
<string>:22: UserWarning: Overwriting bundle with name 'csvdir'
Loading custom pricing data:   [############------------------------]   33% | DWARIKESH: sid 0
Merging daily equity files:  [------------------------------------]  0
Traceback (most recent call last):
  File "/Users/praj3/anaconda3/envs/ZipPallavi/bin/zipline", line 11, in <module>
    load_entry_point('zipline==1.3.0', 'console_scripts', 'zipline')()
  File "/Users/praj3/anaconda3/envs/ZipPallavi/lib/python3.5/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/Users/praj3/anaconda3/envs/ZipPallavi/lib/python3.5/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/praj3/anaconda3/envs/ZipPallavi/lib/python3.5/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/praj3/anaconda3/envs/ZipPallavi/lib/python3.5/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/praj3/anaconda3/envs/ZipPallavi/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/praj3/anaconda3/envs/ZipPallavi/lib/python3.5/site-packages/zipline/__main__.py", line 348, in ingest
    show_progress,
  File "/Users/praj3/anaconda3/envs/ZipPallavi/lib/python3.5/site-packages/zipline/data/bundles/core.py", line 451, in ingest
    pth.data_path([name, timestr], environ=environ),
  File "/Users/praj3/anaconda3/envs/ZipPallavi/lib/python3.5/site-packages/zipline/data/bundles/csvdir.py", line 94, in ingest
    self.csvdir)
  File "/Users/praj3/anaconda3/envs/ZipPallavi/lib/python3.5/site-packages/zipline/data/bundles/csvdir.py", line 156, in csvdir_bundle
    show_progress=show_progress)
  File "/Users/praj3/anaconda3/envs/ZipPallavi/lib/python3.5/site-packages/zipline/data/us_equity_pricing.py", line 257, in write
    return self._write_internal(it, assets)
  File "/Users/praj3/anaconda3/envs/ZipPallavi/lib/python3.5/site-packages/zipline/data/us_equity_pricing.py", line 378, in _write_internal
    ).difference(asset_sessions).tolist(),
AssertionError: Got 497 rows for daily bars table with first day=2016-10-20, last day=2018-10-19, expected 496 rows.
Missing sessions: []
Extra sessions: [Timestamp('2016-10-30 00:00:00+0000', tz='UTC')]

Edit: I even tried adding

@property
    def special_opens_adhoc(self):
        return [(time(9, 15), ['2016-10-30'])]

But this too throws the same exception that, This is not a Trading day.

I am missing something in my special_opens, not sure how to get this working. #help

Sincerely, Punith

quantopian / zipline

How to make a only Sunday(30-10-2016) as a Trading Day #2337

Description

This is the valid exception which i get. #helpwanted.

helpwanted #pleasehelp