Custom bundles and calendars - Can we disable calendar check?

[AndreasClenow]

Hi guys,

Enforcing data alignment to a calendar can be useful for some purposes, but not always. Currently it would seem as if custom bundle ingest will fail if there is the slightest date mismatch.

If a date is missing for a stock, the ingest will fail completely throwing an 'AssertionError'. But in reality, dates could be missing. The stock could have been trade stopped, or perhaps there simply was no trade in that stock on that date.

Padding data, i.e. repeating a close price from previous day, isn't optimal. There simply was no trade that day.

The same problem occurs if a date was supplied where the calendar expects a holiday. In some instances, such as when a bundle is reading data from multiple exchanges, that would be a possible scenario.

Having the entire ingest process fail on such things is probably not a desired behavior for many user.

I trust my data, and would simply like to add it as is. If a day has trades, that means this was not a holiday for this symbol. If there is no data supplied for a date, there simply was no trades done that day.

Can we optionally ignore the calendar check?

[wuup]

Same issue, any update on this?

[AndreasClenow]

@wuup No proper solution, but the workaround is to wrangle your data to fit the calendar.

Use something like this:

def parse_pricing_and_vol(data,
                          sessions,
                          symbol_map):
    for asset_id, symbol in iteritems(symbol_map):
        asset_data = data.xs(
            symbol,
            level=1
        ).reindex(
            sessions.tz_localize(None)
        ).fillna(0.0)
        yield asset_id, asset_data

[0xR]

I'm having the same issue, I'm trying to work with a custom bundle with ETFs from different exchanges. Apparently there is some missing data, probably due to no trades happening. Also the different exchanges have different calendars which can also cause missing data.

Now I have to try padding the data.