Please take a look at SB (sugar)

[spickering-git]

Not sure what is missing. Loaded ICE data for SB (sugar). Not sure what is failing. The futures and options data is there.

[spickering-git]

Sugar has future months

but there are also only options in those same months

3, 5, 7, 10

How do we set up SB to only use those options?

[spickering-git]

Hold off on this for now. It looks like the SB ICE data didn't load correctly. There are options on serial months.

[spickering-git]

@alexveden There is a small bug in the span data load from ICE. There was an issue a while back where the id of the underlying future in the ICE data file doesn't map to the actual future. So you wrote a fix to search for the underlying future. Well for SB, it is not finding the march futures contract for the Nov and Dec options of the previous year.

For example Nov and Dec options in 2016 should be on March 2018 options. The program right now is not able to figure that out.

the sugar data I was using is here: https://app.box.com/s/uehgil2rsz124u48lh969nxd51vojwkc

[spickering-git]

wrote a fix in the ice_span.py file to deal with the unusual roll schedule of sugar.

[spickering-git]

@alexveden I reloaded SB looks like there is data in SB futures https://10.0.1.2:8888/notebooks/tools/test_SB_contracts.ipynb

Ran exo_batch_update.py Having trouble finding problem with SB exos etc. They just keep coming up blank. https://10.0.1.2:8888/notebooks/tools/EXO%20viewer.ipynb

Please have a look to get us on the right track here. @nikolas-joyce is aware of this request

SB rolling march, may, july, oct for futures and there are options for every month

not sure if it is failing due to unusual rolling

[nikolas-joyce]

Alex could you have a quick look at this we seem to be stuck. Thx

[alexveden]

It seems that is rollover helper problem. The problem in difference of expiration date month and contract expiration month: 2018-02-28 F.US.SBEH18 14.72 As we can see the contract expired in Feb 28, but the contract suffix is H18 - March! This is the reason of this issue.

Here is a simple data diagnostic notebook: https://10.0.1.2:8888/notebooks/tools/Data%20Diagnostics.ipynb It will help you to understand the rollover regime for given instrument.

I know how to fix that, but I need your help with that critical section of the code. Basically we have 2 options how to fix this issue:

1. Replace the shifted SB months to rollover helper (i.e. replace 3rd month of March by 2 Feb) This is a workaround, but probably it could mess things up in the future. Because anyone who sees SBEH18 think of it as of March contracts.

2. Fix the algorithm, to use month_int code of the future. This is where I need your help, I need you to check the instruments in the rollover helper regime settings to find the behavior like as SB. I have checked all, but we need to triple check that. You can use notebook above to do these checks. This is critical, because with different rollover rules we will get messed EXOs and other stuff.

This issue will take about 10 minutes to fix, but I need to be sure that it will not ruin the data. I will wait your mark.

[spickering-git]

I don't think we should use option 1

I ran the sheet on all of the instruments. SB is the only one that is returning NONE. https://10.0.1.2:8888/notebooks/tools/Data%20Diagnostics.ipynb

We use the last trade date for the ICE products https://www.theice.com/products/23/Sugar-No-11-Futures/expiry

The energy futures contracts all have expirations prior to the month they are listed in. 2017-07-20 F.CLEQ1 2017-08-22 F.CLEU17

2017-08-29 F.NGEU17

2017-08-31 F.RBEU17

2017-08-31 F.HOEU17

[spickering-git]

@alexveden, There was a problem in the ICE data load where the idoption or idcontract were interpreted as floats and on the upsert could result in a new row on a reload of the data. SB has been cleared and reloaded.

I will be doing this for CC also.

RE:

I can fix it, but I'm afraid to blow production up

f.expiration.month in self.rollover_months: replace to f.month_int in self.rollover_months:

since the energy contracts are in every month both if statements will always be true. I believe at this moment they are the only other contracts that have month_int != expiration.month

So I think it is ok to go ahead and change the code.

https://10.0.1.2:8888/notebooks/tools/Data%20Diagnostics.ipynb has a test with all of the current contracts we are monitoring

[alexveden]

Fixed