Alex, are you working on the iqfeed historical data calls?

[spickering-git]

@alexveden Are you currently working on the iqfeed historical data calls?

[alexveden]

I've created the prototype that can search tickers for a specific market and capable to fetch futures/options chains. The bad news that IQFeed returns only non-expired futures/options contracts so that prevents building asset index for expired contracts.

Because of this, I've faced two fundamental problems:

1. There is no way to extract the future/option expiration date from the IQFeed.
2. There is no way to extract ticker list to expired futures/options.

Problem 1 can be solved by manual definition of expiration rules for each product for the exchange in the algorithm, and then algo will calculate expiration date using these rules. This will require a lot of manual work and specifications reading + debugging of special cases like holidays.

Problem 2 can be solved by generating the list of futures and options. Because we can estimate the months, strikes and compose ticker name, then we can poll each ticker in IQFeed to retrieve history.

So, the final workflow could look like:

1. Define the exchange and product that have to be downloaded.
2. Read specifications for an expiration of futures + options, get understanding how to generate expiration (like 3rd Thursday of a month) + how to address the cases of holidays.
3. Run the special script / notebook which generates V1/V2 asset index (maps v2 ticker to IQFeed ticker name, set the expiration date)
4. Run a scheduled script that downloads/updates quotes for options and futures.

So, I need your help with solving the problem 1, I need the list of the products which you would like to download for EU markets and futures/options expiration rules for those products. And I propose to restructure this project as 2 parts: the first one will be the notebook part for asset index generation, and the second one will be IQFeed downloader script.

Unfortunately, I can't see the way of automatic downloading of the whole market, but this is doable on per product basis.

Also, these new circumstances would require more effort to finish this project. However, we can revise requirements of the project to meet old budget. @spickering-git, mentioned that he would like to replace span files with IQFeed. Steve, could you explain which functionality would you like to see? As for me, I see the biggest problem in ticker/asset index generation, the historical data fetching is not a problem at all.

[spickering-git]

@alexveden Task 1 is to collect data for instruments that are not in the CME span files. Task 2 : (LOW PRIORITY) replace data collection from CME span files is. We can loop back to this later.

So referring to task 1. I will consult with Nik and compile a list of instruments and their corresponding exchanges and include contract months with expiry rules. But we can make a switch with expiry style and pass the switch the parameters for the rule.

[alexveden]

But we can make a switch with expiry style and pass the switch the parameters for the rule.

for sure

[spickering-git]

@alexveden This is just a start on the instruments we want. symbols.xlsx Nik wanted me to get at least something up to you. First we are interested in getting cocoa and sugar from The ICE and the ETF https://www.nyse.com/quote/ARCX:OIL.

All ETF option expirations are as follows Monthly options expire on the third Friday of the expiration month. Many products also list weekly options that expire on Fridays. If an exchange holiday occurs on that Friday, weekly options will expire on the preceding Thursday. https://www.theocc.com/clearing/clearing-services/specifications-etf-options.jsp

[alexveden]

@spickering-git Hi Steve, could you also add information about the strike step and exprirations months (all available) for futures and options. Weeklys information also would be useful. Thx.