notebook for FOP option chain

ilan-pinto commented 3 months ago

Building an option chain for futures/indexes seems to be slightly different then for stocks it would be very helpful if we could have an example

mattsta commented 3 months ago

Is there a difference?

lastTradeDateOrContractMonth for futures must be the futures expiration month (currently 202409) which tracks with regular equity options also being able to be fetched with their expiration dates as month only 202407 (fetches all expirations for the month which will cause data pacing limits) or 20240719 (just fetch single expiration date) etc.

ilan-pinto commented 3 months ago

of course. building an option chain for index (ES,MES) futures requires first to qualify an Index and then request ticker for FuturesOption

On Mon, Jul 1, 2024 at 6:13 PM Matt Stancliff @.***> wrote:

Is there a difference?

lastTradeDateOrContractMonth for futures must be the futures expiration month (currently 202409) which tracks with regular equity options also being able to be fetched as 202407 or 20240719 etc.

— Reply to this email directly, view it on GitHub https://github.com/ib-api-reloaded/ib_async/issues/38#issuecomment-2200428010, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANWIKNKVEA4QEO3LL2OVKK3ZKFWZJAVCNFSM6AAAAABKFAPBYCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMBQGQZDQMBRGA . You are receiving this because you authored the thread.Message ID: @.***>

--

Ilan Pinto

Manager, Ecosystem Application Engineering

Red Hat https://www.redhat.com/ https://www.redhat.com/

mattsta commented 3 months ago

Not sure what the end result is then? You appear to suddenly have the information you asked for already but didn't mention originally.

ilan-pinto commented 3 months ago

the end result is always the same :) you should have an option chain. The way to get there is different; therefore, it's worth having a notebook. i can try sharing one

maqifrnswa commented 3 weeks ago

of course. building an option chain for index (ES,MES) futures requires first to qualify an Index and then request ticker for FuturesOption

Wow, thank you - this fixed a nearly impossible-to-debug bug I've been seeing. IBKR's TWS documentation says to qualify the Future, which is what I've been doing. But it really should be the Index. This is true for /ES, /CL, any other commodities futures. The option's underlying is the future, the future's underlying is the index, yet the options are children of the index, not the future...

I don't know if another notebook is needed for FOPs. https://github.com/ib-api-reloaded/ib_async/blob/main/notebooks/option_chain.ipynb pretty much covers it; it even qualifies the Index the requests the chain. Maybe just adding text to clarify that commodities options chains are found by the commodities index, not by the futures contract that is underlying the option chain. Their symbols are "CL IND" and "ES IND".

gnzsnz commented 3 weeks ago

not sure i follow, the example on the notebook is for index options

@ilan-pinto could you post an example?

mattsta commented 3 weeks ago

it even qualifies the Index the requests the chain.

Well, it does, but also it doesn't.

The notebook is looking up SPX options, but SPX options SPXW options and /ES and /MES options are different instruments.

Looking up index expirations works fine just by combining a symbol name with a correct lastTradeDateOrContractMonth value using regular details.

I couldn't get any lookup to work with Index contracts themselves, but regular OPT options work with just Option(symbol, "SMART") lookups (FOP options lookups require the exchange to be specified).

The reqSecDefOptParams() works different from the regular reqContractDetails() though.

reqSecDefOptParams() has benefits and drawbacks:

benefit: it's faster, because reqSecDefOptParams() is a direct lookup of a single instrument using (name, exchange, type, contract id) versus the regular reqContractDetails() is a generic lookup service which can search multiple matches if your request isn't specific enough.
benefit: you don't need to populate the exchange (can just use "") if the instrument is an index. You still need the symbol if looking up a Future() though.
downside: the returned expiration and strike combinations aren't detailed enough to use for generating quotes or orders.
downside: the "use the index" trick doesn't work everywhere. It appears to mostly work for indexes which are actual indexes with prices instead of IBKR simulated indexes without prices (you cant get a quote for Index("CL") for example because the trading is on Future("CL")).
downside: you need the actual contract id also in the parameters

I guess the short version is:

reqSecDefOptParams() takes a complete underlying contract using symbol/exchange/id and returns all options for it (even if many combinations of expiration/strike aren't valid)
reqContractDetails() takes an incomplete contract (in this case, OPT or FOP) then returns everything "to make it match" in more complete detail (like, if the input request only populates expiration as YYYYMM, the lookup returns more detailed YYYYMMDD results, etc).

The safest way is still using full contracts for lookups like:

FuturesOption(symbol='CL', lastTradeDateOrContractMonth='202409', exchange='NYMEX', currency='USD')

The primary problem with Futures is you need to specify the exchange manually, so you need either a local lookup table mapping names to exchanges, or, sometimes, you can qualify the index as just a symbol to discover the exchange to then use for the futures symbol.

Index(symbol='CL')

to then read the returned expansion of

   ib_async.contract.Index(
        secType='IND',
        conId=17340715,
        symbol='CL',
        exchange='NYMEX',
        currency='USD',
        localSymbol='CL'
    )

Then you can do the normal:

chains = await asyncio.wait_for(
  ib.reqContractDetailsAsync(
    FuturesOption(symbol='CL', lastTradeDateOrContractMonth='202409', exchange='NYMEX', currency='USD')),
  timeout=10)

I haven't found a generalized solution where, just given an instrument type and symbol name and expiration date, full options can be fetched reliably every time. It's a mix of needing manual exchange lookups for Futures or needing exact contract IDs, or filtering the bad date/expiration combinations out of reqSecDefOptParams() when it does work.

maqifrnswa commented 2 weeks ago

@mattsta , I think you might be experiencing TWS undocumented handling of futures options (both the futures and futures options are the children of the index not of the underlying contract). I had lots of trouble with futures option until I stumbled upon that realization - now it is 100% reliable, grabbing futures options contracts exactly as you described: I just give it symbol name and exchange to get the index's option chain, I qualify the futures contract based on the delivery date to get the conId, then filter the index's option chain by the future's conId and options expirations I want.

I adapted options_chain.ipynb for futures options: https://gist.github.com/maqifrnswa/9dd13abcb8166c4ec4e26c5d0647e60f

That one will always grab the earliest delivery futures contract and the chain corresponding to a delivery on the same date. You can make it shorter/easier by skipping the steps where you get all the upcoming futures to find the earliest delivery date if you already know the date and expiries you want. You can add multiple delivery dates and option expirations (that's what I do in my real-life code, I just have a list or dictionary of delivery date keys and a corresponding list of options expirations I want to grab for those keys)

Feel free to adapt this and add it here if it's helpful. I am just keeping it "secret" until it's tweaked to be what's most helpful.

mattsta commented 2 weeks ago

Oh, I see what you're saying now.

It is a quirk of the futures options where this works better than with equity options. Futures options have unique tradingClass names for almost every relevant week/day combination, so it does break out the data better instead of combining all dates and all strikes into one result.

Good point on the Index note too. Another weird un/under-documented part of the API. I hadn't realized there was a tree in the background attaching all of the different parts together so it works with Index(thing) -> Future(...) -> FuturesOption(...).

For my usage I had just settled on a pattern of:

fetch all chains for specific months (just request contract details but truncate the date as 202409 so it returns {date -> strikes} results for every expiration in a month)
cache the symbol/date/strikes results locally
generate more detailed requests to qualify full symbol/expiration/right/strike/exchange contracts as needed (then cache those full contract lookup results too).

westonplatter commented 2 weeks ago

For my usage I had just settled on a pattern of:

fetch all chains for specific months (just request contract details but truncate the date as 202409 so it returns {date -> strikes} results for every expiration in a month)

cache the symbol/date/strikes results locally

generate more detailed requests to qualify full symbol/expiration/right/strike/exchange contracts as needed (then cache those full contract lookup results too).

This is similar to workflow I've developed for future options.

mattsta commented 1 week ago

This week is a good example of the different approaches and problems too.

Also I figured out the "can't qualify 0DTE futures options" problem going into this again.

There are CME "Event Contracts" which are weird things qualifying as futures options, but are binary payout events. These are daily contracts and they pollute a same-day lookup with duplicates because they only appear daily. Plus, this week being quarterly opex, we have up to 3 contracts for the same symbol/expiration combination on Friday if we don't specify the trading class.

So, if you try to qualify a 0DTE futures option, you get "well, do you want the actual futures option or an event contract on the futures option or the quarterly expiration futures option?" error message (and I was sending all the noisy status messages to a log I rarely look at). To disambiguate the lookup, you must also provide tradingClass= in the contract if there are multiple conflicting matches.

To improve user experience around duplicate matches, I am now allowing the duplicates to be returned directly from a lookup so they can be interacted with at a program level instead of only in log messages.

Now, if you have duplicates, in place of the single contract return value in the qualifyContractsAsync() list of matched contracts, you can ask for await qualifyContractsAsync(contractA, contractB, ..., returnAll=True) and if there are duplicates, you will get a list of duplicate matches in place of the return value for a single contract lookup (e.g. return type is now list[Contract | list[Contract] | None]).

Exposing the duplicate contracts directly in the code helps because then we can manipulate the duplicates in code directly to show users how to fix the duplicate requests with more information like:

2024-09-20 08:03:49.854 | ERROR    | icli.cli:qualify:939 - [FuturesOption(symbol='ES', lastTradeDateOrContractMonth='20240920', strike=5725.0, right='P', exchange='CME', currency='USD')] contract request returned multiple matches! Can't determine which one of 3 to cache: [
    ib_async.contract.Contract(
        secType='FOP',
        conId=710137977,
        symbol='ES',
        lastTradeDateOrContractMonth='20240920',
        strike=5725.0,
        right='P',
        multiplier='50',
        exchange='CME',
        currency='USD',
        localSymbol='ESU4 P5725',
        tradingClass='ES'
    ),
    ib_async.contract.Contract(
        secType='FOP',
        conId=710138340,
        symbol='ES',
        lastTradeDateOrContractMonth='20240920',
        strike=5725.0,
        right='P',
        multiplier='50',
        exchange='CME',
        currency='USD',
        localSymbol='EW3U4 P5725',
        tradingClass='EW3'
    ),
    ib_async.contract.Contract(
        secType='EC',
        conId=730265747,
        symbol='ES',
        lastTradeDateOrContractMonth='20240920',
        strike=5725.0,
        right='P',
        multiplier='1',
        exchange='CME',
        currency='USD',
        localSymbol='ECESU420 P5725',
        tradingClass='ECES'
    )
]

These changes are in the next branch here (docs are basically just the commit messages for now), so if it's useful, give it a try and see if it helps.

ib-api-reloaded / ib_async

notebook for FOP option chain #38