Question: Leveraging ORE-SWIG Python Bindings for SACCR Implementation

[vannarho-fas]

Hello Community,

Actually, I cannot find a community for ORE so I am posting here. I tried to contact the forum on the ore webpage but there was no response. 😃

Please redirect me as needed. I am part of the Quantlib user group but don't see any threads there re ORE. Is there a group that I am not aware of?

I have a SACCR calculator using Python but it does not currently cover all instruments in ORE / QL so I'm looking into the option of refactoring the code to leverage ORE and its Python bindings via SWIG. An alternative is porting the code to C++ / ORE, but as I judge this to be more difficult (for me at least, though has a lot of other advantages), I want to explore the SWIG route first.

As a first step in this investigation, I want to populate a dictionary, such as the one below, using details from various trade objects in the portfolio.

Here's an example dictionary structure I'm looking to fill:

trade_dict = {
'TradeType': '', 'SubClass': '', 'Notional': '', 'Currency': '', 'Si': '', 'Ei': '',  'BuySell': '', 'Id': '', 'Counterparty': '', 'OptionType': '', 'UnderlyingPrice': '',  'StrikePrice': '', 'commodity_type': '', 'RefEntity': '', 'CcyPair': '', 'PayLegType': '', 'PayLegRef': '', 'PayLegTenor': '', 'RecLegType': '', 'RecLegRef': '', 'RecLegTenor': '',  'Vol_Strike': '', 'Traded_Price': '', 'Delivery_Type': '', 'Underlying_Instrument': '', 'Exotic_Type': '',  'ISIN': '', 'CouponType': '', 'Issuer': '', 'Maturity': '', 'PaymentFrequency': '',  'CreditRiskWeight': '', 'CdoAttachPoint': '', 'CdoDetachPoint': '', 'FxNearLegFields': '', 'CcyPaying': '', 'AmountPaying': '', 'CcyReceiving': '', 'AmountReceiving': ''
}

The concept was to iterate over the portfolio's trade IDs, extracting the relevant information from each trade:

for id in portfolio.ids():
    trade = portfolio.get(id)
    if trade.tradeType() == 'Swap':
        trade_dict['TradeType'] = trade.tradeType()
        trade_dict['Notional'] = trade.notional()
        trade_dict['Counterparty'] = trade.envelope().counterparty()
        # and so on...
        for leg in trade.legs() # this currently returns a cashflow as far as remember, not leg data
              if leg.legType = "Fixed". # made this up
                     # and so on...
    elif trade_type == 'Swaption':
       # handle swaption trades...

Using the "Input" files from Example_1 (/oreswig/OREAnalytics-SWIG/Python/Examples/Notebooks/Example_1), trade.legs() seems to be returning an empty tuple, even though the trade has fixed and floating legs. print(portfolioXML) shows that the data is fully populated...how to get access to it?

from ORE import *
import sys, time, math
sys.path.append('..')
import utilities
params = Parameters()
params.fromFile("Input/ore.xml")
ore = OREApp(params)
portfolio = ore.getInputs().portfolio()
print("asof date:",ore.getInputs().asof())
print("Number of trades:", portfolio.size())

for id in portfolio.ids():
    trade = portfolio.get(id)
    print("Trade class (trade variable): " , trade)
    print("Trade:  id=%s type=%s" % (id, trade.tradeType()))
    print("Counterparty:" , trade.envelope().counterparty())
    print("NettingSetId:" , trade.envelope().nettingSetId())
    print("additionalFields:" , trade.envelope().additionalFields())
    print("instrument:" , trade.instrument())
    print("notional:" , trade.notional())
    print("maturity:" , trade.maturity())
    print("legs:" , trade.legs())

# no binding available for the below
#     print("portfolioIds:" , trade.portfolioIds() 
#     print("tradeActions:" , trade.tradeActions())
#     print("legPayers:" , trade.legPayers())
#     print("npvCurrency:" , trade.npvCurrency()) 
#     print("notionalCurrency:" , trade.notionalCurrency()) 
#     print("issuer:" , trade.issuer() 

# throws the error "'NoneType' object has no attribute 'qlInstrument'"
#     print("qlInstrument:" , trade.instrument().qlInstrument()) 

portfolioXML = portfolio.toXMLString()
print()
print(portfolioXML)

Here is the console output.

asof date: February 5th, 2016
Number of trades: 1
Trade class (trade variable):  <ORE.ORE.Trade; proxy of <Swig Object of type 'std::vector< ext::shared_ptr< Trade > >::value_type *' at 0x10ee18120> >
Trade:  id=Swap type=Swap
Counterparty: CPTY_A
NettingSetId: CPTY_A
additionalFields: <ORE.ORE.StringStringMap; proxy of <Swig Object of type 'std::map< std::string,std::string,std::less< std::string >,std::allocator< std::pair< std::string const,std::string > > > *' at 0x10ee18ae0> >
instrument: None
notional: 3.4028234663852886e+38
maturity: null date
legs: ()

<Portfolio>
    <Trade id="Swap">
        <TradeType>Swap</TradeType>
        <Envelope>
            <CounterParty>CPTY_A</CounterParty>
            <NettingSetId>CPTY_A</NettingSetId>
            <PortfolioIds/>
            <AdditionalFields/>
        </Envelope>
        <SwapData>
            <LegData>
                <LegType>Fixed</LegType>
                <Payer>false</Payer>
                <Currency>EUR</Currency>
                <PaymentConvention>MF</PaymentConvention>
                <DayCounter>A360</DayCounter>
                <Notionals>
                    <Notional>10000000.000000</Notional>
                    <Exchanges>
                        <NotionalInitialExchange>false</NotionalInitialExchange>
                        <NotionalFinalExchange>false</NotionalFinalExchange>
                        <NotionalAmortizingExchange>false</NotionalAmortizingExchange>
                    </Exchanges>
                </Notionals>
                <ScheduleData>
                    <Rules>
                        <StartDate>20160209</StartDate>
                        <EndDate>20360209</EndDate>
                        <Tenor>1Y</Tenor>
                        <Calendar>TARGET</Calendar>
                        <Convention>MF</Convention>
                        <TermConvention>MF</TermConvention>
                        <Rule>Forward</Rule>
                        <EndOfMonth/>
                        <FirstDate/>
                        <LastDate/>
                    </Rules>
                </ScheduleData>
                <FixedLegData>
                    <Rates>
                        <Rate>0.021000</Rate>
                    </Rates>
                </FixedLegData>
            </LegData>
            <LegData>
                <LegType>Floating</LegType>
                <Payer>true</Payer>
                <Currency>EUR</Currency>
                <PaymentConvention>MF</PaymentConvention>
                <DayCounter>A360</DayCounter>
                <Notionals>
                    <Notional>10000000.000000</Notional>
                    <Exchanges>
                        <NotionalInitialExchange>false</NotionalInitialExchange>
                        <NotionalFinalExchange>false</NotionalFinalExchange>
                        <NotionalAmortizingExchange>false</NotionalAmortizingExchange>
                    </Exchanges>
                </Notionals>
                <ScheduleData>
                    <Rules>
                        <StartDate>20160209</StartDate>
                        <EndDate>20360209</EndDate>
                        <Tenor>6M</Tenor>
                        <Calendar>TARGET</Calendar>
                        <Convention>MF</Convention>
                        <TermConvention>MF</TermConvention>
                        <Rule>Forward</Rule>
                        <EndOfMonth/>
                        <FirstDate/>
                        <LastDate/>
                    </Rules>
                </ScheduleData>
                <FloatingLegData>
                    <Index>EUR-EURIBOR-6M</Index>
                    <IsInArrears>false</IsInArrears>
                    <IsAveraged>false</IsAveraged>
                    <HasSubPeriods>false</HasSubPeriods>
                    <IncludeSpread>false</IncludeSpread>
                    <FixingDays>2</FixingDays>
                    <Caps/>
                    <Floors/>
                    <Gearings/>
                    <Spreads>
                        <Spread>0.000000</Spread>
                    </Spreads>
                    <NakedOption>false</NakedOption>
                </FloatingLegData>
            </LegData>
        </SwapData>
    </Trade>
</Portfolio>

I also tried to load a new Portfolio directly with new data in case it had been processed (e.g. In a subsequent step, initiated by a call to the build member function, if the “raw” trade data is then translated into QuantLib/QuantExt objects up to the QuantLib/QuantExt instrument, linked to a QuantLib/QuantExt pricing engine, in turn linked to the relevant term structures provided by the Market class. e.g.

port = Portfolio()
port.fromFile("/home/vr/oreswig/OREAnalytics-SWIG/Python/Examples/Notebooks/Example_2/Input/portfolio.xml")
print("Number of trades:", port.size())
for id in port.ids():
    trade = port.get(id)
    print("id:",id)
    print("Counterparty:" , trade.envelope().counterparty())
    print("legs:" , trade.legs())

output:

Number of trades: 3
id: Swap_1
Counterparty: CPTY_A
legs: ()
id: Swap_2
Counterparty: CPTY_A
legs: ()
id: Swap_3
Counterparty: CPTY_A
legs: ()

In the swig interface file, the trade.legs() method seems to return a two-dimensional vector of shared_ptrs to CashFlow objects.

%shared_ptr(Trade)
class Trade {
  private:
    Trade();
  public:
    const std::string& id();
    const std::string& tradeType();
    const ext::shared_ptr<InstrumentWrapper>& instrument();
    std::vector<std::vector<ext::shared_ptr<QuantLib::CashFlow>>> legs();
    const Envelope& envelope() const;
    const QuantLib::Date& maturity();
    Real notional();
};

If I want to access the underlying leg and schedule data, I had two ideas, but I can't see the bindings for these / understand how to do this yet:

1. Use the instrument (?does the Portfolio need to be loaded afresh and not built to have access to the raw XML inputs?)
   • Access the QuantLib::Instrument object from the InstrumentWrapper using the qlInstrument() method.
   • Cast the QuantLib::Instrument object to the appropriate derived class (QuantLib::Swap, QuantLib::Bond, etc.).
   • Call the appropriate methods of the derived class to access the leg and schedule data.

However in the above example trade.instrument().qlInstrument() = None and trade.instrument().qlInstrument() throws the error "'NoneType' object has no attribute 'qlInstrument'"

2. Pass the trade to XML and then use utilities (e.g. using c++)

   
   // Assuming 'trade' is your Trade object and 'doc' is your XMLDocument object
   XMLNode* node = trade.toXML(doc);

// use the xml utility functions to extract the data string id = XMLUtils::getChildValue(node, "id"); Real notional = XMLUtils::getChildValueAsDouble(node, "notional"); bool payer = XMLUtils::getChildValueAsBool(node, "payer"); Period tenor = XMLUtils::getChildValueAsPeriod(node, "tenor");

trade.toXML(doc) and the xmlutils would need to be exposed via swig. 

So, I am a bit stuck. 

So, my question for the community is: Can I directly use the ORE-SWIG Python bindings "as is" to accomplish this task (e.g. how to access fixed / floating legdata)? Would I need to extend these SWIG bindings to access the necessary functions or objects? Or are there other better ways to do this? 

I appreciate any insights or guidance you can provide.

Thanks in advance!

[vannarho-fas]

Hi,

Am I right in thinking that ORE-SWIG is an open source repo without a community?

In case anyone reads this....

From further investigation, the answer to "Can I directly use the ORE-SWIG Python bindings "as is" to accomplish this task" is "No". :-(

However, it is possible to get access to portfolio data. It is held in nested classes under Portfolio. I gained access to the legData for Example_1 by casting the trade->instrument()->qlInstrument() to a swap and then using the LegData / FixedLegData / FloatingLegData class methods.

From this investigation, my updated hypothesis is that given the depth of the ORE codebase, it may actually be more beneficial to port the SACCR code to ORE.

Here's some console output to illustrate:

1) Run ORE to produce NPV cube and exposures Loading inputs OK Requested analytics CASHFLOW,NPV,SACCR Pricing: Build Market OK Pricing: Build Portfolio OK Pricing: Cashflow Report OK Pricing: NPV Report OK Pricing: Additional Results OK Pricing: Market Calibration OK Pricing: Curves Report OK SACCR: analytics started SACCR: test printSaccrvTrades SACCRV: Portfolio hasNettingSetDetails >> 0 SACCR: Id >> Swap_20y SACCR: TradeType >> Swap SACCR: Envelope - Counterparty >> CPTY_A SACCR: Envelope - NettingSetId >> CPTY_A SACCR: Envelope - PortfolioIds >> SACCR: Envelope - AdditionalFields >> SACCR: Envelope - isEmpty >> false SACCR: Envelope - hasNettingSetDetails >> true SACCR: PortfolioIds >> SACCR: TradeActions >> None SACCR: Print the legData by casting the trade->instrument()->qlInstrument() to a swap and then printing the LegData/ FixedLegData methods The instrument is a swap Leg 0 is a fixed leg: Rates: 0.02 isPayer: 0 currency: EUR schedule: name: ScheduleData hasDerived: 0 dates: rules: startDate: 20160301 endDate: 20360301 tenor: 1Y calendar: TARGET convention: F termConvention: F rule: Forward endOfMonth: firstDate: lastDate: derived: notionals: 1e+07 Leg 1 is a floating leg: Index: EUR-EURIBOR-6M Spreads: 0 isPayer: 1 currency: EUR schedule: name: ScheduleData hasDerived: 0 dates: rules: startDate: 20160301 endDate: 20360301 tenor: 6M calendar: TARGET convention: MF termConvention: MF rule: Forward endOfMonth: firstDate: lastDate: derived: notionals: 1e+07 SACCR: Legs which are QL cashflows >> Date: March 1st, 2017 Amount: 200000 Date: March 1st, 2018 Amount: 200000 Date: March 1st, 2019 Amount: 200000 Date: March 2nd, 2020 Amount: 200556 Date: March 1st, 2021 Amount: 199444 Date: March 1st, 2022 Amount: 200000 Date: March 1st, 2023 Amount: 200000 Date: March 1st, 2024 Amount: 200000 Date: March 3rd, 2025 Amount: 201111 Date: March 2nd, 2026 Amount: 199444 Date: March 1st, 2027 Amount: 199444 Date: March 1st, 2028 Amount: 200000 Date: March 1st, 2029 Amount: 200000 Date: March 1st, 2030 Amount: 200000 Date: March 3rd, 2031 Amount: 201111 Date: March 1st, 2032 Amount: 198889 Date: March 1st, 2033 Amount: 200000 Date: March 1st, 2034 Amount: 200000 Date: March 1st, 2035 Amount: 200000 Date: March 3rd, 2036 Amount: 201111 Date: September 1st, 2016 Amount: 101886 Date: March 1st, 2017 Amount: 97860.8 Date: September 1st, 2017 Amount: 99490.8 Date: March 1st, 2018 Amount: 97951.1 Date: September 3rd, 2018 Amount: 101418 Date: March 1st, 2019 Amount: 97582.4 Date: September 2nd, 2019 Amount: 100867 Date: March 2nd, 2020 Amount: 99191.4 Date: September 1st, 2020 Amount: 99494.9 Date: March 1st, 2021 Amount: 98432.5 Date: September 1st, 2021 Amount: 100320 Date: March 1st, 2022 Amount: 98676.3 Date: September 1st, 2022 Amount: 100320 Date: March 1st, 2023 Amount: 98666.3 Date: September 1st, 2023 Amount: 100228 Date: March 1st, 2024 Amount: 99132.8 Date: September 2nd, 2024 Amount: 100775 Date: March 3rd, 2025 Amount: 99132.8 Date: September 1st, 2025 Amount: 99132.8 Date: March 2nd, 2026 Amount: 99143.3 Date: September 1st, 2026 Amount: 99772.1 Date: March 1st, 2027 Amount: 98676.3 Date: September 1st, 2027 Amount: 100320 Date: March 1st, 2028 Amount: 99213.5 Date: September 1st, 2028 Amount: 100226 Date: March 1st, 2029 Amount: 98584 Date: September 3rd, 2029 Amount: 101321 Date: March 1st, 2030 Amount: 97489.4 Date: September 2nd, 2030 Amount: 100774 Date: March 3rd, 2031 Amount: 99135.6 Date: September 1st, 2031 Amount: 99168.4 Date: March 1st, 2032 Amount: 99168.4 Date: September 1st, 2032 Amount: 100264 Date: March 1st, 2033 Amount: 98620.9 Date: September 1st, 2033 Amount: 100264 Date: March 1st, 2034 Amount: 98620.9 Date: September 1st, 2034 Amount: 100264 Date: March 1st, 2035 Amount: 98620.9 Date: September 3rd, 2035 Amount: 101359 Date: March 3rd, 2036 Amount: 99162.2 SACCR: LegCurrencies >> EUR EUR SACCR: LegPayers >> 0 1 SACCR: NPVCurrency >> EUR SACCR: Maturity >> March 3rd, 2036 SACCR: print supervisoryFactors Asset_Class SubClass Supervisory_factor Correlation Supervisory_option_volatility

Commodity Electricity 0.4000 0.40 1.50
Commodity Other 0.1800 0.40 0.70
CreditIndex IG 0.0038 0.80 0.80
CreditIndex SG 0.0106 0.80 0.80
CreditSingle A 0.0042 0.50 1.00
CreditSingle AA 0.0038 0.50 1.00
CreditSingle AAA 0.0038 0.50 1.00
CreditSingle B 0.0160 0.50 1.00
CreditSingle BB 0.0106 0.50 1.00
CreditSingle BBB 0.0054 0.50 1.00
CreditSingle CCC 0.0600 0.50 1.00
EQ 0.3200 0.50 1.20
EQ Index 0.2000 0.80 0.75
FX 0.0400 0.00 0.15
IRD 0.0050 0.00 0.50
OtherExposure 0.0800 0.00 1.50
SACCR: test complete run time: 0.389229 sec ORE done.

F

[rolandlichters]

Hi F,

apologies for the late reply- I think your last approach is reasonable, to extend core ORE to cover SA-CCR, with an additional analytic. Have you startend down this route? We have an internal SACCR implementation that covers a few products so far - Swaps, Swaptions, FX Forwards, FX Options, … We are about to extend that scope. And I expect to get approval for releasing that code in ORE with one of the next releases in 2024. Let us know whether you would like to contribute to that!

Best wishes, Roland

[vannarho-fas]

Thanks for the reply Roland.

I did start down this route initially for Swaps, but then I read you were going to release SA-CCR, so I realised my efforts would be wasted (so I ended up working outside of ORE using a FileIO pattern).

I have a few important questions for your roadmap - I appreciate there are necessarily caveats when it comes to release dates / roadmap items, but it would be very helpful to know:

Which release is targeted for SA-CCR? Do you have release dates (even notional) for SA-CVA and FRTB-SA? What is the broad design of your AAD implementation? Will it only be for scripted trade items?

I cannot commit to contributing right at the moment due to other pressures, but I would like to contribute at some point. Are there forums / design resources for contributors especially when it comes to major new features like SA-CCR?

Speak soon,

Forde

On Mon, 13 Nov 2023 at 22:36, Roland Lichters @.***> wrote:

Hi F,

apologies for the late reply- I think your last approach is reasonable, to extend core ORE to cover SA-CCR, with an additional analytic. Have you startend down this route? We have an internal SACCR implementation that covers a few products so far

• Swaps, Swaptions, FX Forwards, FX Options, … We are about to extend that scope. And I expect to get approval for releasing that code in ORE with one of the next releases in 2024. Let us know whether you would like to contribute to that!

Best wishes, Roland

— Reply to this email directly, view it on GitHub https://github.com/OpenSourceRisk/ORE-SWIG/issues/23#issuecomment-1807990550, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEAXL6ZCJQX2GWMWVSZSFR3YEIA4BAVCNFSM6AAAAAAZCHSQ26VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMBXHE4TANJVGA . You are receiving this because you authored the thread.Message ID: @.***>

[pcaspers]

Hi Forde, on this

What is the broad design of your AAD implementation? Will it only be for scripted trade items?

No, it will eventually apply to all trade types. The script framework provides the infrastructure to perform AAD, therefore classical trade types will have to be translated, but that will happen under the hood.

[vannarho-fas]

Roland, Peter,

I am still interested in answers to my questions. If you don't have time to reply, could you perhaps point me in the right direction in terms of people who can?

The questions again: Which release is targeted for SA-CCR? I have a partially completed build in ORE but will finish it if there is no commitment from you to release. Do you have designs, plans, release dates (even notional) for BA/SA-CVA and FRTB-SA? What is the broad design of your AAD implementation? Will it only be for scripted trade items?

Two further questions (one of which I have asked previously and restate):

1. you said "We have an internal SA-CCR implementation that covers a few products". From what I can tell, OREPLus includes SA-CCR, as there are code remnants referencing SA-CCR throughout ORE, like ORE has excluded the functionality. Is that the case?
2. What forums exist for developers specifically to avoid people writing duplicate functionality?

Regards,

Forde

On Tue, 14 Nov 2023 at 08:39, Forde Smith @.***> wrote:

Thanks for the reply Roland.

I did start down this route initially for Swaps, but then I read you were going to release SA-CCR, so I realised my efforts would be wasted (so I ended up working outside of ORE using a FileIO pattern).

I have a few important questions for your roadmap - I appreciate there are necessarily caveats when it comes to release dates / roadmap items, but it would be very helpful to know:

Which release is targeted for SA-CCR? Do you have release dates (even notional) for SA-CVA and FRTB-SA? What is the broad design of your AAD implementation? Will it only be for scripted trade items?

I cannot commit to contributing right at the moment due to other pressures, but I would like to contribute at some point. Are there forums / design resources for contributors especially when it comes to major new features like SA-CCR?

Speak soon,

Forde

On Mon, 13 Nov 2023 at 22:36, Roland Lichters @.***> wrote:

Hi F,

apologies for the late reply- I think your last approach is reasonable, to extend core ORE to cover SA-CCR, with an additional analytic. Have you startend down this route? We have an internal SACCR implementation that covers a few products so far - Swaps, Swaptions, FX Forwards, FX Options, … We are about to extend that scope. And I expect to get approval for releasing that code in ORE with one of the next releases in 2024. Let us know whether you would like to contribute to that!

Best wishes, Roland

— Reply to this email directly, view it on GitHub https://github.com/OpenSourceRisk/ORE-SWIG/issues/23#issuecomment-1807990550, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEAXL6ZCJQX2GWMWVSZSFR3YEIA4BAVCNFSM6AAAAAAZCHSQ26VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMBXHE4TANJVGA . You are receiving this because you authored the thread.Message ID: @.***>