Nested enumerator

[lolabeis]

In certain cases, enumerated values are not flat and need to be nested within some hierarchy. Currently, a complex type is required to handle that nesting and hierarchy.

For example, AssetType is defined in the CDM as a complex type as follows:

type AssetType:
    assetType AssetTypeEnum (1..1)
    securityType SecurityTypeEnum (0..1)
    debtType DebtType (0..1)
    equityType EquityTypeEnum (0..1)
    fundType FundProductTypeEnum (0..1)

Where AssetTypeEnum is:

enum AssetTypeEnum:
    Security
    Cash
    Commodity
    Other

When the assetType value is Security, the securityType attribute allows to specify a security sub-type as either:

enum SecurityTypeEnum:
    Debt
    Equity
    Fund
    Warrant
    Certificate
    LetterOfCredit

And if the securityType value is Debt, the debtType attribute allows to specify a further debt sub-type as:

type DebtType:
    debtClass DebtClassEnum (0..1)
    debtEconomics DebtEconomics (0..*)

And so on... Eventually, all the attributes and, recursively, sub-attributes of AssetType are enumerated values. However these enumerated values have an intrinsic hierarchy, which is why AssetType is not defined simply as a "flat" enumeration of those values.

The integrity of the top-level AssetType object is enforced via a set of validation rules that verify the internal consistency of the various enumeration attributes. E.g.:

condition SecuritySubType:
    if assetType <> AssetTypeEnum -> Security
    then securityType is absent and debtType is absent and equityType is absent and fundType is absent

condition BondSubType:
    if securityType <> SecurityTypeEnum -> Debt
    then debtType is absent

... etc

As an example, a convertible bond would be defined as follows (in JSON):

{
  "assetType": SECURITY,
  "securityType": DEBT
  "debtType":
  {
    "debtClass": CONVERTIBLE
  }
}

Instead, if enumerated values could be nested as follows:

enum AssetTypeEnum:
    Security:
        Debt:
            AssetBacked
            Convertible
            RegCap
            ...etc
        Equity
        Fund
        Warrant
        Certificate
        LetterOfCredit
    Cash
    Commodity
    Other

With that design, the internal-consistency validation rules, which could be error-prone, would not be required because the consistency is enforced by design in the enumeration hierarchy.

The same convertible bond example could be defined more succinctly as:

SECURITY.DEBT.CONVERTIBLE

[lolabeis]

@SimonCockx Keen to hear if you have any good ideas about the above!

[SimonCockx]

Always happy to think up solutions for a well-described problem!

First I would like to see whether there are good alternative ways of modelling this with the current DSL capabilities. I think one way to simplify the conditional logic and to make it less error-prone, is to model it as follows:

type AssetType: <"Represents a class to allow specification of the asset product type.">
    securityType SecurityType (0..1)
    otherAssetType string (0..*) <"Specifies the eligible asset type when not enumerated.">

    condition SubType:
        one-of

type SecurityType:
    debtType DebtType (0..1) <"Represents a filter based on the type of bond.">
    equityType EquityTypeEnum (0..1) <"Represents a filter based on the type of equity.">
    fundType FundProductTypeEnum (0..1) <"Represents a filter based on the type of fund.">

    condition SubType:
        one-of

// same for DebtType etc.

Or more in general: for every non-flat enumeration, create a type with the following:

• each enumerated value is represented as an optional property, where the type is either a flat enum, or again a type representing a non-flat enum.
• consistency is guaranteed by adding the condition one-of.

In this case the conditional logic is robust and doesn't require maintenance when new properties are added. In order to check which "type" of asset you're working with, you can replace checks such as assetType -> assetType = AssetTypeEnum -> Security with assetType -> securityType exists, for example. Would this be an acceptable alternative?

One other point about AssetType. It seems to me that some cardinality constraints require it to be a complex type, i.e.,

type AssetType:
    assetType AssetTypeEnum (1..1)
    securityType SecurityTypeEnum (0..1)
    debtType DebtType (0..1)
    equityType EquityTypeEnum (0..1)
    fundType FundProductTypeEnum (0..1)
    otherAssetType string (0..*) // THIS PROPERTY HERE...

type DebtType:
    debtClass DebtClassEnum (0..1)
    debtEconomics DebtEconomics (0..*) // ... AND THIS PROPERTY HERE

so these types are actually not just simple enumerations. But for the sake of the issue, let's say that that wasn't the case. Then just to make sure that I completely understand the problem, what would be the implications if we were to model this as a flat enum? Would that be problematic for the way it is being used right now? E.g.,

enum AssetTypeEnum:
  Security__Debt__AssetBacked
  Security__Debt__Convertible
  Security__Debt__RegCap
  ...etc
  Security__Equity
  Security__Fund
  Security__Warrant
  Security__Certificate
  Security__LetterOfCredit
  Cash
  Commodity
  Other

[SimonCockx]

This issue is superseded by https://github.com/finos/rune-dsl/issues/818.

[lolabeis]

@SimonCockx Can I ask some clarification on why you think this issue is superseded by having proper support for enum extension and can be closed?

Is it because, using my example above, you propose to implement as follows:

enum SecurityTypenum extends DebtClassEnum, EquityTypeEnum, FundTypeEnum
  Warrant
  Certificate
  LetterOfCredit

enum AssetTypeEnum extends SecurityTypeEnum
  Cash
  Commodity
  Other

?

I agree that this addresses some of what this issue raises, but not all. AssetTypeEnum would ultimately look like a long, flat list of all possible values with no hierarchy. And we would need to make sure there is no name clash between values coming from different enums.

Also how do you propose to write the logic to assert whether assetTypeEnum is Security, Debt etc?

Thanks!

[SimonCockx]

@lolabeis I was cleaning up issues related to enums, and perhaps I was too hasty on this one.

With the improvement for enum extensions in the linked issue we could indeed model AssetTypeEnum in the way you have written. To assert a value is of a certain type, one could write, e.g.,

// check whether assetType is a security
assetType to-enum SecurityTypeEnum exists

// check whether assetType is a debt:
assetType to-enum DebtClassEnum exists

// etc

Since this does not address all of the requests in this issue, I'll reopen it. Just to list out what is missing (please complement this list if anything is missing):

1. Allow values of the same name on different levels of the hierarchy, e.g.,

   enum FooEnum:
   BarEnum:
       MY_VALUE
   QuxEnum:
       MY_VALUE
   MY_VALUE

2. Serialised enum values retain information about their position in the hierarchy, e.g.,

   BarEnum.MY_VALUE

3. Syntax for accessing nested enum values, e.g.,

   FooEnum -> BarEnum -> MY_VALUE

4. (not sure) Using nested enums as a standalone type. For the example of AssetTypeEnum:

   func ProcessSecurity:
   inputs:
       input AssetTypeEnum -> Security (1..1)
   output:
   ...

[SimonCockx]

Could this be modelled using choice types instead?

choice AssetType:
    SecurityType
    OtherAssetTypeEnum

enum OtherAssetTypeEnum:
    Cash
    Commodity
    Other

choice SecurityType:
    DebtClassEnum
    EquityTypeEnum
    FundTypeEnum
    OtherSecurityTypeEnum

enum OtherSecurityTypeEnum:
    Warrant
    Certificate
    LetterOfCredit

enum DebtClassEnum:
    AssetBacked
    Convertible
    RegCap
    ...

// Alternative for SecurityType assuming all options in SecurityType are enums, and their values don't have overlapping names:
enum SecurityTypeEnum extends DebtClassEnum, EquityTypeEnum, FundTypeEnum:
    Warrant
    Certificate
    LetterOfCredit

Going through the four points I described above:

1. Values of the same name on different levels of the hierarchy are allowed.
2. With the serialisation proposal in https://github.com/finos/rune-dsl/issues/797, the serialised form would retain information about its position in the hierarchy by including an @type property:

   {
   "@type": "DebtClassEnum",
   "value": "Convertible"
   }

3. Accessing "nested" enum values can be done directly, e.g.,

   DebtClassEnum -> Convertible

4. "Nested" enums can be used as a standalone type directly, e.g.,

   func ProcessSecurity:
   inputs:
       input SecurityType (1..1)
   output:
   ...

[lolabeis]

Thanks for re-opening.

Indeed, mixing choice and enum may be the way to go. Can we already use an enum today inside a choice?

[SimonCockx]

Can we already use an enum today inside a choice?

Definitely!