Should a relationship definition be allowed to refer to properties of a target node type before the type of the node has been declared/resolved? If not then the Node clause should be documented as required in that case.

[philippemerle]

Should a relationship definition be allowed to refer to properties of a target node type before the type of the node has been declared/resolved?

No.

If not then the Node clause should be documented as required in that case

yes defining the node keyname then allow to refer to properties of this node type.

[tliron]

We have a similar issue whereby a requirement can refer to a capability by name, rather than by type. In that case, too, the capability is un-typed until the requirement is resolved, so we should not be able to refer to capability properties.

[pmjordan]

The mandatory use of node clause in this situation would/is a particular problem in my use case. This is becuase I want to define building blocks in a profile produced by one organisation and have those blocks formed onto a topology by users in a separate organisation. See https://github.com/pmjordan/tosca/blob/dev/profiles/org/tmforum/1.0/profile.yaml line 150.

[philippemerle]

Is https://github.com/pmjordan/tosca publicly available?

[pmjordan]

Is https://github.com/pmjordan/tosca publicly available?

Sorry my mistake, that is a private repo.

[pmjordan]

Here is an extract:

namespace: org.tmforum:1.0

description: Profile Type Definitions for ODA by Paul Jordan
#Note To get this to parse with puccini and 'Cloudnet TOSCA toolbox' comment out interfaces because they use a v2 syntax

capability_types:

  TMFAPI:
    derived_from: tosca.capabilities.Node
    description: a TMForum defined REST API. This is an abstract capability type
    properties:
      id:
        type: string
      name:
        type: string

  TMF620:
    derived_from: TMFAPI 
    description: This is an concrete capability type
    properties:
      id: TMF620
      name: Product Catalog Mgmt

  TMF622:
    derived_from: TMFAPI 
    properties:
      id: TMF622
      name: Product Ordering

relationship_types:
  oda_component_depends_on_TMF_API:
    derived_from: tosca.relationships.DependsOn
    description: Used to connect an ODA component which requires an API with an ODA component which has such an API
    valid_target_types: [ TMFAPI ] # i.e. This relationship is only to be used to connect to nodes which have a capability derived from TMFAPI
    attributes:
      api_provider_sap:
        description: The IP address of the server providing the API
        type: string

node_types:

  oda_component:
    derived_from: tosca.nodes.Root
    properties:
      component_class_ID:
        type: string
        description: >-
          ID of the component, e.g TMFC001.
          Values to be taken from IG1242 ODA Component Inventory
          The node type name must be the same value as component_class_ID
        required: True
      node_instance_name:
        type: string
        description: Must be set to the same value as the name of the node template as used in the service template
        required: true
      definition_file:
        type: string
        description: >-
          A full path name to an ODA component definition yaml file the class of ODA Component
        required: True
    interfaces:
      Standard:
        operations:
          create:
            inputs:
              oda_component_instance_name: { get_property:  [SELF, node_instance_name ] }
              oda_component_defn: { get_property: [ SELF, definition_file ]}

  TMFC001:
    derived_from: oda_component
    properties:
      component_class_ID: TMFC001
      component_class_name: foo
      component_class_short_name: bar
    capabilities:
      TMF620_capability:
        type: TMF620

  TMFC002:
    derived_from: oda_component
    properties:
      component_class_ID: TMFC002
      component_class_name: Product Order Capture & Validation
      component_class_short_name: PrdOrdCptVd
    requirements:
      - TMF620_requirement:
          capability: TMF620
          relationship:
            type: oda_component_depends_on_TMF_API
          # TODO Don't really want to declare a node type here as a node with the required capability will be allocated by the topology template designer. 
          # But some parsers can't resolve the reference properties on the TARGET node without knowing the node type of the node.
          # In this use case it would be acceptable to supply the name of the parent, abstract, node type and that does does define the referenced properties but that is not allowed becuase that parent node type does not have the capability TMF620
          node: TMFC001

... Other nodes types derived from derived_from: oda_component and with different capabilities derived from TMFAPI follow but are ommited

[pmjordan]

My use case has another reason why making the keyword 'node' mandatory at this point is problematic: I need to define a profile in which a node type has a requirement for a capability but the node type with that capability is obtained from some other profile written at a later time. Of course the topology template will import both profiles so that relationships can be resolved at deployment time.

[philippemerle]

You could comment node: TMFC001 in your extract as no properties of TMFC001 are accessed from the requirement definition.

[lauwers]

Philippe is correct. The “extract” you provided does not illustrate the problem you have highlighted.

In the comments, you state:

TODO Don't really want to declare a node type here as a node with the required capability will be allocated by the topology template designer.

But some parsers can't resolve the reference properties on the TARGET node without knowing the node type of the node.

This is the correct behavior. If a ‘get_property’ function tries to access a property on a TARGET node without knowing the node type of that target node, then the validator must flag this as an error, since there is not guarantee that the TARGET node has a property with that name, let alone that the type of that property is compatible with the type of the parameter with the ‘get_property’ function. TOSCA is designed to catch these types of errors at design time rather than waiting for runtime errors to occur.

From: Philippe Merle @.> Sent: Wednesday, March 16, 2022 2:44 AM To: oasis-open/tosca-community-contributions @.> Cc: Subscribed @.***> Subject: Re: [oasis-open/tosca-community-contributions] Should a relationship definition be allowed to refer to properties of a target node type before the type of the node has been declared/resolved? If not then the Node clause should be documented as required...

You could comment node: TMFC001 in your extract as no properties of TMFC001 are accessed from the requirement definition.

— Reply to this email directly, view it on GitHubhttps://github.com/oasis-open/tosca-community-contributions/issues/86#issuecomment-1068926129, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AASPLINZAV77R2K2VABQKKLVAGUPFANCNFSM5QNLOKJA. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you are subscribed to this thread.Message ID: @.**@.>>

[pmjordan]

We discussed this issue during the Language Ad hoc meeting March 29th. As a result I think it would best to ignore my (poor) example go back to my use case:

I have number of software components each providing REST APIs and consuming the APIs from other such components. Both components and APIs are specified by the standards body, TMForum but implemented by independent vendors and will be assembled into topologies by a third group of users. When deploying such a topology an API consumer node must be informed of the URL of the provider node.

One option would be to hold the URL as an attribute of a TOSCA capability but this is not viable as ‘get-attribute’ does not currently allow access to attributes of a capability of a node. This may be resolved by a solution to issue66

A workaround to the above restriction is to hold the URL as an attribute of a TOSCA relationship. The node declaring the requirement has to include not only the capability and the interface containing the get-attribute clause but also a node clause. The node clause is required in this case so that the processor can confirm that the target node type has defined the referenced attribute. But this leads to a number of difficulties:

1. There may be more than one valid node type for the target. For example the relationship might be terminated on an API gateway or proxy instead of directly on the providing component. Nodes representing these gateways would all have to have the named attribute but may not share a common class. The node clause cannot contain a list of node types.

2. If we disregard 1. then we have the case where all possible target nodes are of a type derived from a single, common, node type. The processor can thus confirm that the node type does indeed contain the attribute named in the get-attribute clause. But when the node clause is present in the requirement definition the processor also checks that the capability is present in the referenced node type. However it is possible that the capability is only added as a refinement in one or more children of common node type but not in the common node type itself. If so the test will fail. There is no way to reference both the common type for one test and the list if its children for the other test.

[philippemerle]

@pmjordan: Following is a try to encode your use case where I removed all elements unrelated to the issue.

tosca_definitions_version: tosca_simple_yaml_1_3
capability_types:
  TMFAPI:
    derived_from: tosca.capabilities.Node
    attributes:
      api_provider_sap:
        description: The IP address of the server providing the API
        type: string

  TMF620:
    derived_from: TMFAPI

relationship_types:
  oda_component_depends_on_TMF_API:
    derived_from: tosca.relationships.DependsOn
    description: Used to connect an ODA component which requires an API with an ODA component which has such an API
    valid_target_types: [ TMFAPI ] # i.e. This relationship is only to be used to connect to nodes which have a capability derived from TMFAPI

node_types:
  oda_component:
    derived_from: tosca.nodes.Root

  TMFC001:
    derived_from: oda_component
    capabilities:
      TMF620_capability:
        type: TMF620

  TMFC002:
    derived_from: oda_component
    requirements:
      - TMF620_requirement:
          capability: TMF620
          relationship: oda_component_depends_on_TMF_API
    interfaces:
      Standard:
        operations:
          create:
            inputs:
              url_of_TMF620_requirement:
                type: string
                value: { get_attribute: [ SELF, TMF620_requirement, api_provider_sap ] }

@pmjordan: Is this example illustrated your use case?

Note that the last line get_attribute: [ SELF, TMF620_requirement, api_provider_sap ] expression raises a not found error with both Cloudnet TOSCA toolbox and Ubicity. But perhaps this is an issue of both these parsers. All depends on the semantics we give to the get_attribute: [ SELF, TMF620_requirement, api_provider_sap ] expression.

[philippemerle]

The get_attribute: [ SELF, TMF620_requirement, api_provider_sap ] expression seems a valid expression according to Section 4.5.1.2 of TOSCA 1.3 or Section 5.5.1.2 of TOSCA 2.0 csd03.

[philippemerle]

The not found error produced by Cloudnet TOSCA toolbox seems to be a bug, have a look to https://github.com/Orange-OpenSource/Cloudnet-TOSCA-toolbox/issues/56.

[lauwers]

I don't believe the get_attribute: [ SELF, TMF620_requirement, api_provider_sap ] expression is valid, although Section 5.5.1.2 of TOSCA 2.0 csd03 is not very helpful here. Based on the language in that section, TMF620_requirement refers to the optional name of the requirement or capability name within the modelable entity (i.e., the <modelable_entity_name> which contains the attribute definition the function will return the value from. In this case, TMF620_requirement is clearly a requirement, but requirements don't have attributes. You could argue that a parser should look for attributes of the corresponding relationship instead. However, examples in the spec show that what should be done is to look for "sub-entities" in the node that is the target for the TMF620_requirement. Unfortunately, without defining the target node type, a parser has no way of knowing if the target node has the named attribute, let alone what its type is.

[philippemerle]

Section 5.5.1.3 states "The attribute functions are used in the same way as the equivalent Property functions described above. Please see their examples and replace “get_property” with “get_attribute” function name". The last line of the second example of Section 5.4.2.3 contains the get_property: [ SELF, database_endpoint, port ] expression that could mean get the property port of the requirement database_endpoint. But requirements have no properties. So I interpret this expression as get the property of the capability connected to the requirement database_endpoint. If this is the correct interpretation, then get_attribute: [ SELF, TMF620_requirement, api_provider_sap ] means get the attribute of the capabilitity connected to the requirement TMF620_requirement.

[lauwers]

My interpretation of that example is different: the database_endpoint requirement is fulfilled by a node of type tosca.nodes.Database, which has a property named port. It is that node property that is referenced.

[philippemerle]

Both your and my different interpretations seem valid so this is an ambiguity here that should be resolved explicitely in the spec.

[philippemerle]

Example 13 in Section 2.9.3 of TOSCA 1.3 contains the get_property: [SELF, database_endpoint, url_path ] expression. database_endpoint is a requirement and url_path is a property defined in tosca.capabilities.Endpoint, not in tosca.nodes.Database. So I interpret get_property: [SELF, database_endpoint, url_path ] as get the property url_pathof the capability connected to the requirement database_endpoint.

[lauwers]

Yes, we need to resolve the ambiguity, but more importantly we need to extend the syntax to support all three use cases:

1. Retrieve property values from the target capability of a requirement
2. Retrieve property values from the target node of a requirement
3. Retrieve property values from the relationships established by fulfilling a requirement In addition, we have cases where a node has requirements and capabilities that use the same name (e.g. "host"). This introduces additional ambiguity in the get_property and get_attribute syntax.

[philippemerle]

A proposal:

1. get_attribute: [SELF, REQUIREMENT, <requirement_name>, CAPABILITY, <attribute_name> ] means retrieve the value of the attribute attribute_name from the target capability of a requirement requirement_name
2. get_attribute: [SELF, REQUIREMENT, <requirement_name>, NODE, <attribute_name> ] means retrieve the value of the attribute attribute_name from the target node of a requirement requirement_name
3. get_attribute: [SELF, REQUIREMENT, <requirement_name>, RELATIONSHIP, <attribute_name> ] means retrieve the value of the attribute attribute_name from the relationship established by fulfilling a requirement
4. get_attribute: [SELF, CAPABILITY, <capability_name>, <attribute_name> ] means retrieve the value of the attribute attribute_name from the capability of the self node

[philippemerle]

@pmjordan : expression get_attribute: [ SELF, TMF620_requirement, api_provider_sap ] is now valid for Cloudnet TOSCA toolbox, see https://github.com/Orange-OpenSource/Cloudnet-TOSCA-toolbox/issues/56

[lauwers]

Calin has made a similar proposal in the past for extending the "TOSCA Path" syntax. You can find it at the end of the following document:

https://www.oasis-open.org/committees/document.php?document_id=66199&wg_abbrev=tosca

[lauwers]

By the way, Philippe is correct that example 13 in Section 2.9.3 of TOSCA 1.3 suggests that the property value is intended to be retrieved from the target capability, not from the target node. However, the text in the explanation suggests exactly the opposite (emphasis on node is mine):

This example also shows how the get_property intrinsic function can be used to retrieve
the url_path property from the database **node** that will be selected by the provider and 
connected to my_app at runtime due to fulfillment of the database_endpoint requirement. 

[lauwers]

We should also revisit the SOURCE and TARGET keywords in the context of relationships. These keywords clearly refer to the source and target nodes for the relationship, but there is no syntax in a relationship type or relationship template definition to specify the types of those nodes. This means that it may not be possible to validate correctness of a get_property function that uses these keywords. The example in section 4.4.2.3 uses a TARGET keyword inside a relationship template that is impossible to validate. We may need to extend the "valid_source_types" and "valid_target_types" keywords, and we should also clarify when "valid_target_types" refers to capability types and when it refers to node types.

[lauwers]

My YAML parser doesn't like the '' character we (sort of) agreed on to specify that we expect a list of all values (as opposed to just one single value). It expects the name of an aliased node after the ''. We need to decide on different syntax (or a different keyword) to specify lists (and no, let's not use -1 :-))

[pmjordan]

Is that '' intended to be an asterix?

Perhaps we need an explict sequence to denote a filter, even if initially the only allowed filter value is the one we use to denote match all. Otherwise later we will have three cases rather than two (the three cases would be an integer denoting an index, an asterix for match all, some other demlited filter sequence).

A;so are we conflating two related concepts? One is defining a selection filter and the other is the required format of the results. We only have one filter at the moment, 'match all' denoted with an asterix character but expect more filters later. We have considered two return formats so far, a single value and a list.

This discussion is more appropriate to issue 66

[pmjordan]

I think this issue can be closed as we now have TOSCA Path. However we need a new issue to record the comment below from @lauwers

"We should also revisit the SOURCE and TARGET keywords in the context of relationships. These keywords clearly refer to the source and target nodes for the relationship, but there is no syntax in a relationship type or relationship template definition to specify the types of those nodes. This means that it may not be possible to validate correctness of a get_property function that uses these keywords. The example in section 4.4.2.3 uses a TARGET keyword inside a relationship template that is impossible to validate. We may need to extend the "valid_source_types" and "valid_target_types" keywords, and we should also clarify when "valid_target_types" refers to capability types and when it refers to node types"

[lauwers]

Now that we can specify valid source node types and valid target node types, we should theoretically be able to validate any properties or attributes referenced using the SOURCE and TARGET keywords in a TOSCA Path. For example, within a relationship, the following function refers to an 'address` property in the target node of the relationship:

$get_property: [ SELF, TARGET, address]

However, what is the expected behavior when there are multiple valid target node types? I assume that at the very least, all of these target node types must have an address property. However, is it also required that each of these address properties have the same type, or is it sufficient that they have a common parent type?

[pmjordan]

I would wish that having a common parent type is sufficent.