Advanced Services

[swang-nira]

Advanced services represents service fusion and service filtering.

Service fusion tries to combine the responses from multiple services into a merged/single response. The fusion service could be a new service.

Service filtering can either filtering the recorders from the original response, or filtering out fields/properties from the original response. The filtering service could be a new service.

Both fusion service and filtering service are different with the original services. They are more like derived services. A proposal solution is to create a new class in Profile, for example, service type: original service, fusion service, or filtering service.

At the same time, since Testbed 18 is implementing a POC about service filtering, we may want to implement this change after TB 18 finishes.

[swang-nira]

After discussed with Mark and team, there are more good ideas that we will discover. For example, a composite class to represent fusion service, which is composed by multiple original services.

[swang-nira]

[swang-nira]

Above is the proposed SCDM UML update for the service profile, by adding fusion service and filtered service

[mkaplun]

In SDCM, service is a container for classes representing service metadata (i.e., its properties). Therefore, a service (any service: information, discovery, filtering) cannot be a part of the SDCM's Service; it can/should only be described by SDCM. That is to say to be an instance of the class Service. It appears from the diagram that it is more like a Service taxonomy with two new nodes: Fusioning and Filtering. And this is not what the model (and Profile class specifically) is for.

[swang-nira]

Thanks Mark, it's a good point! I will make the update.

[swang-nira]

[swang-nira]

[swang-nira]

Please check above two changes about SDCM advanced services.

For the Profile, adding two parameters (is filtered service, is fused service), to indicate the service type. For the Model, adding two classes: Filtered Service and Fusion Service.

[mkaplun]

RE: comment by @swang-nira Two new attributes in the class Profile: The role of attributes in the class profile is collectively uniquely to identify the described service. To classify a service by some technological or architectural solution, the SDCM uses the Service Category taxonomy. The proposed solution breaks the logic behind the model design for the reason that it is not clear. Furthermore, it creates an undesirable precedent. If the SWIM decides to deploy microservices, will this require adding the attribute "is microservice"?

[mkaplun]

RE: comment by @swang-nira The concerns outlined in this comment are very similar to the ones described in comment by @mkaplun. The proposed changes contradict the general principles of the SDCM. In the SDCM, the Service is a top-level class containing, among others, the property-class Model. The Model represents a description of the Service's interface. In other words, an interface is a part of a service. In the diagram, two services, Filtered [sic] and Fusion, are shown as the aggregated properties of the interface (Model). Moreso, some Services are shown as composites of classes Filtered and Fusion and sub-classes of an interface at the same time. Together, this design is very confusing, and hard to visualize how it could be implemented as a service description in either a document or registry format.

[swang-nira]

RE: comment by @swang-nira](https://github.com/faa-swim/sdcm/issues/5#issuecomment-1187608101) Two new attributes in the class Profile: The role of attributes in the class profile is collectively uniquely to identify the described service. To classify a service by some technological or architectural solution, the SDCM uses the Service Category taxonomy. The proposed solution breaks the logic behind the model design for the reason that it is not clear. Furthermore, it creates an undesirable precedent. If the SWIM decides to deploy microservices, will this require adding the attribute "is microservice"?

Good point. I made the update according to Mark's feedback as below

[swang-nira]

[swang-nira]

RE: comment by @swang-nira The concerns outlined in this comment are very similar to the ones described in comment by @mkaplun. The proposed changes contradict the general principles of the SDCM. In the SDCM, the Service is a top-level class containing, among others, the property-class Model. The Model represents a description of the Service's interface. In other words, an interface is a part of a service. In the diagram, two services, Filtered [sic] and Fusion, are shown as the aggregated properties of the interface (Model). Moreso, some Services are shown as composites of classes Filtered and Fusion and sub-classes of an interface at the same time. Together, this design is very confusing, and hard to visualize how it could be implemented as a service description in either a document or registry format.

Thanks for the feedback, Mark. Based on your suggestion, I made another version as below. Hopefully it's more intuitive, or may not. One thing I do want to mention is that there are some complexities about fusion service and filtered service. For example, the fusion service can not only be based on original service, but also could be based on filtered service, and even based on another fusion service. Same thing for filtered service. I believe that's the reason why making the relationship is complex.

Please let me know how do you think.

[swang-nira]

[wznira]

My understanding is that - Fusion and Filtering are more about how a service is implemented, e.g., an "IAD Weather Service" provides the weather information at IAD airport by filtering data output from ITWS. It is unlikely that a consumer of "IAD Weather Service" is interested in the fact that the service is created by filtering and fusing other services. In fact, the provider can reimplement the "IAD Weather Service" and remove the fusion component without affecting the service interface.

That being said, "queryable properties" from a service could be part of the service description as we are discussing in this thread. In other words, a service can advertise that it can be filtered to create new services and the filtering capabilities it supports. I am not sure we want to wait until TB18 to conclude before addressing this issue, but this seems more reasonable for the service description.

[caroluri]

I agree with wznira. IMO, "Fusion" and "Filtering" are methods of implementation. If possible, using existing SDCM fields to capture the info should be considered. E.g., the existing field 3.20 Processing Consideration might be used to describe the filters employed on the data, or the 3.11 Function field might describe the filtering or fusing functionality provided by the service. If a new field like "Supports Filters" is desired from an advertising point of view, it could be added to the Profile.

[mkaplun]

Since we are obviously going nowhere, maybe it is necessary to revisit SDCM's most fundamental concepts. To do this, I want to post several statements/questions which should be answered either "True" or "False." In SDCM:

1. Service Description describes one and only one service.
2. Every service has one (or more) service interface, that is, an interface is a part of a service; the opposite is not true.
3. Service Description can describe any kind of service, regardless of its category, by using Profile, Model, and Grounding.
4. Fusion and Filtering services are intermediary services or Service Intermidiatry. In order to continue building on these statements, I would like to receive answers first (True or False).

[swang-nira]

Hi, Mark,

I would say "True" to all your four questions.

Thanks, Sam

[wznira]

Since we are obviously going nowhere, maybe it is necessary to revisit SDCM's most fundamental concepts. To do this, I want to post several statements/questions which should be answered either "True" or "False." In SDCM:

1. Service Description describes one and only one service.
2. Every service has one (or more) service interface, that is, an interface is a part of a service; the opposite is not true.
3. Service Description can describe any kind of service, regardless of its category, by using Profile, Model, and Grounding.
4. Fusion and Filtering services are intermediary services or Service Intermidiatry. In order to continue building on these statements, I would like to receive answers first (True or False).

True to all questions.

[caroluri]

All four statements are true.

[mkaplun]

1) This diagram contradicts statement 2. Services cannot be part of an Interface class described in Module.

2) As the description of SDMX in the NSRR demonstrates, an API Web service can be described by using the SDCM model. However, it also shows the SDCM may not have all desired attributes for describing the API (e.g., resources, type of queries). So, I would suggest starting with the question: "What attributes or characteristics that presently are not in the SDCM are necessary to fully describe an API Web service?" I believe that after those attributes are determined, the next step is to start looking at where and how those pieces may fit in.

3) Another task will be to decide how to approach service composition in the SDCM. We all agreed that "Service Description describes one and only one service." (Statement 1). The question then, if a service provides a composition of two or more services, should its description structure be any different? Apparently, a potential consumer may want to know about composite services, for example, combined services that offer both Flight and Weather data in a single response. This information may be presented as part of service functionality and the interface's definition of data or operations offered by the service. Either way it will be a description of a single service.

Summarizing: To start with, define what pieces of information are currently missing that are necessary to describe API Web service effectively.

[caroluri]

I wanted to add a reminder to consider the SDCM Extension for REST-style web services along with SDCM itself when deciding what additional SDCM attributes are needed for describing API.

[swang-nira]

Hi, Mark,

Let me try to address your questions:

1. This diagram contradicts statement 2. Services cannot be part of an Interface class described in Module.

Could you explain more about this question? In the diagram, the services are kind of "Model", not part of interface

2. As the description of SDMX in the NSRR demonstrates, an API Web service can be described by using the SDCM model. However, it also shows the SDCM may not have all desired attributes for describing the API (e.g., resources, type of queries). So, I would suggest starting with the question: "What attributes or characteristics that presently are not in the SDCM are necessary to fully describe an API Web service?" I believe that after those attributes are determined, the next step is to start looking at where and how those pieces may fit in.

Agree. As a starting point, please refer the above diagram (https://user-images.githubusercontent.com/97531831/180861820-e55a701f-8d44-4a7c-a9fa-de3fe2f72ca5.png). In the service category, there are two new attributes: fused service and filtered service

3. Another task will be to decide how to approach service composition in the SDCM. We all agreed that "Service Description describes one and only one service." (Statement 1). The question then, if a service provides a composition of two or more services, should its description structure be any different? Apparently, a potential consumer may want to know about composite services, for example, combined services that offer both Flight and Weather data in a single response. This information may be presented as part of service functionality and the interface's definition of data or operations offered by the service. Either way it will be a description of a single service.

Good point! I will give more thoughts about that

Thanks, Sam

[mkaplun]

In response to the comment by @swang-nira:

1. This diagram contradicts statement 2. Services cannot be part of an Interface class described in Module.

Could you explain more about this question? In the diagram, the services are kind of "Model", not part of interface

A Model is a logical grouping (a container) of information about a service interface. A Service Description is a container for information about a service. All of us agreed to these two statements:

1. Service Description describes one and only one service.
2. Every service has one (or more) service interface, that is, an interface is not a part of a service; the opposite is true.

So the idea of making a Service (or Services) part of a Model contradicts the whole architecture of the SDCM. In the SDCM the Model does not describe Service, Service Description does; the Model describes the Service Interface.

[caroluri]

There may be some confusion about the word "model" (not "module"). In SDCM v2, a Service Description has 3 parts called Profile, Model, and Grounding, where Model is defined as "The part of a service description that tells a service requester how to construct an invocation message and interpret a response message." A service may have more than one interface, so the Model part of its description may describe more than one interface.