Question: Would it be possible to support ApiExplorer for OpenApi (Swagger) ApiDocumentation?

[markphillips100]

Any ideas on an approach for supporting API documentation, what the limitations would be, etc?

I'm guessing this might be coupled to this to-do item else how would we discover the types?

[mauroservienti]

I have no idea how swagger works and if there is an option to plugin into the documentation building pipeline. If I understood correctly the model documentation requires a concrete type and won't be very happy with dynamic models.

I'll have a look, it's a valid requirement.

[mauroservienti]

Quickly looking at this article, that I have no idea how accurate it is, it seems that most of the documentation is metadata driven using attributes on controller action methods.

In theory it should work out of the box when using the ServiceComposer 1.0.0 attribute based routing, as when endpoints are constructed and added to the data source attributes found on all the requests handlers are copied to the created endpoint: https://github.com/ServiceComposer/ServiceComposer.AspNetCore/blob/2bbdcd14ed1d8c91f9382cc330a0cc34cfc88215/src/ServiceComposer.AspNetCore/EndpointsExtensions.cs#L169-L188

[markphillips100]

"won't be very happy with dynamic models" - that's my understanding, also why I linked to your project item about strong types. I guess I was wondering how that would even be achieved.

[markphillips100]

I'll take a look at the article however I suspect that the ApiExplorer's default metadata provider (which is what Swagger uses) relies on Controller types. I'll see if I can prove myself wrong.

[markphillips100]

Indeed the DefaultApiDescriptionProvider supplied by MVC is for the API metadata in MVC, so controllers and actions all the way. It seems this is being discussed a fair bit on https://github.com/domaindrivendev/Swashbuckle.AspNetCore/issues/1306.

As pointed out in this post (by the endpoint routing developer I believe) a custom IApiDescriptionProvider would be required if using Swashbuckle current features. Something apparently achieved for gRPC in dotnet, as indicated here.

The discussion about how to support endpoint routing outside of being coupled to MVC is ongoing I guess.

For now I'll see if I can get some route and comment documentation using the gRPC Api Description provider approach. Obviously all body and response types will just be dynamic types but some documentation is better than none.

[markphillips100]

I got some basic http method and route pattern documentation using the following code. To do any more would require the creation of a metadata model specifically for viewmodel composition that would allow surfacing of body and response information to the provider. The hardest problem is likely to be how to describe the merging of data across handlers in those list use cases where handlers add on their part of each item.

    internal class ServiceComposerApiDescriptionProvider : IApiDescriptionProvider
    {
        private readonly EndpointDataSource _endpointDataSource;

        public ServiceComposerApiDescriptionProvider(EndpointDataSource endpointDataSource)
        {
            _endpointDataSource = endpointDataSource;
        }

        // Executes after ASP.NET Core
        public int Order => -900;

        public void OnProvidersExecuting(ApiDescriptionProviderContext context)
        {
            var endpoints = _endpointDataSource.Endpoints;

            foreach (var endpoint in endpoints)
            {
                if (endpoint is RouteEndpoint routeEndpoint)
                {
                    var apiDescription = CreateApiDescription(routeEndpoint);

                    context.Results.Add(apiDescription);
                }
            }
        }

        private ApiDescription CreateApiDescription(RouteEndpoint routeEndpoint)
        {
            var httpMethodMetadata = routeEndpoint.Metadata.GetMetadata<HttpMethodMetadata>();
            var verb = httpMethodMetadata?.HttpMethods.FirstOrDefault();

            var apiDescription = new ApiDescription();
             // Default to a GET in case a Route map was registered inline - it's unlikely to be a composition handler in that case.
            apiDescription.HttpMethod = verb ?? "GET";
            apiDescription.ActionDescriptor = new ActionDescriptor
            {
                RouteValues = new Dictionary<string, string>
                {
                    // Swagger uses this to group endpoints together.
                    // Group methods together using the service name.
                    // NOTE: Need a metadata model in service composer to begin supplying more info other than just http verbs and route patterns.
                    ["controller"] = "ViewModelComposition"
                }
            };
            apiDescription.RelativePath = routeEndpoint.RoutePattern.RawText.TrimStart('/');
            apiDescription.SupportedRequestFormats.Add(new ApiRequestFormat { MediaType = "application/json" });
            apiDescription.SupportedResponseTypes.Add(new ApiResponseType
            {
                ApiResponseFormats = { new ApiResponseFormat { MediaType = "application/json" } },
                StatusCode = 200
            });

            return apiDescription;
        }

        public void OnProvidersExecuted(ApiDescriptionProviderContext context)
        {
            // no-op
        }
    }

Add the following to Startup's ConfigureServices method:

            services.AddSwaggerGen();
            services.AddControllers(); // Swagger needs this for the ApiExplorer.
            services.TryAddEnumerable(
                ServiceDescriptor.Transient<IApiDescriptionProvider, ServiceComposerApiDescriptionProvider>());

And then add the following to Startup's Configure method:

            app.UseSwagger();
            app.UseSwaggerUI(c => c.SwaggerEndpoint("/swagger/v1/swagger.json", "My API"));

[mauroservienti]

@markphillips100 wow, thanks. I wasn't hoping for such a detailed investigation. I scheduled some time on Friday morning this week to spike something and investigate how to support models documentation. Thanks again.

[mauroservienti]

I started experimenting with the ApiExplorer API, my initial brainstorming is available at https://github.com/ServiceComposer/Swagger-for-ServiceComposer-Playground. Feel free to comment there, more than happy to give you push access if that makes things simpler.

What I did so far is:

• tweaked your ServiceComposerApiDescriptionProvider to support 2 attributes:
  • ProducesResponseType that comes with ASP.Net Core
  • ApiParameterDescription that is a custom one to describe incoming parameters
• The easiest way to add API documentation is to register an empty request handler for each route, like the following:

namespace Swagger_for_ServiceComposer.Handlers.Documentation
{
    public class DocumentationHandler : ICompositionRequestsHandler
    {
        [HttpGet("/sample/{id}")]
        [ProducesResponseType(typeof(object), StatusCodes.Status200OK)]
        [ProducesResponseType(typeof(string), StatusCodes.Status400BadRequest)]
        [ApiParameterDescription(Name = "id", IsRequired = true, Type = typeof(int), Source = "Path")]
        public Task Handle(HttpRequest request)
        {
            return Task.CompletedTask;
        }
    }
}

Supporting incoming parameters makes so that the API can be tested in the Swagger UI:

I agree with you that a sort of metadata model is required. I'm wondering if it's easier to support strong types though. It's true that dynamic support makes a lot of sense in many contexts and strong typing is a nice addition but can't be the only option.

From the metadata model perspective one option would be to be able to support something like:

[RequestMetadataModel(Route = "sample/{id}", HttpMethod = HttpMethods.Get)]
class SampleRequestMetadataModel : RequestMetadataModel
{
   public SampleRequestMetadataModel()
   {
      //Same values as the custom ApiParameterDescription attribute
      //the attribute can still be used but attributes are quite limited API wise
      AddApiParameterDescription(Name = "id", IsRequired = true, Type = typeof(int), Source = BindingSource.Path);
   }
}

If the ApiParameterDescription in combination with the above RequestMetadataModel model covers all basic usages, then the remaining problem are the produced response types. Produced response types could be solved by using a similar approach (e.g. a ResponseMetadaModel kind of thing) or by leveraging https://github.com/ServiceComposer/ServiceComposer.AspNetCore/pull/155 which aims to introduce support for strongly typed responses. By having strongly typed response models means that reflection and data annotation can be used.

Thoughts?

[mauroservienti]

I made a significant step forward. Setting ModelMetadata when defining response types is key, otherwise nothing is generated for a given type because nothing is automatically inspected. This is far from intuitive. Anyway now I get the following:

(the Swagger-for-ServiceComposer-Playground sample is up to date with latest changes)

The trick is to set the ModelMetadata when defining ApiResponseType(s)

apiDescription.SupportedResponseTypes.Add(new ApiResponseType
{
    Type = producesResponseTypeAttribute.Type,
    ApiResponseFormats = { new ApiResponseFormat { MediaType = "application/json" } },
    StatusCode = producesResponseTypeAttribute.StatusCode,
    IsDefaultResponse = false,
    ModelMetadata = _modelMetadataProvider.GetMetadataForType(producesResponseTypeAttribute.Type)
});

_modelMetadataProvider comes from DI and is what is used by default by ASP.Net Core.

[markphillips100]

Hi @mauroservienti . I've just started delving into how to use source generators to support a composition viewmodel for openapi documentation. I forked the playground, and just pushed a working sample here.

The source generator basically looks for a new attribute ProducesCompositionResponseType where the developer specifies the type of the handler's viewmodel and what dynamic property name they will use when assigning to the view model. Currently only supports one dynamic property per handler (but why a dev would assign more than one property I don't know).

Once the attributes for all the endpoints are known, it groups by the endpoint and then creates a new unique class to represent the composition model, and adds a property for each of the relevant attributes it found for that endpoint. Lastly, it creates a new "DocumentationHandler" for the same endpoint to supply a ProducesResponseType for the new model type.

So comparing with the original playground, it merely does away with needing to create a specific DocumentationHandler and model. Just add openapi attributes (and the new composition attribute) to handlers themselves. I had to slightly modify the description provider so as to avoid duplicate dictionary key issues for ProducesDefaultResponseType (uses "default" as key), and also ProducesResponseType for the same StatusCode value (uses the status code as key).

It doesn't support ICompositionSubscriber yet, nor any package referenced handler implementations. The latter is more complicated as the those are already compiled code. Think I have a way though but wanted the basics first.

I added a controller action too just to be sure things work side-by-side.

Here's the swagger output. Let me know your thoughts please.

[mauroservienti]

Mark, thanks for all the work you're doing and more importantly for your patience.

I had a quick look and it looks promising. I'll schedule some time to go through it and do a proper review.

[markphillips100]

Not had much time lately - ongoing startup funding issues as usual - but will hopefully take a look at modification of the generator to support the following:

• Handlers residing in other packages.
• Handlers returning merged result sets.

The last one is the most tricky. As you've pointed out in the past, there's no formal way that anyone must adhere to in regards to how they merge results into a composition view model. Some could just use multiple ICompositionRequestsHandler and append their data with "their" service-specific property on the model (the simplest to cater for). Others could use ICompositionEventsSubscriber and merge their data by common key.

The latter is much harder obviously and might not be something we want to support but at least provide guidance on.

I personally use some helper extensions to allow simple merging of data by key such that each service gets their own property within each list item. This semantic approach I use for lists would mean I could, theoretically, create some custom attribute to guide the source generator in some way. It is specifically custom to my approach though. Others might want to merge each keyed item's data into one flat record, rather than service-specific property partitions inside each item. This might be doable still. The trick is how to tie a custom attribute with some semantic meaning into behaviour within the generator (so it knows what to do).

FYI, for merging my lists from subscribers, I use a common approach that uses 2 extensions. This allows me to produce results like:

{
    "items": [
        {
            "sales": {
                "orderId": "xyz",
                "seller": "seller stuff"
            },
            "delivery": {
                "orderId": "xyz",
                "deliverer": "deliverer stuff"
            }
        }
    ]
}

The typical result process method in a subscriber:

// Called from subscriber once it has the data specific to the service's request.
private static Task ProcessValueAsync(OrdersIndexRequested @event, List<OrderViewModel> value)
{
    // Append data to existing orders using the order id to group each list item.
    @event.Data.AppendToDictionary(value, order => order.OrderId, "delivery");

    @event.Data.ApplyDefaultForMissingData("delivery", null);
    return Task.CompletedTask;
}

And the extensions:

    public static class DictionaryExtensions
    {
        public static Dictionary<dynamic, Dictionary<dynamic, dynamic>> AppendToDictionary(
            this Dictionary<dynamic, Dictionary<dynamic, dynamic>> dictOuter,
            IEnumerable<dynamic> items,
            Func<dynamic, dynamic> keyAccessor,
            string dataContextPropertyName)
        {
            if (items?.Any() == true)
            {
                foreach (dynamic item in items)
                {
                    var key = keyAccessor(item);
                    if (dictOuter.ContainsKey(key))
                    {
                        var dictItem = (Dictionary<dynamic, dynamic>)dictOuter[key];
                        dictItem[dataContextPropertyName] = item;
                    }
                }
            }
            return dictOuter;
        }

        public static Dictionary<dynamic, Dictionary<dynamic, dynamic>> ApplyDefaultForMissingData(
            this Dictionary<dynamic, Dictionary<dynamic, dynamic>> dictOuter,
            string dataContextPropertyName,
            ExpandoObject defaultValue)
        {
            foreach(var key in dictOuter.Keys)
            {
                var dictItem = (Dictionary<dynamic, dynamic>)dictOuter[key];
                if (!dictItem.ContainsKey(dataContextPropertyName))
                {
                    dictItem[dataContextPropertyName] = defaultValue;
                }
            }
            return dictOuter;
        }
    }