Documentation generation - REST API description does only contain curl requests

[de-jcup]

Situation

The generated REST API description does only contain curl requests - for example:

Wanted

As a developer I want more information about the REST API.

Analyze

Reading https://docs.spring.io/spring-restdocs/docs/current/reference/html5/ respectively https://docs.spring.io/spring-restdocs/docs/current/reference/html5/#documenting-your-api-default-snippets

By default, six snippets are written:

<output-directory>/index/curl-request.adoc

<output-directory>/index/http-request.adoc

<output-directory>/index/http-response.adoc

<output-directory>/index/httpie-request.adoc

<output-directory>/index/request-body.adoc

<output-directory>/index/response-body.adoc

These are the default, but we have not

• "request-fields.adoc"
• "resopnse-fields.adoc"

files generated per default

Reading https://docs.spring.io/spring-restdocs/docs/current/reference/html5/#configuration the "request-fields.adoc" files should still be generated when defined.

Reading https://docs.spring.io/spring-restdocs/docs/current/reference/html5/#configuration-default-snippets it seems the configuration must be changed.

Solution

[de-jcup]

Analyze

Remark: I was already writing an issue for Spring-Rest project, but after creating a simple demo, the problem was not inside the demo, so this is NOT a spring rest doc problem.

But having the information already written, I just added it here to have the details at least here

In a nutshell

Our existing spring-restdoc testcases do no longer output

• "request-fields.adoc"
• "response-fields.adoc"
• "path-paramters.adoc"

Spring boot versions: `2.6.4 Spring rest docs core: 2.0.5.RELEASE Spring rest docs mockmvc: 2.0.5.RELEASE

Additional: we use a special additional epages wrapper for the openapi asciidoc generation.

Details

Example: https://github.com/mercedes-benz/sechub/blob/bcd5f74c9de3e59897d642fd59e07f1572cd9125/sechub-doc/src/test/java/com/mercedesbenz/sechub/restdoc/UserAdministrationRestControllerRestDocTest.java

Here a snippet from former mentioned file:

    @Test
    @UseCaseRestDoc(useCase = UseCaseAdminListsAllAdmins.class)
    public void restdoc_list_all_admins() throws Exception {
        /* prepare */
        String apiEndpoint = https(PORT_USED).buildAdminListsAdminsUrl();
        Class<? extends Annotation> useCase = UseCaseAdminListsAllAdmins.class;

        List<String> admins = new LinkedList<>();
        admins.add("admin1");
        admins.add("admin2");

        when(userListService.listAdministrators()).thenReturn(admins);

        /* execute + test @formatter:off */
        this.mockMvc.perform(
                get(apiEndpoint)
                )./*andDo(print()).*/
        andExpect(status().isOk()).
        andDo(document(RestDocFactory.createPath(useCase),
                resource(
                        ResourceSnippetParameters.builder().
                            summary(RestDocFactory.createSummary(useCase)).
                            description(RestDocFactory.createDescription(useCase)).
                            tag(RestDocFactory.extractTag(apiEndpoint)).
                            responseSchema(OpenApiSchema.USER_LIST.getSchema()).
                            responseFields(
                                    fieldWithPath("[]").description("List of admin Ids").optional()
                            ).
                            build()
                        )
                ));

        /* @formatter:on */
    }

After test execution the output inside $someLocalPath/generated-snippets/ contains only:

So the default snippets are generated (as mentioned at https://docs.spring.io/spring-restdocs/docs/current/reference/html5/#documenting-your-api-default-snippets) but there is no response-fields.adoc generated which should be (as described at https://docs.spring.io/spring-restdocs/docs/current/reference/html5/#documenting-your-api-request-response-payloads)

This is the same for all of our REST-Doc tests below https://github.com/mercedes-benz/sechub/tree/develop/sechub-doc/src/test/java/com/mercedesbenz/sechub/restdoc

We have no longer any

• "request-fields.adoc"
• "response-fields.adoc"
• "path-paramters.adoc" generated.

This had worked before with older versions of Spring Boot ( I am not sure when /on which Spring Boot version the problem appeared). I tested with Spring Boot 2.6.2 and 2.6.4 - both versions did not generate the former mentioned snippets any longer.

[de-jcup]

Analyze 2

Create a simple demo app to reproduce the problem

I created a spring boot demo application having one simple rest service and two restdoc-tests:

• Hello1StandardSimpleMessageRestControllerRestDocTest -> generates "hello1"
• Hello2OpenAPISimpleMessageRestControllerRestDocTest -> generates "hello2"
• Hello3OpenAPISimpleMessageRestControllerRestDocTest -> generates "hello3"

Compare outputs

hello1 and hello3 outputs are as expected. But for hello2 the path-parameters are missing! (demo project is added here : demo-restdoc-testing.zip )

Also comparing hello2 and hello3 resource.json output we have a big difference:

Introspect differences between hello2 and hello3

hello2 adds the path parameters directly to resource snippet params builder:

          ResourceSnippetParameters.builder().
                pathParameters(
                        parameterWithName("who").description("Who shall we greet")

                ).build()),

So we may not add this by resource snippet builder but instead only use the wrapper documentation (and not more).

https://github.com/ePages-de/restdocs-api-spec#mockmvc-based-tests

In your tests you can just replace calls to MockMvcRestDocumentation.document with the corresponding variant of MockMvcRestDocumentationWrapper.document.

MockMvcRestDocumentationWrapper.document will execute the specified snippets and also add a ResourceSnippet equipped with the input from your snippets.

But what currently does not work is adding details like tag, description etc.- when using the parameter resource builder all the data get lost again.

[de-jcup]

I found a solution - it is inside the attached demo-restdoc-testing-resolved.zip the hello6 variant.

The solution does create a ResourceSnippetDetails object and calls the document method which uses this one. Here the details will be merged with other data and all will work as wanted.

Here the example test code:

package com.example.demo;

import static org.springframework.restdocs.mockmvc.RestDocumentationRequestBuilders.*;
import static org.springframework.restdocs.payload.PayloadDocumentation.*;
import static org.springframework.restdocs.request.RequestDocumentation.*;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.*;

import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.restdocs.AutoConfigureRestDocs;
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.web.servlet.MockMvc;

import com.epages.restdocs.apispec.MockMvcRestDocumentationWrapper;
import com.epages.restdocs.apispec.ResourceSnippetDetails;
import com.epages.restdocs.apispec.ResourceSnippetParameters;

@WebMvcTest(SimpleMessageRestController.class)
@ContextConfiguration(classes = { SimpleMessageRestController.class})
@AutoConfigureRestDocs()
public class Hello6OpenAPISimpleMessageRestControllerRestDocTest {

    @Autowired
    private MockMvc mockMvc;

    /* @formatter:off */
    @Test
    public void restdoc_admin_updates_user_email_address() throws Exception {

        ResourceSnippetDetails resourceSnippetDetails = ResourceSnippetParameters.builder().
                    summary("summary1").
                    description("description1").
                    tag("tag1");

        /* prepare */
        /* execute + test @formatter:off */
        this.mockMvc.perform(
                get("http://loalhost:8080/hello/{who}","World")
                )./*andDo(print()).*/
        andExpect(status().isOk()).
        andDo(MockMvcRestDocumentationWrapper.document("hello6",
                         resourceSnippetDetails,
                         pathParameters(
                                parameterWithName("who").description("Who shall we greet")

                        ),responseFields(
                                fieldWithPath("greeting").description("The greeting"),
                                fieldWithPath("time").description("The time when x was greeted")

                         )

                ));

        /* @formatter:on */
    }
}

Whats next?

Change all existing SecHub REST DOC tests :

1. Do not use static imports from com.epages at all!
2. Instead import the class and use com.epages static methods by the classname!
3. Handle the resource details by creating a dedicated ResourceSnippetDetails and give it to documents methods after identifier

[de-jcup]

Maybe It will be a good option to provide own static methods here with complete rename - so we would not have to define MockMvcRestDocumentationWrapper.document( but instead:

import static com.mercedesbenz.sechub.restdoc.RestDocumentation;

//...

andDo(forRestService()).
                withTag("tag1").
                withDescription("description").
                withId("hello7").
                document(...)
      );

This will reduce possiblities to use documentation api wrong...

[de-jcup]

The final API is even more simpler and there is no longer a need to use "com.epages.*" directly - here an example:

import static com.mercedesbenz.sechub.restdoc.RestDocumentation;

andDo(defineRestService().
                with().
                    useCaseData(useCase).
                    tag(RestDocFactory.extractTag(apiEndpoint)).
                    requestSchema(OpenApiSchema.FALSE_POSITVES_FOR_JOB.getSchema()).
                and().
                document(...));