Support parameterize sub section of resource path segment

lnash94 commented 1 year ago

Description: For some use cases of the resource function in the OpenAPI tool, we receive an endpoint with a parameterized sub-section in the path. This issue was previously discussed in https://github.com/ballerina-platform/ballerina-lang/issues/36900, which was raised in ballerina-lang. Subsequently, we received a solution for a previous path segment issue, as discussed in https://github.com/ballerina-platform/ballerina-spec/issues/1138.

However, when generating more connectors, we observed more complex path segment scenarios with multiple path parameters that are hard to handle via the suggested solution.

Here are some examples that we encountered:

01 . One path segment has multiple path parameters

URL endpoint : /{section}.{format} - nytimes.topstories connector
URL endpoint : /payroll/v1/workers/{associateoid}/organizational-pay-statements/{payStatementId}/images/{imageId}.{imageExtension} -adp.paystatements connector

02 . Path segment has special character with the path parameter

URL endpoint : /v3/ClientGroups/GetClientGroupByUserDefinedIdentifier(UserDefinedIdentifier='{userDefinedIdentifier}') - karbon connector
URL endpoint : /companies({company_id})/items({item_id}) - microsoft.dynamics365 connector

Will it be possible to handle this complexity within the resource-path in resource function?

@jclark, @sameerajayasoma, @hasithaa Could you please provide some insight into the handle above scenarios in the resource functions path? currently, we use resource functions in both service and client generation in the OpenAPI tool.

aneeshafedo commented 1 year ago

The another scenario that the suggested solution is not valid is when the path parameter is an integer type.

/form/{id}.json:
    get:
      tags:
        - Form
      summary: Get the details of the specified form
      operationId: getFormById
      description: Get the details of the specified form
      parameters:
        - description: Form ID
          in: path
          name: id
          schema:
            type: integer
          required: true
      responses:
        '200':
          description: Successful response
          content:
            application/json: 
              schema:
                $ref: "#/components/schemas/Form"

lnash94 commented 1 year ago

Description

We receive an endpoint with a parameterized sub-section containing special characters in the path for some use cases of the resource function in the OpenAPI contract.
Here is the sample /v4/spreadsheets/{spreadsheetId}/values/{range}:clear. Here range is a path parameter and :clear is a part of the path.

When generating the resource function for the above URL, this gave syntax issues including missing slash token.

Resource function with URL

 resource function get v4/spreadsheets/[string spreadsheetId/values/[string range]\:clear() returns string {
        // function logic 
        return "test";
    }

To overcome this syntax issue, this workaround was suggested to handle the URL within the function body.

Previously suggested workaround

resource function get v4/spreadsheets/[string spreadsheetId/values/[string rangeClear]() returns string|error {
     if !rangeClear.endsWith(":clear") { return error("bad URL"); }
    string range = rangeClear.substring(0, rangeClear.length() - 7);
       // function logic
        return "test";
    }

Problem

When we attempted to apply the workaround to the APIs listed below, Example 01:

/v4/spreadsheets/{spreadsheetId}/values/{range}:append /v4/spreadsheets/{spreadsheetId}/values/{range}:clear /v4/spreadsheets/{spreadsheetId}/values/{range}

we noticed that all paths led to the same route /v4/spreadsheets/[string spreadsheetId]/values/[string ranageAppendClear]. Therefore, we had to enhance the workaround logic to distinguish the specific API.

We encountered several challenges while handling this:

Each API is enriched with different query parameters, and the final output URL needs to be serialized with these query parameters. Some APIs have as many as 11 query parameters.
Each API has a different response type, request body, and header parameters.

If we use a single resource function for all the mentioned URLs, users may experience some confusion, and it may not be the most user-friendly approach. Due to the complexity of managing these parameters from a user perspective in scenarios [example 01, 02], we have chosen to implement the following solution as the most effective way to address this complexity:

Proposed New Solution

To create a resource-relative path that replaces special characters with '/' in the parameterized path segment to provide an access URL. Within our function body, we can then arrange the original URL for the API call.
We will also include a warning message for users while generating ballerina client code, informing them to use modified URLs when they need to access a specific URL with a path parameter segment containing special characters.

Original URL: /v4/spreadsheets/[ string spreadsheetId]/values/[string range]:clear

Suggested resource function Modified relative path : v4/spreadsheets/[string spreadsheetId]/values/[string range]/clear

resource isolated function post v4/spreadsheets/[string spreadsheetId]/values/[string range]/clear() returns ClearValuesResponse|error {
string resourcePath = string `/v4/spreadsheets/${getEncodedUri(spreadsheetId)}/values/${getEncodedUri(range)}:clear`;
//logic for query parameter serialization
ClearValuesResponse response = check self.clientEp->post(resourcePath, request);
return response;
}

More reference to APIs

Some Contract has around 14 Paths that can have similar behaviour Example 02

/v4/{name}:accept /v4/{name}:associate /v4/{name}:clearAssociation /v4/{name}:complete /v4/{name}:decline /v4/{name}:fetchVerificationOptions ...

Appreciate your input regarding this new approach.

hasithaa commented 1 year ago

I recommend the following modification to @lnash94's proposal:

Original URL: foo/[string x]:clear/bar/[string y] Proposed: foo/[string x]/clear/bar/[string y] New: foo/[string x]/:clear/bar/[string y]

We can consider the following as an alternative solution. When we come across a URL not compatible with the language resource syntax, we could default to the remote function syntax for clients. This would result in a mixture of resource and remote functions for a given OpenAPI specification. While generating remote functions for services is feasible, it seems odd to me. Still service dispatching has to support this.

jclark commented 1 year ago

Ballerina's model of a resource path is a list of string segments. A URL is also syntactically a list of segments. So it's natural by default for the HTTP lister to map a URL segment onto a resource path segment. But if the application wants to use its own characters to delimit the parts of its URL, it's open to the HTTP listener to do things differently, based for example on annotations.

So I'm wondering if we could use regexps in annotations to tell the listener how to map an apps URLs onto a list of segments.

hasithaa commented 1 year ago

One possible syntax is to allow something like a string destructuring pattern in a URL segment (something similar to regex capturing groups).

e.g. [string a]somestring/anotherstring[string b]/

jclark commented 1 year ago

Talking to @hasithaa this is mostly a client problem. When the HTTP API isn't very RESTful (as in the Google case), it seems to me that remote methods might work better than resource functions. It doesn't seem to me to be a good thing to force the API user to be aware of the syntactic details of the application-specific URLs.

ballerina-platform / ballerina-spec