API level policy support for API Manager

[lakshithagunasekara]

Problem

From APIM 4.1.0 onwards, we have increased the granularity of the mediation policies feature from API level to operation level. Influences for this decision were,

1. Providing flexibility to have different policies for different resources than one policy for the entire API.
2. Ability to apply policies one by one and verify them during the design time.

However, this design would be a bit problematic if customized policies per resource are not expected and the resource count is higher. This will add extra overhead during the design time as well as for the pipelines.

Solution

It was decided to implement API level policies support for the API.

The feature will be designed as such,

1. Will include the existing operation level features such as multiple policies support, policy ordering support.
2. At a given time, a single API can have only operation level policies or API level policies and If the API project included both, API level policies will be given prominence.

Affected Component

APIM

Version

No response

Implementation

No response

Related Issues

No response

Suggested Labels

No response

[lakshithagunasekara]

Update: 10th April

API schema Publisher rest api apis/{apiId} will include a new section to input the API level policies mapping.

name: PizzaShackAPI
    . . . . 
   apiPolicies:
     request:
      -
       policyName: addLogMessage
       policyVersion: v1
       policyId: 1bc149df-d150-45bc-8baf-4cdc632994f1
       parameters: {}
      -
       policyName: addHeader
       policyVersion: v1
       policyId: 1606f9dc-5da7-4681-a8c8-b230a5c2997f
       parameters:
         headerName: TestHeader
         headerValue: TestValue
     response:
      -
       policyName: addLogMessage
       policyVersion: v1
       policyId: 1bc149df-d150-45bc-8baf-4cdc632994f1
       parameters: {}
      -
       policyName: jsonToXML
       policyVersion: v1
       policyId: 4d43d54f-985a-4f20-a295-f60f00eee80f
       parameters: {}
     fault:
      -
       policyName: removeHeader
       policyVersion: v1
       policyId: 7d914d75-e70a-4b58-a6a3-6538cd995ee9
       parameters:
         headerName: TestHeader
      -
       policyName: debugJsonFault
       policyVersion: v1
       policyId: 27b613e0-c4b2-4ca9-96ea-bde6af6d9afd
       parameters: {}

Mapped policies will be same as before which includes common API policies and API level policies.

We had a couple of discussions on the subject and two implementations were carried out to see the feasibility.

Option 1: Use API policies as a collection of operation policies within the data persistence layer.

At the mapping layer, all the API-level policies will be copied to the operation level. There will be a parameter at the operation level to decide whether these policies were API level or operation level.

Persisting the API level or operation level in the database can be done either using the existing db schema (Store under parameters column in AM_OPERATION_POLICY_MAPPING table or adding a new column to the table. Implementation: https://github.com/wso2/carbon-apimgt/pull/11953

Pros: This approach will have minimal impact on the current implementation and would be ideal for backporting to APIM 4.2.0 and 4.1.0.

Cons: If the number of operations in the API is n,

• AM_OPERATION_POLICY_MAPPING table will grow with 3*n if there is one policy for request, response and fault flows. If policies are stored at API level, on 3 db entries will be stored.
• To retrieve the API, a 3*n number of DB calls will be sent to populate the API where if policies are stored at API level, only 3 db calls are sufficient.

Option2: Handle API level policies as a different flow and persist with a new table

Instead of using operation policies, there will be separate database operations for API-level policy mappings. Even though this would include different flows, we have decided to proceed with this approach. This will introduce a new data table.

[lakshithagunasekara]

Update: 24th April

Completed the draft backend implementation as per : https://github.com/wso2/carbon-apimgt/pull/11977

A new datatable is introduced to persist the API level mapping status.

Table structure

CREATE TABLE IF NOT EXISTS AM_API_POLICY_MAPPING (
   API_POLICY_MAPPING_ID INTEGER AUTO_INCREMENT,
   API_UUID VARCHAR(45) NOT NULL,
   REVISION_UUID VARCHAR(45),
   POLICY_UUID VARCHAR(45) NOT NULL,
   POLICY_ORDER INTEGER NOT NULL,
   DIRECTION VARCHAR(10) NOT NULL,
   PARAMETERS VARCHAR(1024) NOT NULL,
   FOREIGN KEY (API_UUID) REFERENCES AM_API(API_UUID) ON DELETE CASCADE,
   FOREIGN KEY (POLICY_UUID) REFERENCES AM_OPERATION_POLICY(POLICY_UUID) ON DELETE CASCADE,
   PRIMARY KEY(API_POLICY_MAPPING_ID)
)ENGINE INNODB;

Operation level policy template at the gateway is changed as

<?xml version=\"1.0\" encoding=\"UTF-8\"?>
<sequence xmlns=\"http://ws.apache.org/ns/synapse\" name=\"{{sequence_name}}\">
{% if apiPolicies is defined %}
{% for policy in apiPolicies %}
    {{policy}}
{% endfor %}
{% endif %}
{% if case_list is defined %}
    <property name="switchExpression" expression="fn:concat($ctx:REST_METHOD,'_',$ctx:API_ELECTED_RESOURCE)" />
    <switch source=\"$ctx:switchExpression\">
    {% for case in case_list %}
        <case regex=\"{{case.case_regex}}\">
        {% for policy in case.policy_sequence %}
            {{policy}}
        {% endfor %}
        </case>
    {% endfor %}
    </switch>
{% endif %}
{% if fault_sequence %}
    <sequence xmlns="http://ws.apache.org/ns/synapse" name="fault"/>
{% endif %}
</sequence>

Implemented the flows related to

1. Add/Update API level policies
2. Add/Update/delete both API level and Operation level policies
3. Create/Restore/Delete Revisions
4. Create/Update/Revision/Restore/Delete API Products
5. Deploy APIs/API products with API and operation level policies
6. Import APIs with API and Operation level policies

Remaining Tasks

1. User Interface
2. Migration testing