Explore API Gateway options

[dborovcanin]

With the recent changes in architecture caused by Auth service and SpiceDB, there is a need for API aggregation. A simple example is Domain removal in Auth svc, which should result in all the Things and Groups (and Users) being removed (from the domain). This causes messy and circular dependencies between Auth and other services. Extracting API aggregation and authorization to API GW would solve this problem. It would also simplify domain services. One more benefit is that it would remove the need for private API because internally, we can have all the data we need (say, when we need an email for users), and API GW will take care that those data are not leaking to the end-user.

[arvindh123]

Possibilities usage of API Gateway in Magistrala

• Authentication and Authorization
  Moving Auth PEP (Policy Enforcement Point) out of service

• API Aggregation

  • Retrieve Thing connected channels , Retrieve thing connected channels policies from SpiceDB and retrieve from channels service. SpiceDB doesn't support Offset Pagination (Exists already in Channels service)
  • Retrieve channel connected things, Retrieve channel connected thing policies from SpiceDB and retrieve from things service . SpiceDB doesn't support Offset Pagination (Exists already in things service)
  • Retrieve Group's sub-groups and it related users, things, channels. Same way as describe in above two cases (Exists already in respective entity service)
  • Assigning & Unassigning Entities. Checking the entities in Service and it relations in SpiceDB , after that sending assign/unassign policies request to SpiceDB (Exists already in things service)
  • Assigning entities under domain during the creation process (Exists already in things service)
  • Domain Delete process ,As describe in Domain Delete docs.
  • Entities delete process, Helps in removing policies

• Basic Data Aggregation

  • Retrieve created and update user for all the query
  • Retrieve the domain information for the entities retrieve query (This is not required because all actions takes place on domain level)

API Gateway Advantages :

• Service layer have only business logic and other things are handled in API Gateway
• Authentication
• Authorization
• Rate Limiting
• Caching

Alternative for API Gateway Retrieve Basic data

• Aggregate in front end or UI service

[arvindh123]

Domain Delete

Domain Delete Process required of following

• In Auth Service: Remove Domain from repo and its related policies
• In Things Service : Remove Domain related things ,channels both in respective repos and their its related policies.
• In Users Service: Remove Domain related groups in repo and its related policies

Current Delete API Endpoints in other entities:

At present we have HTTP API endpoints for deleting in other entities are:

• Things
• Channel
• Groups

Standard Approach for delete domain using existing API

On receive of delete domain request following steps will taken by the Delete function in Domain Service

• Change the state of Domain to Delete
• Retrieve All things , channels, groups related to the domain Delete function in Domain service can retrieve all entities belongs to domain by using SDK
• Send delete request to retrieve entities (send individual or bulk delete request to respective entities ) Delete request can sent be to respective entities by domain service Delete function using SDK
• Delete domain from repo and its related policies

Advantages :

• No need for coding new service or endpoints
• Effectively reusing of existing code and endpoints

Disadvantages:

• Cycle request is sent back entities to auth service (where the domain service resides) for delete the entities policies.
• APIs are only in HTTP endpoints, there is no gRPC endpoints
• Response to delete request takes long time if there are many entities in domain, Delete performance will be poor
• If delete process get into error, Then all the entities and domain itself become orphans and not reachable by any user

API Gateway using existing API

API Gateway act as intermediate proxy for domain delete request Once the domain delete is received in API gateway, it does the following steps :

• change the state of Domain to Delete
• Retrieve All things , channels, groups related to the domain
• Send delete request to retrieve entities (send individual or bulk delete request to respective entities )
• Delete domain from repo and its related policies All the above steps are done by API Gateway using SDK, no need for new API endpoints

Advantages:

• No cyclic calls between the service
• Effectively reusing of existing SDKs and endpoints

Disadvantages:

• Creating new service and managing them
• APIs are only in HTTP endpoints, there is no gRPC endpoints
• If delete process get into error, Then all the entities and domain itself become orphans and not reachable by any user
• Response to delete request takes long time if there are many entities in domain, Delete performance will be poor

Scheduler JOB to clean

In this approach , Delete Domain function in Domain service will just change the state of the domain to delete Scheduler will look for the list of Domains with Delete status in Database And picks the list of domains to delete and run individual jobs for each Domains

In Steps:

• Change status of Domain to delete by Domain Delete endpoint
• Scheduler picks all list of domains with delete
• Run Delete Job for individual domains until all below givens steps passes without errors
  • Retrieve All things , channels, groups related to the domain
  • Send delete request to retrieve entities (send individual or bulk delete request to respective entities )
  • Delete domain from repo and its related policies

Advantage:

• No circular dependency
• Delete will faster, since it is kind of soft delete and prevent all user actions on the domain once state change to delete.
• We need JOBs to maintain consistence in distributed system, this will help to maintain consistence or keep in sync, Particularly if we have local cache copy of policies
• Jobs will be keep on repeating until all the process in job get success,

  Disadvantage:

• There is not support for jobs in go-kit, need to find best way to do

[dborovcanin]

@arvindh123 Thanks for the report. I have a couple of questions:

1. What are the general disadvantages of the API gateway approach?
2. Does it make sense to update Things so that each Thing has its domain? Will that simplify Domain removal?

[arvindh123]

Disadvantages of API Gateway:

• Load balancing will be challenge in cluster environment, Work arounds are , having API Gateway in front of each service workers as side car or building API Gateway with Load Balancer feature
• Individual API Performance might get little slower , because of proxying and aggregation processing through API Gateway
• Need to build gRPC endpoints in all services to improve performance
• Maintenance of new service

My View : We have most of functionalities are working condition except delete flow.

If we have API Gateway, Delete process will take time, if there are 1000 of entities to delete, So API becomes slower and irresponsive

So for now, we can proceed with building gRPC endpoint for removal of Things, channels and groups And , Delete the domain with cleanup job.

Later over time, we can add other gRPC endpoints and after that we can start building API Gateway for other API aggregations For now time being, we can do aggregation in UI Service. I believe, UI service is already doing API Gateway kind of aggregation work.

[arvindh123]

2 Does it make sense to update Things so that each Thing has its domain? Will that simplify Domain removal?

Adding domain to entities will help partially (50%) in domain delete process. We can have a separate function in each entities repo to delete all entities related to one domain. But for Policy removal we need to send individual request to remove policy . Remove policy works like matching of Subject Object and Relation. This can be done faster, by sending parallel batch delete request to spiceDB.

As I said in previous answer having cleanup job will simply domain delete process for now and API Gateway might not be a right choice for Delete flow as explained in previous comment

[dborovcanin]

This one might also be addressed by API GW: https://github.com/absmach/magistrala-old/pull/269.

[dborovcanin]

Relevant document: https://docs.google.com/document/d/1tJ78iHx-_-W-GO5YFa8T1yd65Hu-AsxLmSDpwg5mBYw/edit#heading=h.nc9o9iq9wvl6.

[arvindh123]

List of API Gateway

Golang based API Gateway (Random order)

• TYK https://github.com/TykTechnologies/tyk
• Apinto (Goku ) https://github.com/eolinker/apinto
• Krakend CE https://github.com/krakend/krakend-ce
• Gloo https://github.com/solo-io/gloo

OpenResty based API Gateway

• Kong https://github.com/Kong/kong
• Apache APISIX https://github.com/apache/apisix

Note: We can build our own API Gateway according to our need with https://openresty.org/en/

Java

• Repose https://github.com/rackerlabs/repose
• Gravitee https://github.com/gravitee-io/gravitee-api-management
• https://github.com/wso2/product-apim

[dborovcanin]

This may be a part of https://www.github.com/absmach/mgate.