harshraj22 / Dis-Sim

Microservice for calculating image similarity. Uses Message Queues.
MIT License
9 stars 3 forks source link

A subscription model #34

Open harshraj22 opened 1 year ago

harshraj22 commented 1 year ago

Add a subscription model. An existing user can have one of the following subscription tiers:

The limit on the number of API calls by a user should be limited by the type of subscription they own.

harshraj22 commented 1 year ago

Can use reddis to implement it from scratch[1], [2] or use a third party library.

Read More: nordicapis

# docker run -p 6379:6379 --name redis_server redis:4-alpine
import redis

r = redis.StrictRedis(host='localhost', port=6379, decode_responses=True, password="")
print('Ping:', r.ping())
print(r)
print('setting:', r.set("key", "value"))
print('getting:', r.get("key"))

import time

username = 'user1'

WINDOW_LENGTH = 2 # in Seconds
for _ in range(3):
    cur_time = time.time()
    r.zadd(username, {cur_time: cur_time})
    time.sleep(1) # Sleep for 1 Second

p = r.pipeline()
p.zremrangebyscore(username, 0, time.time() - WINDOW_LENGTH)
cur_time = time.time()
p.zadd(username, {cur_time: cur_time})
p.expire(username, WINDOW_LENGTH)
p.execute()

print(r.zrange(username, 0, -1))
harshraj22 commented 1 year ago

A potential Architecutre:

image

Might have to deal with Distributed Commit 😨

The three Major components for this would be:

harshraj22 commented 1 year ago

Read about saga pattern and 2 phase commit

Another helpful stackoverflow thread.

harshraj22 commented 1 year ago

The subscription based Rate Limiting should be developed as a separate and completely independent identity, which is easy to plug in into any existing product. Is should not be tightly coupled with existing product.

A possible Architecture: Subscription Rate Limiter is a library that accept details to connect to existing Redis and MySql Server. It has methods to update the MySql and Redis server and Query them. It should be made in such a way that Replacing MySql Server with any other database is easy.

Similarly Data Monitoring Service can also be separated out into an independent project.

harshraj22 commented 1 year ago

Flow for Rate Limit On user request based on subscription tier they belong to: