prashant280920
commented
3 weeks ago To start the Kafka consumer, we need to know the Kafka topic initially when the plugin starts. Therefore, we must provide the topic name at the initial phase. The specific process we want the consumer to perform needs to be defined in the listener, as illustrated in the image below.
In the consumption message, we receive the tenantId from the service (which belongs to a specific tenant) that pushes the message to the topic.
The idea behind this plugin is to provide a template for others to write similar code by adding more listeners in the listener folder for their specific use cases.
For example, in the future, if someone needs a new consumer for a specific use case, instead of recording rolled-up usage, they can add their new topic and write their own listener.
To address your issue, if you want to create a queue specific to each tenant, we first need to retrieve the tenants and define the topic in a format such as stream.tenantId or stream.tenantApiKey, where tenantId or tenantApiKey are tenant-specific. Each tenant or the service belonging to that tenant can then push their messages to their own queue.
To implement this, we need to fetch all tenants from the tenant table and initialize a listener for each tenant in a separate thread. This means if there are 100 tenants, 100 queues will be created, each processed in its own thread.
However, since the logic is similar for each tenant, it is not good to have a separate queue for each tenant. Tenant-wise validation is managed by the Context and osgipluginAPI interfaces in the listener logic. This approach reduces redundancy and leverages the existing validation mechanisms.
Message Queue Isolation matters when we have different logic for consumption. Tenant wise isolation is anyways taken care by the osgiKillBillAPI interface so creating different kafka queue for each tenant doesn't make much importance.
Kill Bill is mult tenanted. Some tenants will not need usage billing and some will. It would make more sense for each tenant Biller to push usage records into their own queue. This will allow for better tenant isolation and easier operational administration.