search

prashant280920 / killbill-kafka-consumer-plugin

Open source Contribution for Kill Bill to provide a plugin which can support usage push through kafka stream.
2 stars 0 forks source link

Usage Unit Validation - Tenant Catalogs #2

Open shaun-forgie opened 1 month ago

shaun-forgie commented 1 month ago

It should not be possible to load tenant usage records into Kill Bill for usage unit types that are not defined in current or previous tenant catalogs.

This usage message validation will need to be tenant specific and aware of catalog values.

prashant280920 commented 1 month ago

Is it ok while consume usage record we also do the validation that

  1. Get catalog
  2. From catalog get all unit
  3. Verify that the provided usage record units are present in the catalogs defined unit ?
shaun-forgie commented 1 month ago

It is more efficient to parse catalogs and cache usage unit types for a given tenant and then to process all usage messages in their queue. You would not want to do the parsing and caching of these units every time you processed a single message. The obvious optimisation would be pulling and processing short batches of messages in progressive cycles.

prashant280920 commented 3 weeks ago

Since in a queue multiple tenants can push message. So while pulling and processing a short batches of message. In batch itself can have messages of different tenant. And each tenant have different catalog so for each individual message we have to do getCatalog() to extract catalog units.

shaun-forgie commented 3 weeks ago

Reading a tenant catalog to parse the usage unit type values on every message will be an expensive operation. You may want to cache these values in a map with TTL value to optimise this validation step.

pkcashfree commented 1 week ago

I have moved the code base to our cashfree org open source github account and there I have added this validation in the PR itself - Pull Request #1. Please use CashfreeOSS as the new repo