hatchet-dev / hatchet

A distributed, fault-tolerant task queue
https://hatchet.run
MIT License
3.83k stars 126 forks source link

feat: tenant partitioning #649

Closed abelanger5 closed 3 weeks ago

abelanger5 commented 3 weeks ago

Description

Adds partitioning logic across engine instances that perform list operations on tenants.

This isn't a true database partition - each engine version generates a partition id and rebalances tenants across partitions on startup. The logic relies on partitions being active (i.e. sitting on a hot loop) as partition heartbeats are updated when the partition is used. Inactive partitions are automatically removed and tenants on those partitions are rebalanced.

Type of change

vercel[bot] commented 3 weeks ago

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
hatchet-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jun 26, 2024 8:50pm