uber / cadence

Cadence is a distributed, scalable, durable, and highly available orchestration engine to execute asynchronous long-running business logic in a scalable and resilient way.
https://cadenceworkflow.io
MIT License
7.97k stars 773 forks source link

Added a new metric to the workflow cache #6064

Closed jakobht closed 1 month ago

jakobht commented 1 month ago

What changed? In the per workflow Id cache we have added request counting for each workflow Id for the external rate limits. We keep track of the number of requests for the current second for each workflow ID and emit this as a timer metric.

The timer metric keeps track of different percentiles of the metrics, as well as a max count, so we do not need to do this bookkeeping manually.

Why? We use the tracked information above to emit a metric stating the request count per workflow ID in a domain. We have to do the bookkeeping manually. Emitting a metric for each workflow ID would have way to high cardinality, and would overwhelm the metrics system.

How did you test it? Unit tests

Potential risks It should just emit a metric, so should be low risk.

Release notes

Documentation Changes

codecov[bot] commented 1 month ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 68.14%. Comparing base (2f08de5) to head (73afab2). Report is 5 commits behind head on master.

:exclamation: Current head 73afab2 differs from pull request most recent head 557e21c

Please upload reports for the commit 557e21c to get more accurate results.

Additional details and impacted files | [Files](https://app.codecov.io/gh/uber/cadence/pull/6064?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=uber) | Coverage Δ | | |---|---|---| | [service/history/workflowcache/cache.go](https://app.codecov.io/gh/uber/cadence/pull/6064?src=pr&el=tree&filepath=service%2Fhistory%2Fworkflowcache%2Fcache.go&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=uber#diff-c2VydmljZS9oaXN0b3J5L3dvcmtmbG93Y2FjaGUvY2FjaGUuZ28=) | `91.95% <100.00%> (+0.18%)` | :arrow_up: | | [service/history/workflowcache/metrics.go](https://app.codecov.io/gh/uber/cadence/pull/6064?src=pr&el=tree&filepath=service%2Fhistory%2Fworkflowcache%2Fmetrics.go&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=uber#diff-c2VydmljZS9oaXN0b3J5L3dvcmtmbG93Y2FjaGUvbWV0cmljcy5nbw==) | `100.00% <100.00%> (ø)` | | ... and [13 files with indirect coverage changes](https://app.codecov.io/gh/uber/cadence/pull/6064/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=uber) ------ [Continue to review full report in Codecov by Sentry](https://app.codecov.io/gh/uber/cadence/pull/6064?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=uber). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=uber) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://app.codecov.io/gh/uber/cadence/pull/6064?dropdown=coverage&src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=uber). Last update [2f08de5...557e21c](https://app.codecov.io/gh/uber/cadence/pull/6064?dropdown=coverage&src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=uber). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=uber).
coveralls commented 1 month ago

Pull Request Test Coverage Report for Build 018fc948-db5d-46cf-88b8-780d605911b9

Details


Files with Coverage Reduction New Missed Lines %
common/task/weighted_round_robin_task_scheduler.go 1 88.06%
common/task/parallel_task_processor.go 2 93.06%
service/matching/tasklist/db.go 2 73.23%
common/dynamicconfig/constants.go 2 99.05%
service/matching/tasklist/task_list_manager.go 2 76.48%
common/task/fifo_task_scheduler.go 2 85.57%
common/persistence/sql/sqlplugin/mysql/task.go 2 73.68%
common/membership/hashring.go 2 84.69%
common/persistence/sql/sqlplugin/mysql/db.go 2 79.49%
service/history/task/task.go 3 84.81%
<!-- Total: 299 -->
Totals Coverage Status
Change from base Build 018fc614-f430-4db4-8939-669fc38230e0: -0.1%
Covered Lines: 102545
Relevant Lines: 147497

💛 - Coveralls