digitalocean / firebolt

Golang framework for streaming ETL, observability data pipeline, and event processing apps
Other
698 stars 50 forks source link

Support configurable bucket ranges for bulk_process_time Histogram #48

Closed ful09003 closed 1 year ago

ful09003 commented 1 year ago

Greetings,

First time contributor, please let me know if this is an acceptable PR!

To address #45, I am proposing a set of changes which amount to:

Using the defaults above, and assuming a 5000ms bulk-index-timeout-ms in unit testing, this PR results in the following buckets on my machine: 0.01, 0.026826957952797256, 0.07196856730011518, 0.19306977288832497, 0.5179474679231209, 1.3894954943731366, 3.7275937203149367, 9.999999999999991

ful09003 commented 1 year ago

Mmmm, in hindsight of course unit testing assumptions from one platform would not work (and we see that here, with the buckets generated in unit testing differing very slightly from the ones I had, locally). I'll think about how to address that and come back once ready!