DataDog / libdatadog

Datadog shared rust-based library. For now only used in other products (e.g. Ruby or PHP libraries).
Apache License 2.0
47 stars 8 forks source link

Design RFC for crashtracker #696

Closed danielsn closed 3 days ago

danielsn commented 1 week ago

What does this PR do?

Adds an RFC laying out the design considerations, risks, and mitigations for the crashtracker.

Motivation

We should clearly document this as the tool evolves.

Additional Notes

Anything else we should know when reviewing?

How to test the change?

This is documentation. No tests

codecov-commenter commented 1 week ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 71.73%. Comparing base (214f046) to head (eb35127).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #696 +/- ## ========================================== - Coverage 71.73% 71.73% -0.01% ========================================== Files 271 271 Lines 41143 41143 ========================================== - Hits 29514 29513 -1 - Misses 11629 11630 +1 ``` | [Components](https://app.codecov.io/gh/DataDog/libdatadog/pull/696/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DataDog) | Coverage Δ | | |---|---|---| | [crashtracker](https://app.codecov.io/gh/DataDog/libdatadog/pull/696/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DataDog) | `20.15% <ø> (+0.05%)` | :arrow_up: | | [crashtracker-ffi](https://app.codecov.io/gh/DataDog/libdatadog/pull/696/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DataDog) | `8.97% <ø> (ø)` | | | [datadog-alloc](https://app.codecov.io/gh/DataDog/libdatadog/pull/696/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DataDog) | `98.73% <ø> (ø)` | | | [data-pipeline](https://app.codecov.io/gh/DataDog/libdatadog/pull/696/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DataDog) | `91.86% <ø> (-0.07%)` | :arrow_down: | | [data-pipeline-ffi](https://app.codecov.io/gh/DataDog/libdatadog/pull/696/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DataDog) | `0.00% <ø> (ø)` | | | [ddcommon](https://app.codecov.io/gh/DataDog/libdatadog/pull/696/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DataDog) | `82.74% <ø> (ø)` | | | [ddcommon-ffi](https://app.codecov.io/gh/DataDog/libdatadog/pull/696/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DataDog) | `69.12% <ø> (ø)` | | | [ddtelemetry](https://app.codecov.io/gh/DataDog/libdatadog/pull/696/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DataDog) | `59.10% <ø> (ø)` | | | [ddtelemetry-ffi](https://app.codecov.io/gh/DataDog/libdatadog/pull/696/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DataDog) | `22.13% <ø> (ø)` | | | [dogstatsd](https://app.codecov.io/gh/DataDog/libdatadog/pull/696/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DataDog) | `89.22% <ø> (ø)` | | | [dogstatsd-client](https://app.codecov.io/gh/DataDog/libdatadog/pull/696/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DataDog) | `79.77% <ø> (ø)` | | | [ipc](https://app.codecov.io/gh/DataDog/libdatadog/pull/696/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DataDog) | `82.65% <ø> (ø)` | | | [profiling](https://app.codecov.io/gh/DataDog/libdatadog/pull/696/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DataDog) | `84.30% <ø> (ø)` | | | [profiling-ffi](https://app.codecov.io/gh/DataDog/libdatadog/pull/696/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DataDog) | `77.46% <ø> (ø)` | | | [serverless](https://app.codecov.io/gh/DataDog/libdatadog/pull/696/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DataDog) | `0.00% <ø> (ø)` | | | [sidecar](https://app.codecov.io/gh/DataDog/libdatadog/pull/696/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DataDog) | `36.13% <ø> (ø)` | | | [sidecar-ffi](https://app.codecov.io/gh/DataDog/libdatadog/pull/696/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DataDog) | `0.00% <ø> (ø)` | | | [spawn-worker](https://app.codecov.io/gh/DataDog/libdatadog/pull/696/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DataDog) | `50.36% <ø> (ø)` | | | [tinybytes](https://app.codecov.io/gh/DataDog/libdatadog/pull/696/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DataDog) | `94.77% <ø> (ø)` | | | [trace-mini-agent](https://app.codecov.io/gh/DataDog/libdatadog/pull/696/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DataDog) | `72.45% <ø> (ø)` | | | [trace-normalization](https://app.codecov.io/gh/DataDog/libdatadog/pull/696/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DataDog) | `98.25% <ø> (ø)` | | | [trace-obfuscation](https://app.codecov.io/gh/DataDog/libdatadog/pull/696/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DataDog) | `95.77% <ø> (ø)` | | | [trace-protobuf](https://app.codecov.io/gh/DataDog/libdatadog/pull/696/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DataDog) | `77.67% <ø> (ø)` | | | [trace-utils](https://app.codecov.io/gh/DataDog/libdatadog/pull/696/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DataDog) | `93.14% <ø> (ø)` | |
pr-commenter[bot] commented 1 week ago

Benchmarks

Comparison

Benchmark execution time: 2024-10-31 18:56:47

Comparing candidate commit eb35127 in PR branch dsn/crashtracker-design-rfc with baseline commit 214f046 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 51 metrics, 2 unstable metrics.

Candidate

Candidate benchmark details ### Group 1 |cpu_model |git_commit_sha|git_commit_date|git_branch | |----------------------------------------------|--------------|---------------|---------------------------| |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|eb35127 |1730400376 |dsn/crashtracker-design-rfc| |scenario |metric |min |mean ± sd |median ± mad |p75 |p95 |p99 |max |peak_to_median_ratio|skewness|kurtosis|cv |sem |runs|sample_size| |----------------------------------------------|--------------|---------|-------------------|-------------------|---------|---------|---------|---------|--------------------|--------|--------|-----|-------|----|-----------| |benching string interning on wordpress profile|execution_time|142.878µs|143.953µs ± 0.606µs|143.886µs ± 0.159µs|144.058µs|144.494µs|145.629µs|150.916µs|4.89% |7.860 |86.185 |0.42%|0.043µs|1 |200 | |scenario |metric |95% CI mean |Shapiro-Wilk pvalue|Ljung-Box pvalue (lag=1)|Dip test pvalue| |----------------------------------------------|--------------|--------------------------------------------|-------------------|------------------------|---------------| |benching string interning on wordpress profile|execution_time|[143.869µs; 144.037µs] or [-0.058%; +0.058%]|None |None |None | ### Group 2 |cpu_model |git_commit_sha|git_commit_date|git_branch | |----------------------------------------------|--------------|---------------|---------------------------| |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|eb35127 |1730400376 |dsn/crashtracker-design-rfc| |scenario |metric |min |mean ± sd |median ± mad |p75 |p95 |p99 |max |peak_to_median_ratio|skewness|kurtosis|cv |sem |runs|sample_size| |----------------------------------------|--------------|---------|--------------------|-------------------|---------|---------|---------|---------|--------------------|--------|--------|-----|-------|----|-----------| |normalization/normalize_trace/test_trace|execution_time|257.591ns|269.617ns ± 15.704ns|261.573ns ± 2.896ns|276.730ns|299.699ns|310.975ns|312.680ns|19.54% |1.313 |0.360 |5.81%|1.110ns|1 |200 | |scenario |metric |95% CI mean |Shapiro-Wilk pvalue|Ljung-Box pvalue (lag=1)|Dip test pvalue| |----------------------------------------|--------------|--------------------------------------------|-------------------|------------------------|---------------| |normalization/normalize_trace/test_trace|execution_time|[267.441ns; 271.793ns] or [-0.807%; +0.807%]|None |None |None | ### Group 3 |cpu_model |git_commit_sha|git_commit_date|git_branch | |----------------------------------------------|--------------|---------------|---------------------------| |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|eb35127 |1730400376 |dsn/crashtracker-design-rfc| |scenario |metric |min |mean ± sd |median ± mad |p75 |p95 |p99 |max |peak_to_median_ratio|skewness|kurtosis|cv |sem |runs|sample_size| |-------------------------------------------------------------------------------------------------------|--------------|----------------|---------------------------------|--------------------------------|----------------|----------------|----------------|----------------|--------------------|--------|--------|-----|-------------|----|-----------| |normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...|execution_time|187.391µs |191.931µs ± 1.326µs |192.129µs ± 0.957µs |192.900µs |193.896µs |194.246µs |194.660µs |1.32% |-0.435 |-0.112 |0.69%|0.094µs |1 |200 | |normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...|throughput |5137151.120op/s |5210445.458op/s ± 36122.501op/s |5204837.447op/s ± 25919.411op/s |5235116.146op/s |5269703.313op/s |5299856.883op/s |5336431.085op/s |2.53% |0.471 |-0.045 |0.69%|2554.247op/s |1 |200 | |normalization/normalize_name/normalize_name/bad-name |execution_time|18.177µs |18.208µs ± 0.029µs |18.213µs ± 0.024µs |18.226µs |18.249µs |18.303µs |18.378µs |0.91% |1.654 |5.982 |0.16%|0.002µs |1 |200 | |normalization/normalize_name/normalize_name/bad-name |throughput |54413035.519op/s|54919979.937op/s ± 87193.722op/s |54905836.632op/s ± 72113.963op/s|54994919.378op/s|55009022.529op/s|55013534.192op/s|55014501.916op/s|0.20% |-1.629 |5.812 |0.16%|6165.527op/s |1 |200 | |normalization/normalize_name/normalize_name/good |execution_time|11.228µs |11.256µs ± 0.021µs |11.249µs ± 0.011µs |11.276µs |11.295µs |11.309µs |11.325µs |0.68% |0.852 |-0.138 |0.18%|0.001µs |1 |200 | |normalization/normalize_name/normalize_name/good |throughput |88299367.970op/s|88838903.804op/s ± 162955.592op/s|88899847.030op/s ± 85720.977op/s|88957444.810op/s|89023976.592op/s|89056342.645op/s|89063924.240op/s|0.18% |-0.846 |-0.157 |0.18%|11522.700op/s|1 |200 | |scenario |metric |95% CI mean |Shapiro-Wilk pvalue|Ljung-Box pvalue (lag=1)|Dip test pvalue| |-------------------------------------------------------------------------------------------------------|--------------|----------------------------------------------------------|-------------------|------------------------|---------------| |normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...|execution_time|[191.747µs; 192.115µs] or [-0.096%; +0.096%] |None |None |None | |normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...|throughput |[5205439.227op/s; 5215451.689op/s] or [-0.096%; +0.096%] |None |None |None | |normalization/normalize_name/normalize_name/bad-name |execution_time|[18.204µs; 18.212µs] or [-0.022%; +0.022%] |None |None |None | |normalization/normalize_name/normalize_name/bad-name |throughput |[54907895.726op/s; 54932064.148op/s] or [-0.022%; +0.022%]|None |None |None | |normalization/normalize_name/normalize_name/good |execution_time|[11.254µs; 11.259µs] or [-0.025%; +0.025%] |None |None |None | |normalization/normalize_name/normalize_name/good |throughput |[88816319.726op/s; 88861487.881op/s] or [-0.025%; +0.025%]|None |None |None | ### Group 4 |cpu_model |git_commit_sha|git_commit_date|git_branch | |----------------------------------------------|--------------|---------------|---------------------------| |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|eb35127 |1730400376 |dsn/crashtracker-design-rfc| |scenario |metric |min |mean ± sd |median ± mad |p75 |p95 |p99 |max |peak_to_median_ratio|skewness|kurtosis|cv |sem |runs|sample_size| |-------------------------------------------------------------------------------------------------------|--------------|----------------|---------------------------------|--------------------------------|----------------|----------------|----------------|----------------|--------------------|--------|--------|-----|------------|----|-----------| |normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...|execution_time|622.363µs |624.274µs ± 0.550µs |624.332µs ± 0.391µs |624.691µs |625.030µs |625.402µs |625.588µs |0.20% |-0.517 |0.260 |0.09%|0.039µs |1 |200 | |normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...|throughput |1598495.040op/s |1601861.347op/s ± 1411.390op/s |1601711.910op/s ± 1003.929op/s |1602752.934op/s |1604355.753op/s |1605552.256op/s |1606779.508op/s |0.32% |0.522 |0.268 |0.09%|99.800op/s |1 |200 | |normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて |execution_time|461.280µs |462.189µs ± 0.475µs |462.120µs ± 0.314µs |462.486µs |463.021µs |463.451µs |463.551µs |0.31% |0.531 |-0.038 |0.10%|0.034µs |1 |200 | |normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて |throughput |2157260.425op/s |2163619.622op/s ± 2224.268op/s |2163940.966op/s ± 1468.382op/s |2165168.944op/s |2166812.802op/s |2167696.697op/s |2167878.948op/s |0.18% |-0.525 |-0.045 |0.10%|157.279op/s |1 |200 | |normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters |execution_time|178.179µs |178.371µs ± 0.107µs |178.369µs ± 0.045µs |178.401µs |178.577µs |178.767µs |178.851µs |0.27% |1.476 |4.126 |0.06%|0.008µs |1 |200 | |normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters |throughput |5591241.007op/s |5606287.162op/s ± 3354.719op/s |5606361.497op/s ± 1417.499op/s |5608410.838op/s |5610837.664op/s |5611716.389op/s |5612347.493op/s |0.11% |-1.469 |4.097 |0.06%|237.214op/s |1 |200 | |normalization/normalize_service/normalize_service/[empty string] |execution_time|47.140µs |47.314µs ± 0.075µs |47.306µs ± 0.046µs |47.359µs |47.434µs |47.484µs |47.630µs |0.69% |0.610 |1.083 |0.16%|0.005µs |1 |200 | |normalization/normalize_service/normalize_service/[empty string] |throughput |20995002.296op/s|21135535.039op/s ± 33467.443op/s |21139078.497op/s ± 20569.331op/s|21156756.147op/s|21187168.120op/s|21195735.718op/s|21213339.851op/s|0.35% |-0.597 |1.047 |0.16%|2366.506op/s|1 |200 | |normalization/normalize_service/normalize_service/test_ASCII |execution_time|48.933µs |49.450µs ± 0.319µs |49.330µs ± 0.165µs |49.691µs |50.066µs |50.197µs |50.246µs |1.86% |0.769 |-0.603 |0.64%|0.023µs |1 |200 | |normalization/normalize_service/normalize_service/test_ASCII |throughput |19902032.803op/s|20223153.170op/s ± 129897.864op/s|20271803.891op/s ± 67901.191op/s|20326635.326op/s|20366885.123op/s|20391365.206op/s|20436040.818op/s|0.81% |-0.753 |-0.630 |0.64%|9185.166op/s|1 |200 | |scenario |metric |95% CI mean |Shapiro-Wilk pvalue|Ljung-Box pvalue (lag=1)|Dip test pvalue| |-------------------------------------------------------------------------------------------------------|--------------|----------------------------------------------------------|-------------------|------------------------|---------------| |normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...|execution_time|[624.198µs; 624.350µs] or [-0.012%; +0.012%] |None |None |None | |normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...|throughput |[1601665.742op/s; 1602056.952op/s] or [-0.012%; +0.012%] |None |None |None | |normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて |execution_time|[462.123µs; 462.255µs] or [-0.014%; +0.014%] |None |None |None | |normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて |throughput |[2163311.360op/s; 2163927.884op/s] or [-0.014%; +0.014%] |None |None |None | |normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters |execution_time|[178.356µs; 178.386µs] or [-0.008%; +0.008%] |None |None |None | |normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters |throughput |[5605822.230op/s; 5606752.094op/s] or [-0.008%; +0.008%] |None |None |None | |normalization/normalize_service/normalize_service/[empty string] |execution_time|[47.303µs; 47.324µs] or [-0.022%; +0.022%] |None |None |None | |normalization/normalize_service/normalize_service/[empty string] |throughput |[21130896.773op/s; 21140173.305op/s] or [-0.022%; +0.022%]|None |None |None | |normalization/normalize_service/normalize_service/test_ASCII |execution_time|[49.406µs; 49.495µs] or [-0.089%; +0.089%] |None |None |None | |normalization/normalize_service/normalize_service/test_ASCII |throughput |[20205150.576op/s; 20241155.765op/s] or [-0.089%; +0.089%]|None |None |None | ### Group 5 |cpu_model |git_commit_sha|git_commit_date|git_branch | |----------------------------------------------|--------------|---------------|---------------------------| |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|eb35127 |1730400376 |dsn/crashtracker-design-rfc| |scenario |metric |min |mean ± sd |median ± mad |p75 |p95 |p99 |max |peak_to_median_ratio|skewness|kurtosis|cv |sem |runs|sample_size| |-----------------------|--------------|-------|-----------------|-----------------|-------|-------|-------|-------|--------------------|--------|--------|-----|-------|----|-----------| |tags/replace_trace_tags|execution_time|2.657µs|2.681µs ± 0.016µs|2.677µs ± 0.005µs|2.682µs|2.731µs|2.735µs|2.736µs|2.19% |2.298 |4.974 |0.60%|0.001µs|1 |200 | |scenario |metric |95% CI mean |Shapiro-Wilk pvalue|Ljung-Box pvalue (lag=1)|Dip test pvalue| |-----------------------|--------------|----------------------------------------|-------------------|------------------------|---------------| |tags/replace_trace_tags|execution_time|[2.678µs; 2.683µs] or [-0.083%; +0.083%]|None |None |None | ### Group 6 |cpu_model |git_commit_sha|git_commit_date|git_branch | |----------------------------------------------|--------------|---------------|---------------------------| |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|eb35127 |1730400376 |dsn/crashtracker-design-rfc| |scenario |metric |min |mean ± sd |median ± mad |p75 |p95 |p99 |max |peak_to_median_ratio|skewness|kurtosis|cv |sem |runs|sample_size| |--------------------------------------|--------------|-------|-----------------|-----------------|-------|-------|-------|-------|--------------------|--------|--------|-----|-------|----|-----------| |concentrator/add_spans_to_concentrator|execution_time|9.072ms|9.105ms ± 0.012ms|9.105ms ± 0.008ms|9.112ms|9.125ms|9.131ms|9.145ms|0.44% |0.114 |0.402 |0.13%|0.001ms|1 |200 | |scenario |metric |95% CI mean |Shapiro-Wilk pvalue|Ljung-Box pvalue (lag=1)|Dip test pvalue| |--------------------------------------|--------------|----------------------------------------|-------------------|------------------------|---------------| |concentrator/add_spans_to_concentrator|execution_time|[9.103ms; 9.106ms] or [-0.019%; +0.019%]|None |None |None | ### Group 7 |cpu_model |git_commit_sha|git_commit_date|git_branch | |----------------------------------------------|--------------|---------------|---------------------------| |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|eb35127 |1730400376 |dsn/crashtracker-design-rfc| |scenario |metric |min |mean ± sd |median ± mad |p75 |p95 |p99 |max |peak_to_median_ratio|skewness|kurtosis|cv |sem |runs|sample_size| |----------------------------------------------------------------------------|--------------|-------|-----------------|-----------------|-------|-------|-------|-------|--------------------|--------|--------|-----|-------|----|-----------| |benching deserializing traces from msgpack to their internal representation |execution_time|1.120µs|1.179µs ± 0.023µs|1.183µs ± 0.015µs|1.197µs|1.201µs|1.204µs|1.205µs|1.87% |-1.079 |0.417 |1.92%|0.002µs|1 |200 | |scenario |metric |95% CI mean |Shapiro-Wilk pvalue|Ljung-Box pvalue (lag=1)|Dip test pvalue| |----------------------------------------------------------------------------|--------------|----------------------------------------|-------------------|------------------------|---------------| |benching deserializing traces from msgpack to their internal representation |execution_time|[1.175µs; 1.182µs] or [-0.267%; +0.267%]|None |None |None | ### Group 8 |cpu_model |git_commit_sha|git_commit_date|git_branch | |----------------------------------------------|--------------|---------------|---------------------------| |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|eb35127 |1730400376 |dsn/crashtracker-design-rfc| |scenario |metric |min |mean ± sd |median ± mad |p75 |p95 |p99 |max |peak_to_median_ratio|skewness|kurtosis|cv |sem |runs|sample_size| |----------------------------|--------------|--------|------------------|------------------|--------|--------|--------|--------|--------------------|--------|--------|-----|-------|----|-----------| |redis/obfuscate_redis_string|execution_time|37.957µs|38.641µs ± 1.159µs|38.108µs ± 0.104µs|38.306µs|41.120µs|41.206µs|41.867µs|9.86% |1.684 |0.918 |2.99%|0.082µs|1 |200 | |scenario |metric |95% CI mean |Shapiro-Wilk pvalue|Ljung-Box pvalue (lag=1)|Dip test pvalue| |----------------------------|--------------|------------------------------------------|-------------------|------------------------|---------------| |redis/obfuscate_redis_string|execution_time|[38.480µs; 38.801µs] or [-0.416%; +0.416%]|None |None |None | ### Group 9 |cpu_model |git_commit_sha|git_commit_date|git_branch | |----------------------------------------------|--------------|---------------|---------------------------| |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|eb35127 |1730400376 |dsn/crashtracker-design-rfc| |scenario |metric |min |mean ± sd |median ± mad |p75 |p95 |p99 |max |peak_to_median_ratio|skewness|kurtosis|cv |sem |runs|sample_size| |----------------------------------------------------------|--------------|-----------------|----------------------------------|---------------------------------|-----------------|-----------------|-----------------|-----------------|--------------------|--------|--------|-----|-------------|----|-----------| |credit_card/is_card_number/ |execution_time|4.606µs |4.615µs ± 0.002µs |4.614µs ± 0.001µs |4.616µs |4.618µs |4.619µs |4.624µs |0.21% |0.392 |1.814 |0.05%|0.000µs |1 |200 | |credit_card/is_card_number/ |throughput |216261465.001op/s|216707850.904op/s ± 103398.903op/s|216718507.762op/s ± 62413.120op/s|216777047.982op/s|216850578.875op/s|216888369.720op/s|217114580.291op/s|0.18% |-0.387 |1.810 |0.05%|7311.407op/s |1 |200 | |credit_card/is_card_number/ 3782-8224-6310-005 |execution_time|90.336µs |91.318µs ± 0.586µs |91.220µs ± 0.267µs |91.539µs |92.169µs |93.074µs |96.057µs |5.30% |3.091 |20.539 |0.64%|0.041µs |1 |200 | |credit_card/is_card_number/ 3782-8224-6310-005 |throughput |10410538.645op/s |10951140.555op/s ± 68970.669op/s |10962496.220op/s ± 32177.965op/s |10990067.454op/s |11029958.141op/s |11059552.019op/s |11069806.198op/s |0.98% |-2.854 |18.090 |0.63%|4876.963op/s |1 |200 | |credit_card/is_card_number/ 378282246310005 |execution_time|82.892µs |83.836µs ± 0.579µs |83.758µs ± 0.324µs |84.091µs |84.692µs |85.226µs |87.898µs |4.94% |2.120 |11.217 |0.69%|0.041µs |1 |200 | |credit_card/is_card_number/ 378282246310005 |throughput |11376823.136op/s |11928572.084op/s ± 81222.188op/s |11939205.574op/s ± 46010.028op/s |11981945.914op/s |12031924.501op/s |12047916.524op/s |12063909.768op/s |1.04% |-1.949 |9.757 |0.68%|5743.276op/s |1 |200 | |credit_card/is_card_number/37828224631 |execution_time|4.599µs |4.615µs ± 0.003µs |4.614µs ± 0.002µs |4.616µs |4.618µs |4.620µs |4.631µs |0.35% |0.239 |10.822 |0.06%|0.000µs |1 |200 | |credit_card/is_card_number/37828224631 |throughput |215945959.897op/s|216704734.020op/s ± 126013.180op/s|216708581.664op/s ± 75867.785op/s|216784979.713op/s|216857388.588op/s|216898758.875op/s|217448653.971op/s|0.34% |-0.217 |10.818 |0.06%|8910.477op/s |1 |200 | |credit_card/is_card_number/378282246310005 |execution_time|79.498µs |80.503µs ± 0.401µs |80.508µs ± 0.265µs |80.770µs |81.174µs |81.422µs |81.497µs |1.23% |0.061 |-0.207 |0.50%|0.028µs |1 |200 | |credit_card/is_card_number/378282246310005 |throughput |12270331.301op/s |12422130.020op/s ± 61829.174op/s |12421175.268op/s ± 40873.384op/s |12463782.211op/s |12518222.032op/s |12573078.414op/s |12578903.689op/s |1.27% |-0.034 |-0.207 |0.50%|4371.983op/s |1 |200 | |credit_card/is_card_number/37828224631000521389798 |execution_time|59.264µs |59.526µs ± 0.203µs |59.522µs ± 0.167µs |59.640µs |59.870µs |60.060µs |60.092µs |0.96% |0.512 |-0.485 |0.34%|0.014µs |1 |200 | |credit_card/is_card_number/37828224631000521389798 |throughput |16641155.152op/s |16799711.145op/s ± 57165.501op/s |16800634.341op/s ± 47351.850op/s |16856908.281op/s |16870704.191op/s |16872475.981op/s |16873515.330op/s |0.43% |-0.499 |-0.511 |0.34%|4042.211op/s |1 |200 | |credit_card/is_card_number/x371413321323331 |execution_time|7.218µs |7.223µs ± 0.006µs |7.223µs ± 0.002µs |7.224µs |7.226µs |7.228µs |7.296µs |1.01% |11.082 |141.794 |0.08%|0.000µs |1 |200 | |credit_card/is_card_number/x371413321323331 |throughput |137065784.748op/s|138448087.279op/s ± 106395.816op/s|138450130.269op/s ± 29625.165op/s|138489270.544op/s|138518501.917op/s|138524924.520op/s|138536213.366op/s|0.06% |-11.033 |140.952 |0.08%|7523.320op/s |1 |200 | |credit_card/is_card_number_no_luhn/ |execution_time|4.602µs |4.614µs ± 0.002µs |4.614µs ± 0.001µs |4.616µs |4.618µs |4.620µs |4.628µs |0.30% |0.485 |8.429 |0.05%|0.000µs |1 |200 | |credit_card/is_card_number_no_luhn/ |throughput |216063778.598op/s|216712404.188op/s ± 111197.925op/s|216714975.005op/s ± 66621.588op/s|216781780.927op/s|216854180.070op/s|216873771.194op/s|217308855.953op/s|0.27% |-0.470 |8.413 |0.05%|7862.881op/s |1 |200 | |credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 |execution_time|73.956µs |74.422µs ± 0.171µs |74.437µs ± 0.083µs |74.508µs |74.660µs |74.739µs |75.560µs |1.51% |1.178 |8.884 |0.23%|0.012µs |1 |200 | |credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 |throughput |13234518.212op/s |13436932.236op/s ± 30814.861op/s |13434121.087op/s ± 15034.139op/s |13454077.199op/s |13483526.976op/s |13503869.719op/s |13521635.970op/s |0.65% |-1.114 |8.448 |0.23%|2178.940op/s |1 |200 | |credit_card/is_card_number_no_luhn/ 378282246310005 |execution_time|66.327µs |66.838µs ± 0.249µs |66.809µs ± 0.174µs |67.010µs |67.280µs |67.440µs |67.637µs |1.24% |0.453 |-0.243 |0.37%|0.018µs |1 |200 | |credit_card/is_card_number_no_luhn/ 378282246310005 |throughput |14784718.239op/s |14961825.897op/s ± 55745.107op/s |14968031.315op/s ± 39160.882op/s |15004219.203op/s |15040398.220op/s |15045180.919op/s |15076910.335op/s |0.73% |-0.436 |-0.272 |0.37%|3941.774op/s |1 |200 | |credit_card/is_card_number_no_luhn/37828224631 |execution_time|4.578µs |4.614µs ± 0.003µs |4.615µs ± 0.001µs |4.616µs |4.618µs |4.619µs |4.619µs |0.11% |-7.744 |88.433 |0.07%|0.000µs |1 |200 | |credit_card/is_card_number_no_luhn/37828224631 |throughput |216478261.317op/s|216712068.717op/s ± 149281.700op/s|216706046.108op/s ± 59925.998op/s|216768889.153op/s|216840753.502op/s|216871094.166op/s|218446743.853op/s|0.80% |7.807 |89.373 |0.07%|10555.810op/s|1 |200 | |credit_card/is_card_number_no_luhn/378282246310005 |execution_time|62.350µs |62.652µs ± 0.172µs |62.615µs ± 0.107µs |62.751µs |62.987µs |63.136µs |63.219µs |0.97% |0.896 |0.299 |0.27%|0.012µs |1 |200 | |credit_card/is_card_number_no_luhn/378282246310005 |throughput |15817955.891op/s |15961397.176op/s ± 43721.111op/s |15970645.955op/s ± 27256.494op/s |15995560.832op/s |16008317.763op/s |16034342.713op/s |16038439.328op/s |0.42% |-0.884 |0.270 |0.27%|3091.549op/s |1 |200 | |credit_card/is_card_number_no_luhn/37828224631000521389798|execution_time|59.261µs |59.578µs ± 0.230µs |59.589µs ± 0.179µs |59.743µs |59.958µs |60.100µs |60.106µs |0.87% |0.346 |-0.778 |0.39%|0.016µs |1 |200 | |credit_card/is_card_number_no_luhn/37828224631000521389798|throughput |16637249.581op/s |16785050.559op/s ± 64709.375op/s |16781744.084op/s ± 50306.088op/s |16850095.102op/s |16871160.186op/s |16872798.568op/s |16874551.489op/s |0.55% |-0.333 |-0.794 |0.38%|4575.644op/s |1 |200 | |credit_card/is_card_number_no_luhn/x371413321323331 |execution_time|7.218µs |7.222µs ± 0.003µs |7.222µs ± 0.002µs |7.224µs |7.227µs |7.229µs |7.238µs |0.22% |1.380 |3.902 |0.04%|0.000µs |1 |200 | |credit_card/is_card_number_no_luhn/x371413321323331 |throughput |138161876.440op/s|138457882.966op/s ± 54428.719op/s |138469889.775op/s ± 36910.909op/s|138498898.934op/s|138522865.277op/s|138534142.017op/s|138537917.828op/s|0.05% |-1.375 |3.873 |0.04%|3848.692op/s |1 |200 | |scenario |metric |95% CI mean |Shapiro-Wilk pvalue|Ljung-Box pvalue (lag=1)|Dip test pvalue| |----------------------------------------------------------|--------------|------------------------------------------------------------|-------------------|------------------------|---------------| |credit_card/is_card_number/ |execution_time|[4.614µs; 4.615µs] or [-0.007%; +0.007%] |None |None |None | |credit_card/is_card_number/ |throughput |[216693520.810op/s; 216722180.997op/s] or [-0.007%; +0.007%]|None |None |None | |credit_card/is_card_number/ 3782-8224-6310-005 |execution_time|[91.237µs; 91.400µs] or [-0.089%; +0.089%] |None |None |None | |credit_card/is_card_number/ 3782-8224-6310-005 |throughput |[10941581.884op/s; 10960699.226op/s] or [-0.087%; +0.087%] |None |None |None | |credit_card/is_card_number/ 378282246310005 |execution_time|[83.756µs; 83.916µs] or [-0.096%; +0.096%] |None |None |None | |credit_card/is_card_number/ 378282246310005 |throughput |[11917315.469op/s; 11939828.698op/s] or [-0.094%; +0.094%] |None |None |None | |credit_card/is_card_number/37828224631 |execution_time|[4.614µs; 4.615µs] or [-0.008%; +0.008%] |None |None |None | |credit_card/is_card_number/37828224631 |throughput |[216687269.805op/s; 216722198.235op/s] or [-0.008%; +0.008%]|None |None |None | |credit_card/is_card_number/378282246310005 |execution_time|[80.448µs; 80.559µs] or [-0.069%; +0.069%] |None |None |None | |credit_card/is_card_number/378282246310005 |throughput |[12413561.091op/s; 12430698.949op/s] or [-0.069%; +0.069%] |None |None |None | |credit_card/is_card_number/37828224631000521389798 |execution_time|[59.497µs; 59.554µs] or [-0.047%; +0.047%] |None |None |None | |credit_card/is_card_number/37828224631000521389798 |throughput |[16791788.557op/s; 16807633.734op/s] or [-0.047%; +0.047%] |None |None |None | |credit_card/is_card_number/x371413321323331 |execution_time|[7.222µs; 7.224µs] or [-0.011%; +0.011%] |None |None |None | |credit_card/is_card_number/x371413321323331 |throughput |[138433341.842op/s; 138462832.716op/s] or [-0.011%; +0.011%]|None |None |None | |credit_card/is_card_number_no_luhn/ |execution_time|[4.614µs; 4.615µs] or [-0.007%; +0.007%] |None |None |None | |credit_card/is_card_number_no_luhn/ |throughput |[216696993.225op/s; 216727815.151op/s] or [-0.007%; +0.007%]|None |None |None | |credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 |execution_time|[74.398µs; 74.446µs] or [-0.032%; +0.032%] |None |None |None | |credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 |throughput |[13432661.593op/s; 13441202.879op/s] or [-0.032%; +0.032%] |None |None |None | |credit_card/is_card_number_no_luhn/ 378282246310005 |execution_time|[66.803µs; 66.872µs] or [-0.052%; +0.052%] |None |None |None | |credit_card/is_card_number_no_luhn/ 378282246310005 |throughput |[14954100.161op/s; 14969551.633op/s] or [-0.052%; +0.052%] |None |None |None | |credit_card/is_card_number_no_luhn/37828224631 |execution_time|[4.614µs; 4.615µs] or [-0.009%; +0.009%] |None |None |None | |credit_card/is_card_number_no_luhn/37828224631 |throughput |[216691379.709op/s; 216732757.725op/s] or [-0.010%; +0.010%]|None |None |None | |credit_card/is_card_number_no_luhn/378282246310005 |execution_time|[62.628µs; 62.675µs] or [-0.038%; +0.038%] |None |None |None | |credit_card/is_card_number_no_luhn/378282246310005 |throughput |[15955337.850op/s; 15967456.501op/s] or [-0.038%; +0.038%] |None |None |None | |credit_card/is_card_number_no_luhn/37828224631000521389798|execution_time|[59.546µs; 59.610µs] or [-0.053%; +0.053%] |None |None |None | |credit_card/is_card_number_no_luhn/37828224631000521389798|throughput |[16776082.462op/s; 16794018.656op/s] or [-0.053%; +0.053%] |None |None |None | |credit_card/is_card_number_no_luhn/x371413321323331 |execution_time|[7.222µs; 7.223µs] or [-0.005%; +0.005%] |None |None |None | |credit_card/is_card_number_no_luhn/x371413321323331 |throughput |[138450339.669op/s; 138465426.263op/s] or [-0.005%; +0.005%]|None |None |None | ### Group 10 |cpu_model |git_commit_sha|git_commit_date|git_branch | |----------------------------------------------|--------------|---------------|---------------------------| |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|eb35127 |1730400376 |dsn/crashtracker-design-rfc| |scenario |metric |min |mean ± sd |median ± mad |p75 |p95 |p99 |max |peak_to_median_ratio|skewness|kurtosis|cv |sem |runs|sample_size| |--------------------|--------------|-------|-----------------|-----------------|-------|-------|--------|--------|--------------------|--------|--------|------|-------|----|-----------| |write only interface|execution_time|1.341µs|3.192µs ± 1.389µs|3.044µs ± 0.024µs|3.067µs|3.114µs|13.830µs|14.881µs|388.92% |7.585 |57.745 |43.41%|0.098µs|1 |200 | |scenario |metric |95% CI mean |Shapiro-Wilk pvalue|Ljung-Box pvalue (lag=1)|Dip test pvalue| |--------------------|--------------|----------------------------------------|-------------------|------------------------|---------------| |write only interface|execution_time|[2.999µs; 3.385µs] or [-6.032%; +6.032%]|None |None |None | ### Group 11 |cpu_model |git_commit_sha|git_commit_date|git_branch | |----------------------------------------------|--------------|---------------|---------------------------| |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|eb35127 |1730400376 |dsn/crashtracker-design-rfc| |scenario |metric |min |mean ± sd |median ± mad |p75 |p95 |p99 |max |peak_to_median_ratio|skewness|kurtosis|cv |sem |runs|sample_size| |-----------------|--------------|--------|-------------------|------------------|--------|--------|--------|--------|--------------------|--------|--------|------|-------|----|-----------| |two way interface|execution_time|18.201µs|24.258µs ± 10.502µs|19.455µs ± 0.247µs|20.121µs|39.181µs|41.501µs|98.196µs|404.74% |3.451 |18.088 |43.18%|0.743µs|1 |200 | |scenario |metric |95% CI mean |Shapiro-Wilk pvalue|Ljung-Box pvalue (lag=1)|Dip test pvalue| |-----------------|--------------|------------------------------------------|-------------------|------------------------|---------------| |two way interface|execution_time|[22.802µs; 25.713µs] or [-6.000%; +6.000%]|None |None |None | ### Group 12 |cpu_model |git_commit_sha|git_commit_date|git_branch | |----------------------------------------------|--------------|---------------|---------------------------| |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|eb35127 |1730400376 |dsn/crashtracker-design-rfc| |scenario |metric |min |mean ± sd |median ± mad |p75 |p95 |p99 |max |peak_to_median_ratio|skewness|kurtosis|cv |sem |runs|sample_size| |------------------------|--------------|--------|------------------|------------------|--------|--------|--------|--------|--------------------|--------|--------|-----|-------|----|-----------| |sql/obfuscate_sql_string|execution_time|72.084µs|72.202µs ± 0.132µs|72.179µs ± 0.026µs|72.206µs|72.335µs|72.535µs|73.662µs|2.05% |7.674 |76.353 |0.18%|0.009µs|1 |200 | |scenario |metric |95% CI mean |Shapiro-Wilk pvalue|Ljung-Box pvalue (lag=1)|Dip test pvalue| |------------------------|--------------|------------------------------------------|-------------------|------------------------|---------------| |sql/obfuscate_sql_string|execution_time|[72.183µs; 72.220µs] or [-0.025%; +0.025%]|None |None |None |

Baseline

Omitted due to size.