private-attribution / ipa

A raw implementation of Interoperable Private Attribution
MIT License
40 stars 23 forks source link

Carve out the attribution circuit from aggregation routine #1114

Closed akoshelev closed 3 months ago

akoshelev commented 3 months ago

It should help to better understand the bottlenecks, so more tracing was added to each routine to see the latency breakdown

I also had to fix some clippy issues after refactoring, so new type alias is there for this exact reason

Testing

2024-06-04T00:51:33.032403Z  INFO ipa_core::cli::verbosity: Logging setup at level info
2024-06-04T00:51:33.424129Z  INFO ipa_core::net::server: server listening on http://0.0.0.0:3002
2024-06-04T00:53:05.842863Z  INFO stall_detector{role=H3}: ipa_core::helpers::gateway::stall_detection::gateway: new
2024-06-04T00:53:05.846146Z  INFO oprf_ipa_query{sz=50000}: ipa_core::query::runner::oprf_ipa: new
2024-06-04T00:53:05.846201Z  INFO oprf_ipa_query{sz=50000}: ipa_core::query::runner::oprf_ipa: New query: IpaQueryConfig { per_user_credit_cap: 8, max_breakdown_key: 20, attribution_window_seconds: None, num_multi_bits: 3, plaintext_match_keys: true }
2024-06-04T00:53:05.848522Z  INFO oprf_ipa_query{sz=50000}:shuffle_inputs: ipa_core::protocol::ipa_prf::shuffle: new
2024-06-04T00:53:06.577035Z  INFO oprf_ipa_query{sz=50000}:shuffle_inputs: ipa_core::protocol::ipa_prf::shuffle: close time.busy=434ms time.idle=294ms
2024-06-04T00:53:06.577098Z  INFO oprf_ipa_query{sz=50000}:compute_prf_for_inputs: ipa_core::protocol::ipa_prf: new
2024-06-04T00:53:16.703835Z  INFO oprf_ipa_query{sz=50000}:compute_prf_for_inputs: ipa_core::protocol::ipa_prf: close time.busy=4.38s time.idle=5.75s
2024-06-04T00:53:16.706598Z  INFO oprf_ipa_query{sz=50000}:histograms_ranges_sortkeys: ipa_core::protocol::ipa_prf::prf_sharding: new
2024-06-04T00:53:16.717030Z  INFO oprf_ipa_query{sz=50000}:histograms_ranges_sortkeys: ipa_core::protocol::ipa_prf::prf_sharding: close time.busy=10.4ms time.idle=22.7µs
2024-06-04T00:53:16.854015Z  INFO oprf_ipa_query{sz=50000}:attribute_cap_aggregate: ipa_core::protocol::ipa_prf::prf_sharding: new
2024-06-04T00:53:16.855536Z  INFO oprf_ipa_query{sz=50000}:attribute_cap_aggregate:attribute_cap{unique_match_keys=8161}: ipa_core::protocol::ipa_prf::prf_sharding: new
2024-06-04T00:53:20.499627Z  INFO oprf_ipa_query{sz=50000}:attribute_cap_aggregate:attribute_cap{unique_match_keys=8161}: ipa_core::protocol::ipa_prf::prf_sharding: close time.busy=1.41s time.idle=2.24s
2024-06-04T00:53:20.499669Z  INFO oprf_ipa_query{sz=50000}:attribute_cap_aggregate:aggregate{streams=40926}: ipa_core::protocol::ipa_prf::aggregation: new
2024-06-04T00:53:30.700421Z  INFO oprf_ipa_query{sz=50000}:attribute_cap_aggregate:aggregate{streams=40926}: ipa_core::protocol::ipa_prf::aggregation: close time.busy=1.98s time.idle=8.22s
2024-06-04T00:53:30.700461Z  INFO oprf_ipa_query{sz=50000}:attribute_cap_aggregate: ipa_core::protocol::ipa_prf::prf_sharding: close time.busy=1.30s time.idle=12.5s
2024-06-04T00:53:30.700469Z  INFO oprf_ipa_query{sz=50000}: ipa_core::query::runner::oprf_ipa: close time.busy=1.85s time.idle=23.0s
2024-06-04T00:53:35.845004Z  INFO stall_detector{role=H3}: ipa_core::helpers::gateway::stall_detection::gateway: close time.busy=18.8µs time.idle=30.0s
codecov[bot] commented 3 months ago

Codecov Report

Attention: Patch coverage is 95.45455% with 1 line in your changes missing coverage. Please review.

Project coverage is 91.22%. Comparing base (b50beae) to head (0749f98). Report is 2 commits behind head on main.

Files Patch % Lines
ipa-core/src/protocol/ipa_prf/aggregation/mod.rs 88.88% 1 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1114 +/- ## ========================================== - Coverage 91.22% 91.22% -0.01% ========================================== Files 187 187 Lines 26554 26480 -74 ========================================== - Hits 24224 24156 -68 + Misses 2330 2324 -6 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.