privacysandbox / aggregation-service

This repository contains instructions and scripts to set up and test the Privacy Sandbox Aggregation Service
Apache License 2.0
64 stars 34 forks source link

AggregationService always returning PRIVACY_BUDGET_EXHAUSTED error after test with "debug_privacy_epsilon" #61

Closed dbrito closed 5 months ago

dbrito commented 5 months ago

Hello team, how are you?

Guys, after uploading the AggregationService to the AWS environment, I was carrying out some tests to generate the summary report, with these tests I noticed that the noise was greatly impacting the values ​​of the metrics, given this scenario I implemented scaling the value and defining the epislon, but after adding the epislon definition to /createJob, the AggregationService is always returning the PRIVACY_BUDGET_EXHAUSTED error, regardless of whether they are new reports (new .avro files).

I wanted to see if anyone had any tips on how to identify the source of this error and consequently how I could get around it.

CreateJob body Request

{
    "input_data_blob_prefix": "reports_17",
    "input_data_bucket_name": "uolcsm-uolads-aggregate-reports",
    "output_data_blob_prefix": "output/summary_report_25.avro",
    "output_data_bucket_name": "uolcsm-uolads-aggregate-reports",
    "job_parameters": {           
        "debug_privacy_epsilon": "10",     
        "attribution_report_to": "https://attribution.ads.uol.com.br",
        "output_domain_blob_prefix": "output_domain.avro",
        "output_domain_bucket_name": "uolcsm-uolads-aggregate-reports"
    },
    "job_request_id": "test80"
}

AggregationService getJob response

{
    "job_status": "FINISHED",
    "request_received_at": "2024-06-19T20:20:56.246430Z",
    "request_updated_at": "2024-06-19T20:21:13.512574536Z",
    "job_request_id": "test80",
    "input_data_blob_prefix": "reports_17",
    "input_data_bucket_name": "uolcsm-uolads-aggregate-reports",
    "output_data_blob_prefix": "output/summary_report_25.avro",
    "output_data_bucket_name": "uolcsm-uolads-aggregate-reports",
    "postback_url": "",
    "result_info": {
        "return_code": "PRIVACY_BUDGET_EXHAUSTED",
        "return_message": "com.google.aggregate.adtech.worker.exceptions.AggregationJobProcessException: Insufficient privacy budget for one or more aggregatable reports. No aggregatable report can appear in more than one aggregation job. \n com.google.aggregate.adtech.worker.aggregation.concurrent.ConcurrentAggregationProcessor.consumePrivacyBudgetUnits(ConcurrentAggregationProcessor.java:472) \n com.google.aggregate.adtech.worker.aggregation.concurrent.ConcurrentAggregationProcessor.process(ConcurrentAggregationProcessor.java:329) \n com.google.aggregate.adtech.worker.WorkerPullWorkService.run(WorkerPullWorkService.java:142)",
        "error_summary": {
            "error_counts": [],
            "error_messages": []
        },
        "finished_at": "2024-06-19T20:21:13.503639102Z"
    },
    "job_parameters": {
        "debug_privacy_epsilon": "10",
        "output_domain_bucket_name": "uolcsm-uolads-aggregate-reports",
        "output_domain_blob_prefix": "output_domain.avro",
        "attribution_report_to": "https://attribution.ads.uol.com.br"
    },
    "request_processing_started_at": "2024-06-19T20:21:02.916500310Z"
}

*I was taking a look at NoiseLab and the solution to mitigate the impact of noise would be to scale my values ​​and use an epislon greater than 0

Reports used during testing:

//Histogram 1
[
 {
  "key": "0x29a",
  "value": 10054
 }
]

//Report 1
{
   "aggregation_coordinator_origin": "https://publickeyservice.msmt.aws.privacysandboxservices.com",
   "aggregation_service_payloads": [ {
      "debug_cleartext_payload": "omRkYXRhlKJldmFsdWVEAAAnRmZidWNrZXRQAAAAAAAAAAAAAAAAAAACmqJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAGlvcGVyYXRpb25paGlzdG9ncmFt",
      "key_id": "b16a91ad-6d09-491f-af3e-31faef3d116f",
      "payload": "tQpGPd6LRE1I16SoUk9ifwdyvc9zZAG1Qj1zAQSLhktOi8k1kLcBgYpTwq+OfpLIBpePCxHC1LeXiCdPAgEPWQW6gM6wumBTgSF611YGN+i52IZPYyR1k4F9yGocRg5AE6wjMDxLVr71QyJ7At17Ulje6Jj5rIlByPTAnNUWwDB0T1XuMx9gWn3mhS3M1fTvo1Z77VhnysZJuUdu5Q1BeLX1EUkl1o08tw8bRFPjIinBgVEX1iJRymxeCmfiqV/oZOUf7yU1XUWOEHiVgWQFc9ZLgbja3xLzOL+WlOEXD6Wn1Nu6Gq1LD0U7G450jP+x2wa8fMmHUS9LPmyM/uP8c7dkEHHRV02TuhtR7sepGmnaMfGHJNT/poiEMX1XRF6/3iqhn9o6kyrFuMZ0VPEdrCtN4RIpReQUJXD318jTYPCpxtVUQsZojdU05+IavAUYtVYMepUXV87VBjRVEzCagLYekw9AOGlPOtfVSfGl7DZech+pUHwRA2GPT/W6mnNfWMxq76XsEgO+Xc1Ap1FKNmUZhEgzdP9PFwrYbn9GEwjhvyUxuPk85lF10yqRbrkOSiYH3RHMy80L1uGJIeiAFPvGtHwWNmrfLMKqxNQLlYnJPUjnKgM7jgO0VNDrCJxUuSo8u0WHlGkDK8F7tAlD3W2k45ZkGFPKlzXeaP1mqbpP7YSolyMFHvzwipq8ztGqzGN19BKxQmhlJQLW80B/kjzqK1GV4xzPHWtqq5yctV3OScnfws9RRJVUsoNIXakX2EwXLuapd2AhgXuRp3Ojcg/l9JjxKZliFYA2aMjT1yYfD1JCcB3l7cBYvPb4QKPgYXOXaR6lCSX6ZMVgFnQm7Lk7CYhZTaSuaXDj5j3irIWLnN7aXOMsZ/SDamRdQ30Hm5eFzaANpaENWNO3oil+fRrlIe8bCj5TPPIwZcNdTbJ6564cT2MQBMUiZeqFq7+z61H2E8MJbSuXuXRjMFoc095nM2sIkci9Fyb80VwOPlXHdZN4Azf85taPhGjUOrXrs7X8HGIAouqhEiHjBa0Z6IDBdYNu3f6hSwwn"
   } ],
   "shared_info": "{\"api\":\"attribution-reporting\",\"attribution_destination\":\"https://cliente.com\",\"debug_mode\":\"enabled\",\"report_id\":\"f23783ec-b335-41ff-8885-dc529ccf800f\",\"reporting_origin\":\"https://attribution.ads.uol.com.br\",\"scheduled_report_time\":\"1718829420\",\"source_registration_time\":\"0\",\"version\":\"0.1\"}",
   "source_debug_key": "647775351539539",
   "trigger_debug_key": "647775351539539"
}

//Histogram 2
[
 {
  "key": "0x29a",
  "value": 10074
 }
]

//Report 2
{
   "aggregation_coordinator_origin": "https://publickeyservice.msmt.aws.privacysandboxservices.com",
   "aggregation_service_payloads": [ {
      "debug_cleartext_payload": "omRkYXRhlKJldmFsdWVEAAAnWmZidWNrZXRQAAAAAAAAAAAAAAAAAAACmqJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAGlvcGVyYXRpb25paGlzdG9ncmFt",
      "key_id": "3e9d0be8-6cc6-4e80-862f-b5c1ba34b03d",
      "payload": "3fuou19wP7qL616ylGdTVp26HiFLpylrHIXf5sHF/xs+3prz3S3cQ2J16ojKQkyD9AWOKU+tBYMZJXCT6/jDD8Tv0kP6TjPGAiyOyWQ5ATdw8xsPqCxZuvtVeywxHZEZ65NBgOWm/txLBMMtQLAFHc0s+gdq4pHPem9Q1PHrAtCIO3dUO0xlvSMSIZJzhjPrNzB5XBzfLHXs0ACl533RLbv7SRuxb9njkVE0WIx80OY6jeBiUFT5NHaP610YU+tULCaA6mg7XtQUWh1voWDKplJwYlfPOWoPItMX0v1OS69+Zmc669CUN/hV+PI/meyX9PagbwfXs4Rt3IrQAN5MzaXHSPq5Z/eMsbiOfhCO3lGYxBKo/9KpWG8Qt9BWAALntHqg+y7BItiCC8NyY9pJZ3n9ghul2QdJy1QkDOuFOwquiKQsPmFMY5kQUaN4d0E37PwKT7fmZiDyblIUCBHjaS+jM7eUH3wYCoylm4HeqR9gzA2BdN+Vm+zctUfvYmT2QOLEVKwBMp+UOZ67P5ABOMLV2jUIkx6uoqY9eDngJYk9bRwQbNmqTp54hbdOnjg8eYt6MocXjWUttfg6NKFQTEHkQ125SjDBG+T4angbM91rzZA2gUWqH32yszTAsCVsgRUu00iQTpnjtVdCxIyKKPZShGZiVloJHv7bG2sbvgp8aVQ0Y3xjL38Sea9xUvQDdsgwnVSeYeJUNakmi0Ni0bVdbplVugMuE1SoZ/Xf6Y7klg1peEygBoVC1W7i2Z/VPoaoQ7ctwyqmMXoMXlVcLsdB8Tp1ph6+aJTfZcF8ZwpU0vfq+MRjEO9dp6kqu3WBD5Q+jaAOykFW3wXBN+CQx1jk07dGvSf2JZcwjG62f/JUVPvdpyJEmDaaIfupoMap23BdD82F/9y43prFJwIGDOzLolDae7bJh1q8A2BTDldKg5z4x7Hc85MkpmiAuY7kZ1fyChxsu6COMfucqmbKrzDzZGwpLnnKl/95kk7eTlbr4o7VTCs+cPVHV0GtMTNydUGUgbrvPiuh9AmehHvGavWVx2++oQSpyEvv"
   } ],
   "shared_info": "{\"api\":\"attribution-reporting\",\"attribution_destination\":\"https://cliente.com\",\"debug_mode\":\"enabled\",\"report_id\":\"15cfee94-a102-4215-9367-1e17862d7769\",\"reporting_origin\":\"https://attribution.ads.uol.com.br\",\"scheduled_report_time\":\"1718829459\",\"source_registration_time\":\"0\",\"version\":\"0.1\"}",
   "source_debug_key": "647775351539539",
   "trigger_debug_key": "647775351539539"
}

//Histogram 3 
[
 {
  "key": "0x29a",
  "value": 10029
 }
]

//Report 3
{
   "aggregation_coordinator_origin": "https://publickeyservice.msmt.aws.privacysandboxservices.com",
   "aggregation_service_payloads": [ {
      "debug_cleartext_payload": "omRkYXRhlKJldmFsdWVEAAAnLWZidWNrZXRQAAAAAAAAAAAAAAAAAAACmqJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAGlvcGVyYXRpb25paGlzdG9ncmFt",
      "key_id": "c29773d9-a72b-4147-bd60-3777a3844054",
      "payload": "e3oi8KV4uK8aYUgZIlYFFrZsENSY0gsgcdzYtKECqnFLzGF+VwHsLvbzg5QAAfBhqM9y/6bgCUssbEBfOYwNcSGNP6KTxuCteocfbL7FcpijnsRQ8001oc8Hhcx1efr/h9Kh/14w3HDd74rRMMnNS2L7RIEnGqkcZM6Y4/HGDIK0nd9CptuxZjjxuLs37fnIj2ZRgTrKvVEk8EoE0ro+54cBbfYLMC+SMLJ5NefRmNA29gUnXuzvkOCBc5zp9XvtedyVjZzcatFJooPdY5UJhCYso121vngmk5O4U2KAyf7mOL6vPkhOAf4irC5NUymmSqjHrfCwMZ5aGGJcfSYXwd5xp/MRBXlP9/hgiyg7vLUucul9jvNMZ9RDXlE/pZGpq2u9pOsmlSl2wKh4I7xhv0PQgOwjj5N43YnGvarFC6Js7mVyVyDccnga6u6RdJsfH2i0ObhF2vJenkexycLLykCPS5TZrUkIBolglkq/Z5tiXIbnugh8jrmy4IAd8z/XBK9CgPq+sGmdLG8ZKVKiguylwXTL/Zl0IxdNhl6GIIRkfABaDUSUr7tgCiLWHVAKLMbAyDalluGdfkRwhf7fMnI9fN6YIpq6VJ+H7JXNbdnsP2maL4z+8eGs+Q2eqa8eMkB2qYinPCSQIHLR47Xk9xn63H2EQhko1LcNq9lF0Id3xfvZI5WU/i+lUv4ZpF1XTV/3aciHD8C5gKzgyceFmv65EjiHhOQ19bu9tULuxaROAozrJ9wL5SieFgVb8QqpvshrhhQGu4E0FtnNwMOUrGm1R8so5J0n4vqUM8t2f7GAdqq7LIMXfEatjENV4bdWyJev8IRnQtZPstjLKHFJM+v2QtsLIx2z2nGG0NqgvGlpT1jBOvVYg0hiosbtSAF8zb8fwTPhg7mgG/sNR2d2uVfQ/Ou8APip+BBTv871r2OMk9d6x0Zey61T5E7kP4di3GCXKT2qkUmr4yw1r9RvVkPZi37oWruN2y82DzGsB3XXbFtZ8J3gV0oO8xDx7+vHCQk1AarV88D85s1aV4wb808p06NRtXGE/R5D"
   } ],
   "shared_info": "{\"api\":\"attribution-reporting\",\"attribution_destination\":\"https://cliente.com\",\"debug_mode\":\"enabled\",\"report_id\":\"63e0ef38-d97e-431c-9b79-2abc6ee92793\",\"reporting_origin\":\"https://attribution.ads.uol.com.br\",\"scheduled_report_time\":\"1718829513\",\"source_registration_time\":\"0\",\"version\":\"0.1\"}",
   "source_debug_key": "647775351539539",
   "trigger_debug_key": "647775351539539"
}

//Histogram 4
[
 {
  "key": "0x29a",
  "value": 10096
 }
]

//Report 4
{
   "aggregation_coordinator_origin": "https://publickeyservice.msmt.aws.privacysandboxservices.com",
   "aggregation_service_payloads": [ {
      "debug_cleartext_payload": "omRkYXRhlKJldmFsdWVEAAAncGZidWNrZXRQAAAAAAAAAAAAAAAAAAACmqJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAGlvcGVyYXRpb25paGlzdG9ncmFt",
      "key_id": "71009a63-aa17-45b9-a5a9-49d78067210f",
      "payload": "GvnO9L57AFyTdlejpGgXRmC8AfQG318hEl6K0gOoVWB4GHHNg0h0Gs/LwmhVCzUMq89rB6LuVD8SZ9XXpMFVWbP2XvJMU5gokgOo1TTtAWwvEBFxfcvJZ5hqfLMsQGRb91UrwRhUI+6GyhrDXV76sKS4j3D5OevAfamcdd9fp8SZhfB7umPL08ESXekkEDPHmmblBjrKLLyw0w4/23MMyCGRwnYtdc8uZYR1p0uyE3VoPYYnaRmHUBddsgRsvg+u2cwRho0PxIReHHRgHS50hELkSf0DXJjJHRd5ukOIu15LVBFBQ0VIoMB9RB61PkynjyhuLlqZk0d1Pif+ALQ7jyE4xiYKCgmkq/SowT6EW2v1UmX54KvcDXZx/eWh2rz373ZlGitYa/TtBPNDngM9PYVZrrRANWBjya4nBB6ElPAsVjgjF6+Zo3rkSj3tF0+WxdBOM7jbYbIjPOVoWViKUq6b2b5FMNR4FFYIDsVwNaLLrLBoDot09DFa+1gSHBNcYOyFdsy2sx9Mg/NOa3VeA8W6nTKLZd/KzDq70H7jRcj4wwZCL+gy4s45XBBHRtcaCfcokvEpCr3USvDP7miICzqD9+XL73qiyLpGeTpEZwAVGomX1kLSpTxsjaNcpoO0eNZMl7DVmsd37vjq6z43DqRuhSpigK3nbLK3lMtOVNUbCzItVqyYcSNpIdDiZGGtODPwppGeE1Xp8T0AxXp76PMR/pAUTaglxjmRVAhUZBIPuUKAXrDHCoQyBktqQHWEIBt0QomqcggQPgOTCP4nQ5nTE42FdhevV5DZoZik7UFLEx7WV09/96GBogV91dbkI9pG4717jgn8pebVDxP7xJ7WZRDbOmI4ESCcIXz5YqjKqoL1UFAfeOfLDVALh99LZJayy1igxQFdzeteKmzYmUPUaosPCXdJd552SynGow7mjZbWidLn8p51tdUvPyEtaTq9P4D360zhrld3ZkYYbdclBl7ywgxWFauczrh+oJOgWdm6C2F9rAKXwz43fiI86+bCZEnhoUFnJZihSoGNTrP519rDx0Bf/qw+"
   } ],
   "shared_info": "{\"api\":\"attribution-reporting\",\"attribution_destination\":\"https://cliente.com\",\"debug_mode\":\"enabled\",\"report_id\":\"e09684d5-4634-4ca7-af62-18bb03090f35\",\"reporting_origin\":\"https://attribution.ads.uol.com.br\",\"scheduled_report_time\":\"1718829605\",\"source_registration_time\":\"0\",\"version\":\"0.1\"}",
   "source_debug_key": "647775351539539",
   "trigger_debug_key": "647775351539539"
}

//Histogram 5
[
 {
  "key": "0x29a",
  "value": 10172
 }
]

//Report 5
{
   "aggregation_coordinator_origin": "https://publickeyservice.msmt.aws.privacysandboxservices.com",
   "aggregation_service_payloads": [ {
      "debug_cleartext_payload": "omRkYXRhlKJldmFsdWVEAAAnvGZidWNrZXRQAAAAAAAAAAAAAAAAAAACmqJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAKJldmFsdWVEAAAAAGZidWNrZXRQAAAAAAAAAAAAAAAAAAAAAGlvcGVyYXRpb25paGlzdG9ncmFt",
      "key_id": "3e9d0be8-6cc6-4e80-862f-b5c1ba34b03d",
      "payload": "BtI4rdy6Lult3ikxY/QN3CTgPbhWyo1rzKwFn8yTZydUWSs5Kg1k+GW1kzfVQriFM1GWifKPfjyrFsJcEtS00JTpMKDWYuvquywq30KUgZrLowaEW5ajbDMS9+mDsJgEiIVy/Y7l4Zx7j14sdklOF7IJqXj9Di3sSJOFlB5IWuVLim03IB4cvOKu+KZKqkncrXnIliKrhZW3PuUmV3Beb5cHgR6gC1YOo4xHdr5fhM8IMSA4YBRKkp02Gxc1bv/B8PGDBkPFKwH+xtFWBw9myRja/ExvgNet7QTYReiOiKXsJom3iT8f2bObAQ2Hi4EGdrXSxxYUVsLPsFElRkGIy//mJyCDTqC/ItM3EpdHtJySADaeBp3viDilUyAWbDceNMtQeKyeQe9IWEBYClHAgFS8lqsBp/Lm/d2i9OieDTyUb+tIzIXkNpdEB7Zftfkr2ovo66rhFI/JMheoU29t/6mQHVUMJb+unaUzdgjT3E6CWggqsVBiICueW48N2sUQkX9VSSJmlYTlltI/I+I1AOTPgMI9zlaB9+L5qUzTYREanw+DuGdbM/2eqvQ8mxHEKvOJJ2XRSG7nQVLcWsMSh366Rdl830sv6NGyTCrZBaELV2mxIp7kcr2xx3xxSVJRIEG9L5wAgbc33HQmC+8x7i3I/+PvKHLx9RDaMw9yaCmEXH3B57no87kvOqIXLHULBt/mZ9346TaYChs3smxYzjErtCJI/CKFeWYIFpD3Ix0p/M96+qVpIp2xmPghXjhM+OGFd0ieQDGofgvN1S1qAccw5tXyyAc+3S0EaNkXeEE/Y4WN4lgxpbI06Qbk6X/ltXsm7xHILLmGOv4ic/O3/tCNWRQey8DawjmCCNINiHWo+QHiry3fjMkCKpiHKnxq4wJxGjSHh445prf7KZSVGCX2JSnkhXpZlJWsyN2zF3Tv3LRU/eOSqfiE9xuIccQsSJaK97wapgkvEmer+UecD7aWaPKDid4KnUiPIyTS9lWr3YzjOIhSlp4tYUWvdjsTkKKivQRFWFcQsRRMidINa0f7YeJCk6sc+7T2"
   } ],
   "shared_info": "{\"api\":\"attribution-reporting\",\"attribution_destination\":\"https://cliente.com\",\"debug_mode\":\"enabled\",\"report_id\":\"4b597fd1-b455-4b2f-8d13-591aa894efa8\",\"reporting_origin\":\"https://attribution.ads.uol.com.br\",\"scheduled_report_time\":\"1718829826\",\"source_registration_time\":\"0\",\"version\":\"0.1\"}",
   "source_debug_key": "647775351539539",
   "trigger_debug_key": "647775351539539"
}

Thanks in advance

maybellineboon commented 5 months ago

Hi @dbrito

Just to clarify if you are reusing your reports everytime you batch? PRIVACY_BUDGET_EXHAUSTED error is caused by the shared ID of reports being batched more than once. Each report will be assigned a "shared ID" which will consist of the shared_info fields api, reporting_origin, destination_site, source_registration_time (truncated by day), scheduled_report_time (truncated by hour) and version. This will mean that multiple reports can belong to the same "shared ID" should they share the same attributes of the shared_info field.

Based on the disjoint batches, would you be able to ensure that there are no overlaps on the batches based on the Shared ID? Please ensure that source_registration_time (truncated by day) and scheduled_report_time (truncated by hour) is taken into consideration. We also have some guidelines on batching strategies that you can look into.

An example is if you have the following shared_info field (below), you can see that the API is the same (attribution-reporting), the attribution_destination is the same (https://privacy-sandcastle-dev-shop.web.app), the reporting_origin is the same (https://privacy-sandcastle-dev-dsp.web.app). The source_registration_time is the same (0). So, we only have scheduled_report_time which is different. But if we take a look at the scheduled_report_time, one is "Tuesday, January 2, 2024 5:19:12 PM" and the other report is "Tuesday, January 2, 2024 5:24:22 PM" if we truncate them by the hour, they are both "Tuesday, January 2, 2024 5 PM". Which means that both reports have one privacy budget. So you can have hundreds/thousands/more reports which is equivalent to 1 privacy budget. All of the reports with the same shared id will have to go in the same batch.

"shared_info": "{"api":"attribution-reporting","attribution_destination":"https://privacy-sandcastle-dev-shop.web.app\",\"debug_mode\":\"enabled\",\"report_id\":\"af0cfc09-18d3-4234-8d02-1e36a189a7c4\",\"reporting_origin\":\"https://privacy-sandcastle-dev-dsp.web.app\",\"scheduled_report_time\":\"1704215952\",\"source_registration_time\":\"0\",\"version\":\"0.1\"}",

"shared_info": "{"api":"attribution-reporting","attribution_destination":"https://privacy-sandcastle-dev-shop.web.app\",\"debug_mode\":\"enabled\",\"report_id\":\"1a1b25aa-5e1b-43fc-b80e-9cc9e8ce7658\",\"reporting_origin\":\"https://privacy-sandcastle-dev-dsp.web.app\",\"scheduled_report_time\":\"1704216262\",\"source_registration_time\":\"0\",\"version\":\"0.1\"}",

dbrito commented 5 months ago

Hi @maybellineboon, I think I understand now!

During my tests, I certainly activated the AgregationService more than once within the minimum period of 1 hour, although each test had new reports, they were created at the same time and consequently maintained the same "shared Id", probably causing the error "PRIVACY_BUDGET_EXHAUSTED" , we will adjust the way we run our tests so that the AggregationService does not trigger more than once within the same hour.

Thank you so much for the help

*AggregationService triggered for the first time at the “new” hour image