Long Runtime for One Day's Data Retrieval

[akml12]

I am writing to report a runtime issue with the aw_freeform_table() function in the adobeanalyticsr package.

Problem: When using the aw_freeform_table() function to retrieve data for a single day, the runtime is excessively long, taking more than 10 minutes to complete.

Additional Notes:

I have already attempted to optimize the code by specifying a smaller date range, and ordering dimensions strategically, but the issue persists. My 'top' parameter is set as c(0,500,500) my dimensions are (date,trackingcode,campaign name) I have validated the metrics and dimensions used in the query to avoid unnecessary API calls for validation.

Expected Behavior: The function should fetch data for the specified date within a reasonable timeframe, typically a few seconds or a minute at most.

Actual Behavior: The data retrieval process takes more than 10 minutes, which significantly impacts the efficiency of data analysis workflows.

Please let me know if there are any recommended steps to further optimize the data retrieval process and reduce the runtime for fetching data for a single day. I am eager to hear your insights on how to address this issue and improve the performance of the aw_freeform_table() function.

[charlie-gallagher]

The query times are related to the number of dimension levels in your breakdown. To run a (0,500,500) query, for a single day of data, it still must make 500 requests to the API. There is no way to execute this number of requests quickly, unfortunately :/ at a rate of 1 second per query, it should take about 8 minutes.

The adobe analytics API is just not optimized for book breakdowns. The older API (1.4) was better for this, but I haven't used it in a couple years so I'm not sure what the status of it is.

If you'd like a more detailed description of how the API works, I can expand on this a bit more

[benrwoodard]

@akml12 Did the answer solve this for you?

[akml12]

I got the logic. But my requirements will need to apply multiple dimensions breakdown and for this the runtime is very high

---

From: Ben Woodard @.> Sent: Tuesday, October 24, 2023 8:49 PM To: benrwoodard/adobeanalyticsr @.> Cc: Muhammed, Akmal (BLR-INA) @.>; Mention @.> Subject: Re: [benrwoodard/adobeanalyticsr] Long Runtime for One Day's Data Retrieval (Issue #160)

@akml12 Did the answer solve this for you? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned. Message ID: benrwoodard/adobeanalyticsr/issues/160/1777464309@ github. com

@akml12https://urldefense.com/v3/__https://github.com/akml12__;!!N96JrnIq8IfO5w!njYFY_U15nwKzxw9RXYryxJDfLj1_rz08f57dFe14MuyiVBW96pppU71bk2VZrS-pHezqtvi6ahi2D1kO2w-oJb9f_3u9mMp38tfDg$ Did the answer solve this for you?

— Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https://github.com/benrwoodard/adobeanalyticsr/issues/160*issuecomment-1777464309__;Iw!!N96JrnIq8IfO5w!njYFY_U15nwKzxw9RXYryxJDfLj1_rz08f57dFe14MuyiVBW96pppU71bk2VZrS-pHezqtvi6ahi2D1kO2w-oJb9f_3u9mNQD0bboA$, or unsubscribehttps://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/BBVQOOQPYQCZ7BJRES7G5VDYA7L7TAVCNFSM6AAAAAA3BCB6BOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONZXGQ3DIMZQHE__;!!N96JrnIq8IfO5w!njYFY_U15nwKzxw9RXYryxJDfLj1_rz08f57dFe14MuyiVBW96pppU71bk2VZrS-pHezqtvi6ahi2D1kO2w-oJb9f_3u9mPbyp3zDA$. You are receiving this because you were mentioned.Message ID: @.***>

This message contains information which may be confidential and privileged. Unless you are the intended recipient (or authorized to receive this message for the intended recipient), you may not use, copy, disseminate or disclose to anyone the message or any information contained in the message. If you have received the message in error, please advise the sender by reply e-mail, and delete the message. Thank you very much.