search

mbilling / PBIConnectors

The Microsoft Power BI custom connector support-project
MIT License
71 stars 33 forks source link

API Throttling #19

Open brechtvan opened 4 years ago

brechtvan commented 4 years ago

I'm getting blocked by the facebook api (needed to upgrade to v5) because of rate limits. Any solution for this? Currently trying to add some delays with Function.InvokeAfter. https://developers.facebook.com/docs/marketing-api/insights/best-practices/

mbilling commented 4 years ago

Good point - I have been thinking about this could turn out to be a issue previously. My initial thoughts are that applying exponential back-off in PQ/M sounds like a big challenge. Did you do any searching on this subject? I can ask Chris Webb, he would be the one to know if anyone does.

brechtvan commented 4 years ago

I have zero experience in PQ/M, so wouldn't know how to start with exponential back-off.. This would also result in a very slow connector. Currently looking into the build-in Ads reporting (https://business.facebook.com/adsmanager/reporting) raw data export to .xslx, but the seems to be no api support for this at first sight

brechtvan commented 4 years ago

Currently querying on campaign, with level set to adset and time increment to all_days.. This already gets me further without getting throttled. FacebookAds.GetPagedData(baseUrl & campaignId & "/insights?" & Uri.BuildQueryString([access_token=access_token,fields="campaign_id,adset_id,ad_name,account_id,adset_name,campaign_name,reach,impressions,frequency,unique_clicks,spend,unique_inline_link_clicks,cpc,cpm,clicks,account_currency,account_name,social_spend,objective,inline_link_clicks,inline_post_engagement", level="adset", date_preset=date_preset, time_increment="all_days", limit=fetchLimit ]), {})

mbilling commented 4 years ago

Have you read, Insights Call Load Limits seciton of https://developers.facebook.com/docs/marketing-api/insights/best-practices/

Maybe we could leverage this wrapper for retry https://gist.github.com/CurtHagenlocher/68ac18caa0a17667c805