(429): Too Many Requests Error

[rhalabi]

Hi, I finally got the PowerBIRestAPI to refresh via a gateway connection, but I get the below error when refreshing, anyone know why? Is there an API limit with this connector? Any way to fix?

Data source error: {"error":{"code":"DM_GWPipeline_Gateway_MashupDataAccessError","pbi.error":{"code":"DM_GWPipeline_Gateway_MashupDataAccessError","parameters":{},"details":[{"code":"DM_ErrorDetailNameCode_UnderlyingErrorCode","detail":{"type":1,"value":"-2147467259"}},{"code":"DM_ErrorDetailNameCode_UnderlyingErrorMessage","detail":{"type":1,"value":"Web.Contents failed to get contents from 'https://api.powerbi.com/v1.0/myorg/admin/Groups?$expand=reports,users,datasets,dataflows,dashboards&$top=1' (429): Too Many Requests"}},{"code":"DM_ErrorDetailNameCode_UnderlyingHResult","detail":{"type":1,"value":"-2147467259"}},{"code":"Microsoft.Data.Mashup.ValueError.DataSourceKind","detail":{"type":1,"value":"PowerBIRESTAPI"}},{"code":"Microsoft.Data.Mashup.ValueError.DataSourcePath","detail":{"type":1,"value":"PowerBIRESTAPI"}},{"code":"Microsoft.Data.Mashup.ValueError.Reason","detail":{"type":1,"value":"DataSource.Error"}},{"code":"Microsoft.Data.Mashup.ValueError.Url","detail":{"type":1,"value":"https://api.powerbi.com/v1.0/myorg/admin/Groups?$expand=reports,users,datasets,dataflows,dashboards&$top=1"}}],"exceptionCulprit":1}}} Table: Capacity_Datasets. Cluster URI: WABI-US-NORTH-CENTRAL-C-PRIMARY-redirect.analysis.windows.net Activity ID: e624c738-0489-4134-931f-eca054d2df57 Request ID: 28941de3-1126-d8eb-562d-2a2573dbda86 Time: 2020-06-26 13:12:51

[klinejordan]

There is a 200 requests per hour limit on this end point: https://docs.microsoft.com/en-us/rest/api/power-bi/admin/groups_getgroupsasadmin

[rhalabi]

There is a 200 requests per hour limit on this end point: https://docs.microsoft.com/en-us/rest/api/power-bi/admin/groups_getgroupsasadmin

Wow, that stinks. How does the connector work, does it request more than 200 calls at a time?

[klinejordan]

You can make this connector make more than 200 calls at a time using the function that lets you design your own URL but the Workspaces (Admin) call chunks the GetGroupsAsAdmin calls by 5000 workspaces. Also remember that if you're using referenced queries in Power Query you're making the same calls twice so you need to be judicious about how you architect your queries. I suggest using Fiddler to understand how many calls you're making through the connector.

[rhalabi]

You can make this connector make more than 200 calls at a time using the function that lets you design your own URL but the Workspaces (Admin) call chunks the GetGroupsAsAdmin calls by 5000 workspaces. Also remember that if you're using referenced queries in Power Query you're making the same calls twice so you need to be judicious about how you architect your queries. I suggest using Fiddler to understand how many calls you're making through the connector.

Not sure I understand, can you please explain how I increase the connector to more than 200 calls, using what function?

[migueesc123]

Hey @rhalabi ! You can check the "functions" folder inside the Navigator window which holds the "Get Data" function. You can use that one to query the endpoint of your choice.

You can read the license for this repo here which does let you modify the custom connector as you need should you desire to change any of the behavior to fit your specific needs and scenarios.

Should you need paid assistance setting this up or modifying the custom connector for your specific needs, you can contact me via email at info@poweredsolutions.co.

Best!

[rhalabi]

Hey @rhalabi ! You can check the "functions" folder inside the Navigator window which holds the "Get Data" function. You can use that one to query the endpoint of your choice.

You can read the license for this repo here which does let you modify the custom connector as you need should you desire to change any of the behavior to fit your specific needs and scenarios.

Should you need paid assistance setting this up or modifying the custom connector for your specific needs, you can contact me via email at info@poweredsolutions.co.

Best!

Thx appreciate it.

[klinejordan]

You can make this connector make more than 200 calls at a time using the function that lets you design your own URL but the Workspaces (Admin) call chunks the GetGroupsAsAdmin calls by 5000 workspaces. Also remember that if you're using referenced queries in Power Query you're making the same calls twice so you need to be judicious about how you architect your queries. I suggest using Fiddler to understand how many calls you're making through the connector.

Not sure I understand, can you please explain how I increase the connector to more than 200 calls, using what function?

This connector by itself doesn't make more than 200 calls per hour, and that limitation is a Microsoft limitation, not one enforced by the connector.

Depending on the design of your queries you could be making more than 200 calls per hour and thereby hitting the limit. Using a tool like Fiddler while you refresh your data model in Power BI Desktop will help you understand how your query design is impacting the number of API calls you're making. My guess is you're using referenced queries quite a bit without knowing that referenced queries can actually trigger the base query more than once but we can't really support this one. We'll fix bugs in the connector but this is a bug in your data model.

As a sanity check, I use this connector every day in a tenant with 50,0000+ workspaces and I never hit the 200 calls per hour limit on GetGroupsAsAdmin. It's up to you to design queries and a data model that will work within the API's limitations.