meraki / dashboard-api-python

Official Dashboard API library (SDK) for Python
MIT License
289 stars 151 forks source link

getOrganizationNetworks hangs #193

Closed sorscode closed 2 years ago

sorscode commented 2 years ago

We have a decent size deployment of almost ~9k networks. When we try to use the following:

dashboard.organizations.getOrganizationNetworks( 1234, total_pages='all' )

The logging shows:

2022-07-19 18:31:21       meraki:     INFO > GET https://api.meraki.com/api/v1/organizations/1234/networks
2022-07-19 18:31:29       meraki:     INFO > organizations, getOrganizationNetworks; page 1 - 200 OK

It never times out or anything (using the defaults)

dashboard = meraki.DashboardAPI(meraki_key, suppress_logging=True)

Now, if I build my own python requests and pull all the data manually it works zero issue, takes time but to be expected on a large network, but the SDK just hangs. I have used the SDK for other items where we set total_pages='all' with zero issue.

SDK v1.22.1

Thoughts?

TKIPisalegacycipher commented 2 years ago

Thanks for reporting this. The logic to handle total_pages='all' is defined in rest_session.py.

Curious if you see anything in that module that stands out as a problem. I read it over and I don't see anything in particular that shouldn't work. Which version of the requests library is installed on your client machine? I see a few bugs that have been fixed in the requests library, so if you're on an older version it's a good idea to upgrade.

@coreGreenberet you might want to review this as well.

coreGreenberet commented 2 years ago

@TKIPisalegacycipher I can't reproduce this issue

I've created an organization with 10170 networks. After ~11 seconds all pages were downloaded and I've got a list with all networks

my code:

import asyncio
import meraki

def main():
    dashboard = meraki.DashboardAPI(
        log_file_prefix=__file__[:-3],
        print_console=True,
        use_iterator_for_get_pages = False)
    result = dashboard.organizations.getOrganizationNetworks( <ORGID>, total_pages='all' )
    for x in result:
        print(x['name'])

if __name__ == "__main__":
    main()
sorscode commented 2 years ago

@coreGreenberet it was determined to be an issue on the shard. We are working to consolidate down but till then will have to deal. Thanks.

TKIPisalegacycipher commented 2 years ago

Thanks for the responses, folks! Closing accordingly.