[ISSUE] Pagination doesn't work - unable to fetch all data

[martinkiska]

zpa.server_groups.list_groups() doesn't return all configured server group as described in function description. Instead it follows default pagezise=20 and page=1 and returns only 20 values. It looks to me that pagination is not working and function is unable to roll over all data.

https://github.com/zscaler/zscaler-sdk-python/blob/668f1e5c25fa84e2979238d4f05b25bf1317f2a7/zscaler/zpa/server_groups.py#L36

        a = self.zpa.server_groups.list_groups(pagesize=500)
        b = self.zpa.server_groups.list_groups()
        print (f"Size of 'a' with customized pagesize {len(a)}")
        print (f"Size of 'b' with customized pagesize {len(b)}")

Output:

Size of 'a' with customized pagesize 22
Size of 'b' with customized pagesize 20

While debugging it further. Issues seems to be with L505 https://github.com/zscaler/zscaler-sdk-python/blob/668f1e5c25fa84e2979238d4f05b25bf1317f2a7/zscaler/zpa/__init__.py#L505

you expect there "nextPage" in dict, but in real that is not returned from API server, due to that I am unable to fetch all data from the server and process it:

[kvzonrise]

Hello, the same issue apply for list_users

[martinkiska]

Hello, the same issue apply for list_users

Yep, I agree - issue is in underlying get_paginated_data() function, which is being used by all the other functions. So pagination will not work in every function.

[willguibr]

@martinkiska / @kvzonrise Please take a look at the following branch zpa-#199-fix/fixed-complex-condition-template compile the code, and try the pagination again. Once you confirm it's working, we'll put out a new minor release out.

Zscaler DevRel

[martinkiska]

Hi @willguibr, thanks for quick response. I was checking it and pagination still doesn't work. I can see that you increased default size of pagesize to 500, it was working for our 429 production segments. To overcome the number I randomly generated 100+ app segments so currently I have more than 500 (exactly 534).

As nextPage key doesn't exist in the response, this if statement is not working

Probably if you replace this line with below code it will start working - https://github.com/zscaler/zscaler-sdk-python/blob/d5991ab80bcd0be152f3f8e639c286089618e41c/zscaler/zpa/__init__.py#L512 I tested it with pagesize=20 and I was able to roll over all app segments

                if int(params.get("page")) < int(response_data.get("totalPages")):
                    next_page = int(params.get("page"))+1

[willguibr]

@martinkiska / @kvzonrise We have pushed a new adjustment, to the custom pagination for both ZPA and ZIA. Please test and let us know if that fixes the issue you've reported. Thank you

Zscaler DevRel.