assigned_groups field in okta_application is limited to the first 20 groups

[tinder-tder]

Describe the bug When getting the 'assigned_groups' field of from okta_application table, the data is limited to the first 20 groups and the api calls are not pagenating. Also there is no way to pass the query param 'limit' (ie: “/api/v1/apps/${APPID}/groups?limit=200”) to work around the current default pagenation size of 20 (to reduce api calls).

https://developer.okta.com/docs/reference/api/apps/#list-groups-assigned-to-application

Steampipe version (steampipe -v) v0.10.0

Plugin version (steampipe plugin list) v0.2.0

To reproduce have an okta application with more than 20 groups assigned to it and query its assigned groups

select jsonb_pretty(assigned_groups) as groups from okta.okta_application where filter = 'group.id eq "xxxxxxxxx"' AND label = 'foo'

query is formatted with the where and condition due to issue #40

Expected behavior all groups to be returned in the 'assigned_groups' field

Additional context Use case is to get all group assignments for an application and then convert the returned group ids to the group name for auditing/reporting.

I believe something similar to https://github.com/turbot/steampipe-plugin-okta/blob/4a2daf2a14a255206c744a666aa0e7bf41e37fd2/okta/table_okta_group.go#L183-L200 will need to be done around https://github.com/turbot/steampipe-plugin-okta/blob/4a2daf2a14a255206c744a666aa0e7bf41e37fd2/okta/table_okta_application.go#L207

but not sure (not too familiar with this code base still)

equiv functionality we are trying to reproductince in steampipe is

!/bin/bash
set -e
APPID='SOME_APP_ID'
OKTA_API_TOKEN='SOME_API_TOKEN'

function okta_api() {
  API="$1"
curl -s -X GET \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: SSWS ${OKTA_API_TOKEN}" \
"https://foo.okta.com/${API}"
}

#get application group assignments
okta_api "api/v1/apps/${APPID}/groups?limit=200" > blob.json

#convert groupids to names
for I in $(jq -r '.[].id' blob.json); do
  NAME=$(okta_api "api/v1/groups/${I}" | jq -r .profile.name | tr ' ' '_')
  jq -r '.[] | select(.id == "'${I}'") | .profile.samlRoles[]' blob.json > ${NAME}.txt
done 

[e-gineer]

Thanks @tinder-tder for letting us know about this. Definitely the column data should be complete.

But, in my opinion, this actually raises a bigger problem that these columns maybe should have been implemented as tables instead? Paging within the hydration for a single column seems like a performance risk / anti-pattern to me. I also imagine these columns could get very large in enterprise environments.

@c0d3r-arnab @rajlearner17 @cbruno10 Can / should we implement these new columns as tables instead?

[tinder-tder]

@e-gineer Thanks for the quick response I see the new table tickets. Let me know if you need anything else. We are really digging the product!

[cbruno10]

@tinder-tder We're closing this issue as this bug has been fixed with the addition of the okta_app_assigned_group table. Other issues related to that table, like missing profile information, are already being discussed in https://github.com/turbot/steampipe-plugin-okta/issues/57.