permission cache should try to adjust itself if there are lots of records

[tarzanek]

In scope of https://github.com/scylladb/scylla-docs/issues/3169

the permission cache defaults are finite and when the permissions records are more than the size of cache (1000 records) the nodes start to do lots of non-paged queries

Also auth queries should be paged to avoid false alerts from cql monitoring.

scylla should try to check for this situation and if enough memory is there increase the cache

Number of permissions can be found by: SELECT count(*) FROM system_auth.role_permissions;

https://github.com/scylladb/scylla-docs/issues/3169 tracks documenting and how to manually do this

This bug should try to address the auto adjust of Scylla, so it will be a boring database :-)

[slivne]

Lets start with a warning to make this noticed for users - readjusting without user knowing about it is not perfect - there are at times other solutions (e.g. some of the permission are redundant and maybe removed)

[kostja]

@nuivall should be addressed with the new caching implementation for permissions; essentially a duplicate of the issue you already have in your backlog for auth-on-raft follow-ups, but I am assigning this one since it's an indication that the current implementation has been leading to issues in the field, and a design we're about to implement was suggested earlier. Please close along with moving auth cache to push from pull design and implementing a full cache.

[nyh]

@mykaul I think you misplaced your last two comments here, you wanted to put them in https://github.com/scylladb/scylladb/issues/17788. I'll copy it over for you.

[mykaul]

@mykaul I think you misplaced your last two comments here, you wanted to put them in #17788. I'll copy it over for you.

whoops-a-daisy.

Thanks.