Proposal to use gorm Adapter for RBAC Policy Management

[tken2039]

Summary

• Currently, RBAC Policy is set in ConfigMap based on the existing specification. However, there is a constraint on the ConfigMap capacity, limiting it to 1MiB, which poses a problem when managing a large number of Projects.
• Upon reading Argo CD's source code, it is evident that RBAC Policy management relies on the casbin package. Therefore, as a simple solution, I propose using the gorm adapter to manage RBAC Policy in a database.

Motivation

The Argo CD we manage contains over 10,000 lines of RBAC Policy. As per the current specifications, RBAC Policy is set in ConfigMap, but the 1MiB limit on ConfigMap capacity poses a risk if we continue with this approach.

Proposal

I consider the use of the gorm adapter as a straightforward solution to the problem.

• Addition of gorm Adapter
  • Policies will be retrieved from the database instead of ConfigMap.
  • The usage of Gorm Adapter will be configurable through startup options. (By default, ConfigMap will be used as before.)

I tested casbin/gorm-adapter on my local environment, and it seems that the implementation changes would be as simple as adding the Gorm Adapter configuration to rbac.go and making minor code adjustments in other parts.

However, one issue remains: event monitoring for the database. Currently, with the approach used for ConfigMap, the Argo CD can monitor ConfigMap using an Informer and load the Policy into memory when it gets updated. On the other hand, I lack knowledge on how to achieve the same effect with database events, so I'm not sure how to monitor the database effectively.

Regarding Policy loading, I am considering the following methods:

• When using the gorm adapter, load the Policy every time the Enforce function is called.
  • https://github.com/argoproj/argo-cd/blob/master/util/rbac/rbac.go#L239
• Utilize gorm's event hooks to trigger LoadPolicy using some method.

Conclusion

If there are any misunderstandings or if you have any better suggestions, please let me know.

Thank you.

[crenshaw-dev]

If the 1MB limit is the only constraint, how would you feel about splitting the RBAC across multiple ConfigMaps? There have been other requests for that enhancement.

[tken2039]

I recognize the approach of using multiple ConfigMaps as an effective temporary solution, as described in the following link: https://github.com/argoproj/argo-cd/pull/11445.

On the other hand, I understand that the need to prepare multiple ConfigMaps may go against the philosophy of Kubernetes ConfigMaps, especially in the context of large-scale Argo CD operations. It could be seen as data that contradicts the purpose of ConfigMaps in Kubernetes. While I'm not certain whether storing the data in a database is the correct answer, I propose this as a simple resolution.

In any case, I hope to address the capacity-related problem concerning policies either through the aforementioned PR or this proposal. If necessary, I can also adjust the content of #11445 and create a new Pull Request. Thank you for considering this matter.

[crenshaw-dev]

the need to prepare multiple ConfigMaps may go against the philosophy of Kubernetes ConfigMaps

That's a fair concern. I'd be interested in understanding why that is the philosophy and how a diversion from that philosophy might negatively impact users. With those reasons enumerated, we can weigh pros and cons.

If necessary, I can also adjust the content of https://github.com/argoproj/argo-cd/pull/11445

I do think that's a valuable PR and could be useful for reasons beyond scalability... with 312 open PRs, it's just been a victim of insufficient reviewer time. :-)

[tken2039]

Thank you for explaining the situation.

I appreciate your efforts in keeping Argo CD up to date, and I am grateful to the reviewers! I will patiently await updates on that PR.

Regarding Kubernetes' underlying philosophy about ConfigMaps, I wasn't aware of it, but the official documentation provides the following explanation:

A ConfigMap is not designed to hold large chunks of data. The data stored in a ConfigMap cannot exceed 1 MiB. If you need to store settings that are larger than this limit, you may want to consider mounting a volume or use a separate database or file service.

https://kubernetes.io/docs/concepts/configuration/configmap/#motivation

I can see some potential issues with managing multiple ConfigMaps due to the capacity limitations:

• Even though they serve the same purpose, there is a need to split them into multiple ConfigMaps, which could increase operational overhead not only in Kubernetes but also in other contexts.
• The monitoring and communication overhead would also increase as ConfigMaps are monitored through Informers to fetch the latest information. Having more ConfigMaps would amplify these aspects.

However, I agree that having two or three ConfigMaps won't likely cause significant problems. Therefore, #11445 seems to be a valid solution.

On the other hand, considering the long term, Argo CD is a product that depends on Kubernetes, and it feels natural to adhere to its underlying philosophy. Moreover, aiming to avoid complexity in management, enabling the utilization of large-scale storage appears to be essential.

What are your thoughts on this?