data-dot-all / dataall

A modern data marketplace that makes collaboration among diverse users (like business, analysts and engineers) easier, increasing efficiency and agility in data projects on AWS.
https://data-dot-all.github.io/dataall/
Apache License 2.0
236 stars 82 forks source link

Optimize splitting policies into chunks #1684

Open TejasRGitHub opened 3 weeks ago

TejasRGitHub commented 3 weeks ago

Is your idea related to a problem? Please describe. With this feature enhancement, requestors IAM policy is split into multiple policies and then attached to the role. This solves the problem of limiting size of policy document ( 6144 bytes ).

But the splitting made with this approach is not optimal and can lead to suboptimal splitting and use of extra policies.

For example, It might happen that the statements are split into following chunks - [4000, 500, 2000]. Here, the policy documents are created clubbing the following - [{4000, 500}, {2000}]. But optimally, it should be [{4000, 2000}, {500}] to utilize the maximum size of policy

Here, sorting policies by size in descending order also doesn't work. Support splitted policy statements are - [1000, 5000, 144, 6000], then sorted policy chunks - [6000, 5000, 1000, 144] will create 3 policy documents of [{6000}, {5000, 1000}, {144}] whereas the optimal policy documents are - [{6000, 144}, {5000, 1000}].

Describe the solution you'd like Create a dynamic solution which will optimize for the space available in the policy document.

Effectively solve this problem - for statements = [s1, s2, s3, .... ] and their sizes ( weights ) = [size0, size1, size2, ... ]. Put statements in policies ( service-policy-0, service-policy-1, service-policy-2 ... ) such that Min(Empty Space left ).

P.S. Don't attach files. Please, prefer add code snippets directly in the message body.