Memory consumption during enforce method

[Danneborg]

We are using enforce( 1.22.6 version jcabin) method in our system and observing strange memory consumption. We upload around 3200 politics when the app is starting. When I call enforce method I see, that my app consumes around 700mb of heap memory, which I think quite strange. Also, sometimes my app is falling with OOM, we have 2 pods and each consumes more than 10GB of memory! I used many heap dump analyzers and found that the most suspicious object belongs to com.googlecode.aviator.utils.Env library, it takes from 30% to 70% of my heap! Does any one have any suggestion why it happens and what I can to do with it?

[casbin-bot]

@tangyang9464 @imp2002

[hsluoyz]

@Danneborg does other version have this issue? If yes, then it's Java issue. You write Java you know it, Java takes a lot of memory. I don't see abnormal here

If you want to report a bug, say it clearly

[Danneborg]

does other version have this issue

I counted some data from my prod environment, we call enforce around 80-90 times per second, here is the picture of memory consumption. I can't change lib version in prod quickly( I'm curious about 3-4 gigabytes jumps, is it okay?

But I can do it locally. Local test gives the same picture, the same jumps when I call enforce. I just want to know, is it okay that jcasbin consumes that much memory for its performance?

[hsluoyz]

@Danneborg jCasbin uses aviator (https://github.com/killme2008/aviatorscript/blob/master/README-EN.md) for enforcement. You can test that lib directly to see its memory usage. This will help locate the root cause.

[Danneborg]

So, I did some performance test for aviator.compile and expression.execute and I can conclude that this two methods consume a lot fo memory, that sad. Also I found one suspicious place in jcabin. The digit first looks like the library always iterates the same list of policies without any cache or attempts to reduce number of cycles and inside near digit 2 the code always creates new HashMap that allocates enormous number of object in heap. Do you have any plans to reduce number of cycles or loop peeling?

[liewstar]

So, I did some performance test for aviator.compile and expression.execute and I can conclude that this two methods consume a lot fo memory, that sad. Also I found one suspicious place in jcabin. The digit first looks like the library always iterates the same list of policies without any cache or attempts to reduce number of cycles and inside near digit 2 the code always creates new HashMap that allocates enormous number of object in heap. Do you have any plans to reduce number of cycles or loop peeling?

The code you mentioned has been improved in the latest version

your version:1.22.6

for (int i = 0; i < model.model.get("p").get(pType).policy.size(); i++) {
    List<String> pvals = model.model.get("p").get(pType).policy.get(i);
    // Util.logPrint("Policy Rule: " + pvals);
    // Select the rule based on request size
    Map<String, Object> parameters = new HashMap<>();
    getRTokens(parameters, rvals);
    for (int j = 0; j < model.model.get("p").get(pType).tokens.length; j++) {
        String token = model.model.get("p").get(pType).tokens[j];
        parameters.put(token, pvals.get(j));
    }
    Object result = expression.execute(parameters);

latest:

final List<List<String>> policy = model.model.get("p").get(pType).policy;
final String[] pTokens = model.model.get("p").get(pType).tokens;
final int policyLen = policy.size();
int explainIndex = -1;
if (policyLen != 0 && expString.contains(pType+"_")) {
    policyEffects = new Effect[policyLen];
    matcherResults = new float[policyLen];
    for (int i = 0; i < policy.size(); i++) {
        List<String> pvals = policy.get(i);
        Map<String, Object> parameters = new HashMap<>(rvals.length + pTokens.length);
        getPTokens(parameters, pType, pvals, pTokens);
        getRTokens(parameters, rType, rvals);
        Object result = expression.execute(parameters);

[Danneborg]

Thanks for your answers, this tread can be closed