WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.

[onegoldfish]

What happened + What you expected to happen

I use ray in java. And I used JEP to call python in the remote method, but I got: WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.

There is no problem with the current calculation results. Could you please tell me What are the hidden dangers?

I read the part of Java calling python in ray. It seems that there are some limitations, so we want to use JEP to implement Java calling python in ray. Will the above warnings affect the use?

Versions / Dependencies

jdk11 ubuntu20.0.4 maven ray2.0.0 python3.9.12

Reproduction script

public class RayDemo {
    public static int square(int x) {
        JepConfig config = new JepConfig();
        config.addIncludePaths("/usr/local/mytest");
        config.addIncludePaths(path);
        SharedInterpreter.setConfig(config);
        SharedInterpreter interp = new SharedInterpreter();
        try{
            interp.eval("from pydemo import *");
            String  res = interp.invoke("square",x).toString();
            return Integer.valueOf(res);
        }finally {

        }
    }

    public static class Counter {

        private int value = 0;

        public void increment() {
            SharedInterpreter interp = new SharedInterpreter();
            try {
                interp.eval("from pydemo import *");//python文件名 invoke
                String  res = interp.invoke("add",this.value).toString();
                this.value= Integer.valueOf(res);
            }finally {

            }

        }

        public int read() {
            return this.value;
        }
    }

    public static void main(String[] args) {
        // Intialize Ray runtime.
        RayRuntimeFactory factory=new RayRuntimeFactory() {
            @Override
            public RayRuntime createRayRuntime() {
                return null;
            }
        };
        Ray.init();
        {
            List<ObjectRef<Integer>> objectRefList = new ArrayList<>();
            // Invoke the `square` method 4 times remotely as Ray tasks.
            // The tasks will run in parallel in the background.
            for (int i = 0; i < 3; i++) {
                objectRefList.add(Ray.task(RayDemo::square, i).remote());
            }
            // Get the actual results of the tasks with `get`.
            System.out.println(Ray.get(objectRefList)); // [0, 1, 4, 9]
        }

        {
            List<ActorHandle<Counter>> counters = new ArrayList<>();
            // Create 4 actors from the `Counter` class.
            // They will run in remote worker processes.
            for (int i = 0; i < 6; i++) {
                counters.add(Ray.actor(Counter::new).remote());
            }

            // Invoke the `increment` method on each actor.
            // This will send an actor task to each remote actor.
            for (ActorHandle<Counter> counter : counters) {
                counter.task(Counter::increment).remote();
            }
            // Invoke the `read` method on each actor, and print the results.
            List<ObjectRef<Integer>> objectRefList =
                    counters.stream()
                            .map(counter -> counter.task(Counter::read).remote())
                            .collect(Collectors.toList());
            System.out.println(Ray.get(objectRefList)); // [1, 1, 1, 1]
        }

        {
            try (Interpreter interp = new SharedInterpreter()) {
                interp.eval("import numpy");
                interp.eval("import sys");
                interp.exec("from java.lang import System");//jep执行的python语句中可以引入java包
                interp.exec("sys.s = 'Hello World'");
                interp.exec("print(sys.s)");
                //interp.exec("System.out.println(s)");
                //interp.exec("print(s[1:-1])");
            }
        }

        {
            try (Interpreter interp = new SharedInterpreter()) {
                interp.eval("import numpy");
                interp.eval("import sys");
                interp.exec("from java.lang import System");//jep执行的python语句中可以引入java包
                //interp.exec("s = 'Hello World'");
                interp.exec("print(sys.s)");
            }
        }

    }

}

pydemo .py

def add(a):
    return a + 1

def square(x):
    return x * x

Issue Severity

No response

[fishbone]

@wumuzi520 could you take a look of this issue?

[wumuzi520]

@wumuzi520 could you take a look of this issue?

Okey, let me have a look. @jovany-wang Please take a look too.

[jovany-wang]

Does it happen in JDK11 without JEP? I'm not sure if it's related to JDK11 or JEP. If it's related to JDK11, I think we should fix it, otherwise, we don't have the bandwidth to port Ray cross-languages invocation on JEP runtime.

[jovany-wang]

I read the part of Java calling python in ray. It seems that there are some limitations

BTW, could you tell what the limits you met?

[onegoldfish]

Does it happen in JDK11 without JEP?

No, it doesn't happen without JEP. The author of JEP said that JEP didn't use the sun.reflect.Reflection.getCallerClass, I don't know where this warning came from. If you think this problem is not ray related, please tell me.

As for the Cross-language in ray, I just see that there are not many serializable types in the document, which may not be convenient for use. There is no problem at present

[jovany-wang]

@onegoldfish So could you give a reproduction script without JEP ?

[jovany-wang]

Rich types in Ray serialization will be supported in this REP https://github.com/ray-project/enhancements/pull/6

If your issue is not urgent, maybe it's better to use Ray native cross language invocation.

[onegoldfish]

I just use demo in https://docs.ray.io/en/master/ray-overview/index.html#ray-core-quick-start, the warning doesn't happen. Thank you for your reply. If Ray native cross language can meet the demand, I will give priority to it. Thank you and your team for the optimization

[jovany-wang]

@onegoldfish It's ok. So I'm closing this issue, and feel free to file a new issue if you have any other problem.

[onegoldfish]

@jovany-wang When I use pure Java without JEP, the warning happened again.

package org.example;

import io.ray.api.ObjectRef;
import io.ray.api.Ray;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.util.LinkedList;
import java.util.List;
import java.util.Random;

import static java.lang.Thread.sleep;

public class JavaRayTask {
    public static final Logger log= LoggerFactory.getLogger("JavaTaskO");

    public static Integer square(Integer x) throws InterruptedException {
        log.info("JavaTask quare start ,input is $x, will return ${x*x}");
        sleep(500);
        return x*x;}

    public static Integer sprint()
    {
        log.info("JavaTask sprint start ,no input");
        Integer result =new Random().nextInt();
        log.info("JavaTask sprint end ,return: $result");
        return result;
    }

    public static void main (String[] args){
        Ray.init();

        Integer count= Integer.valueOf(args[0]);
        List<ObjectRef<Integer>> results = new LinkedList<>();
//    val results =mutableListOf<Int>()
        long start =System.currentTimeMillis();
        for (int i=0; i<count;i++ ){
            log.info("--- {} times loop begin start------",i);
            ObjectRef<Integer> step = Ray.task(JavaRayTask::sprint).remote();
            log.info("step 1 return：{}",step);
            ObjectRef<Integer> result= Ray.task(JavaRayTask::square,i).remote();
            log.info ("---{} times loop end , the result is {}------",i, result);
            results.add(step);
            results.add(result);

        }
        log.info("---print result List: {}------", Ray.get(results));
        long end= System.currentTimeMillis();
        log.info("cost time: end-start={}", end-start);

    }

}

the outputs are as follow:

java -cp testRayJavaNoJEP.jar -Dray.address=10.50.219.20:6379 -Dray.job.code-search-path=/home/ccc org.example.JavaRayTask 5
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
[2022-11-25 18:44:10,069 I 261465 261466] logging.cc:230: Set ray log level from environment variable RAY_BACKEND_LOG_LEVEL to -1
2022-11-25 18:44:10,081 INFO JavaTaskO [main]: --- 0 times loop begin start------
2022-11-25 18:44:10,123 INFO JavaTaskO [main]: step 1 return：ObjectRef(52803897c098a351ffffffffffffffffffffffff2200000001000000)
2022-11-25 18:44:10,134 INFO JavaTaskO [main]: ---0 times loop end , the result is ObjectRef(34254bb0ae9f7689ffffffffffffffffffffffff2200000001000000)------
2022-11-25 18:44:10,135 INFO JavaTaskO [main]: --- 1 times loop begin start------
2022-11-25 18:44:10,135 INFO JavaTaskO [main]: step 1 return：ObjectRef(8198ec92f705973effffffffffffffffffffffff2200000001000000)
2022-11-25 18:44:10,141 INFO JavaTaskO [main]: ---1 times loop end , the result is ObjectRef(ace36dceef5a1a37ffffffffffffffffffffffff2200000001000000)------
2022-11-25 18:44:10,142 INFO JavaTaskO [main]: --- 2 times loop begin start------
2022-11-25 18:44:10,142 INFO JavaTaskO [main]: step 1 return：ObjectRef(8c1e38b14aa4e2f0ffffffffffffffffffffffff2200000001000000)
2022-11-25 18:44:10,142 INFO JavaTaskO [main]: ---2 times loop end , the result is ObjectRef(cd131a74fdaaa915ffffffffffffffffffffffff2200000001000000)------
2022-11-25 18:44:10,142 INFO JavaTaskO [main]: --- 3 times loop begin start------
2022-11-25 18:44:10,142 INFO JavaTaskO [main]: step 1 return：ObjectRef(5a1e859d9151a1d8ffffffffffffffffffffffff2200000001000000)
2022-11-25 18:44:10,143 INFO JavaTaskO [main]: ---3 times loop end , the result is ObjectRef(67e35ffb2642a211ffffffffffffffffffffffff2200000001000000)------
2022-11-25 18:44:10,143 INFO JavaTaskO [main]: --- 4 times loop begin start------
2022-11-25 18:44:10,143 INFO JavaTaskO [main]: step 1 return：ObjectRef(e7332f86bd7b3f5fffffffffffffffffffffffff2200000001000000)
2022-11-25 18:44:10,143 INFO JavaTaskO [main]: ---4 times loop end , the result is ObjectRef(7ad5a28136a3eab7ffffffffffffffffffffffff2200000001000000)------
2022-11-25 18:44:16,048 INFO JavaTaskO [main]: ---print result List: [819406792, 0, 1421224626, 1, 514415638, 4, -1154365701, 9, 243385814, 16]------
2022-11-25 18:44:16,048 INFO JavaTaskO [main]: cost time: end-start=5970
root@monitor-virtual-machine:/home/ciccinfra#

Does this mean that the problem is related to jdk11?

[jovany-wang]

I'll take a look.

[onegoldfish]

@jovany-wang We find that the warning may be caused by log4j2 in jdk11, could you please take a look？

[jovany-wang]

Yes. It's introduced in log4j2. See https://issues.apache.org/jira/browse/LOG4J2-2537

We should use multiple jars. I'll investigate how we use multiple jars in bazel...

[rwanderc]

I'm still trying to understand, but this error appeared to me when I introduced log4j in the classpath. If I remove it, the error doesn't appear. I'm using JDK 20 (zulu).

        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-core</artifactId>
            <version>2.20.0</version>
        </dependency>

[ron190]

Add Multi-Release: true to maven-assembly-plugin:

<configuration>
    <archive>
        <manifestEntries>
            <!-- Remove 'WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.' -->
            <Multi-Release>true</Multi-Release>
        </manifestEntries>
    </archive>
</configuration>