apache / fury

A blazingly fast multi-language serialization framework powered by JIT and zero-copy.
https://fury.apache.org/
Apache License 2.0
3.08k stars 246 forks source link

[Java] CPU Utilization 100% #1840

Closed eryanwcp closed 1 month ago

eryanwcp commented 1 month ago

Search before asking

Version

0.7.0

code

public class FurySerializer implements Serializer {

private final ThreadSafeFury fury;

public FurySerializer(){
    LoggerFactory.useSlf4jLogging(true);
    this.fury = Fury.builder().withLanguage(Language.JAVA)
            // Allow to deserialize objects unknown types, more flexible
            // but may be insecure if the classes contains malicious code.
            .withRefTracking(true)
            .requireClassRegistration(false)
            .buildThreadSafeFury();
}

@Override
public String name() {
    return "fury";
}

@Override
public byte[] serialize(Object obj) throws IOException {
    return fury.serialize(obj);
}

@Override
public Object deserialize(byte[] bytes) throws IOException {
    return fury.deserialize(bytes);
}

}

public class SerializationUtils {

private static Serializer g_serializer;

/**
 * 初始化序列化器
 * @param ser  serialization method
 * @param props serializer properties
 */
public static void init() {
    g_serializer = new FurySerializer();
}

 public static Object deserialize(byte[] bytes) throws IOException {
    if (bytes == null || bytes.length == 0)
        return null;
        return g_serializer.deserialize(bytes);
    }
}

出现大量线程CPU100%,异常:

at java.util.WeakHashMap.put(java.base@17.0.12/WeakHashMap.java:459) at org.apache.fury.ThreadLocalFury.lambda$new$2(ThreadLocalFury.java:60) at org.apache.fury.ThreadLocalFury$$Lambda$1294/0x00007f9fb0bcd7e8.get(Unknown Source) at java.lang.ThreadLocal$SuppliedThreadLocal.initialValue(java.base@17.0.12/ThreadLocal.java:305) at java.lang.ThreadLocal.setInitialValue(java.base@17.0.12/ThreadLocal.java:195) at java.lang.ThreadLocal.get(java.base@17.0.12/ThreadLocal.java:172) at org.apache.fury.ThreadLocalFury.deserialize(ThreadLocalFury.java:130) at com.eryansky.j2cache.util.FurySerializer.deserialize(FurySerializer.java:40)

Anything Else?

No response

Are you willing to submit a PR?

chaokunyang commented 1 month ago

@eryanwcp Do you create too much Fury instance? As the document cleared: image

chaokunyang commented 1 month ago

Could you use ThreadPoolFury instead? Seems you created too much threads, and every thread will create a new Fury instance

eryanwcp commented 1 month ago

@chaokunyang Only One Fury instance.

Spring Boot 3.3+dragonwell17

https://github.com/eryanwcp/ec/blob/3.2/j2cache-spring-boot-starter/src/main/java/com/eryansky/j2cache/cache/support/util/J2CacheSerializer.java

SerializationUtils.deserialize(bytes) Occasionally, CPU 100% occurs and persists

chaokunyang commented 1 month ago

private final ThreadSafeFury fury; should be private final static ThreadSafeFury fury; instead.

qinzhikui commented 2 weeks ago

`private final WeakHashMap<LoaderBinding, Object> allFury = new WeakHashMap();

public ThreadLocalFury(Function<ClassLoader, Fury> furyFactory) {
    this.bindingThreadLocal = ThreadLocal.withInitial(() -> {
        LoaderBinding binding = new LoaderBinding(furyFactory);
        binding.setBindingCallback(this.factoryCallback);
        binding.setClassLoader(Thread.currentThread().getContextClassLoader());
        this.allFury.put(binding, (Object)null);
        return binding;
    });
    Fury fury = ((LoaderBinding)this.bindingThreadLocal.get()).get();
    ClassResolver._addGraalvmClassRegistry(fury.getConfig().getConfigHash(), fury.getClassResolver());
}`

It seems that allFury. put() has caused a concurrency conflict

chaokunyang commented 2 weeks ago

Could you provide more details? ThreadLocalFury is created only once, I don't see concurrency conflict

qinzhikui commented 2 weeks ago

This is threadLocal, and each thread will create one. When multiple threads are created simultaneously, they will be put into allFury at the same time

chaokunyang commented 2 weeks ago

I see, thanks. This is indeed a bug, would you like to create a pr?

qinzhikui commented 2 weeks ago

sorry,I am not familiar with using PR. using Collections.synchronizedMap(new WeakHashMap()) or ConcurrentHashMap can solve this problem. sorry

chaokunyang commented 2 weeks ago

@qinzhikui thanks for your suggestion, I fixed it in https://github.com/apache/fury/pull/1889