search

CoolProp / CoolProp

Thermophysical properties for the masses
http://coolprop.org/
MIT License
773 stars 307 forks source link

Java Wrapper hangs in multi threaded environment #1859

Open mar1u50 opened 4 years ago

mar1u50 commented 4 years ago

Description

When trying to run multiple CoolProp calculations in parallel (from java using threads), one thread works, second one shows:

Unable to load fluid [R125] due to error: Cannot load fluid [R125:354-33-6] because it is already in library; index = [5] of [6]; Consider enabling the config boolean variable OVERWRITE_FLUIDS Exception in thread "Thread-1" java.lang.RuntimeException: Initialize failed for backend: "?", fluid: "nitrogen" fractions "[ 1.0000000000 ]"; error: key [nitrogen] was not found in string_to_index_map in JSONFluidLibrary at CoolPropJNI.PropsSI(Native Method) at CoolProp.PropsSI(CoolProp.java:100) at TestCoolProp.specificEnthalpyPT(TestCoolProp.java:41) at TestCoolProp.calc10000(TestCoolProp.java:33) at TestCoolProp.lambda$main$0(TestCoolProp.java:16) at java.base/java.lang.Thread.run(Thread.java:834)

Basically my code is not doing too much: just spawned 2 threads and doing 10000 calculations on each thread.

Steps to Reproduce

Run the code: public class TestCoolProp { static { System.loadLibrary("CoolProp"); }

public static void main(String[] args) throws InterruptedException {
    long startTime = System.currentTimeMillis();

    List<Thread> threads = IntStream.range(0, 2)
            .mapToObj(i -> new Thread(() -> calc10000()))
            .collect(Collectors.toList());

    threads.forEach(Thread::start);

    for (Thread thread : threads) {
        thread.join();
    }

    System.out.println("XXXX:" + (System.currentTimeMillis() - startTime));
}

private static void calc10000() {
    long startTime = System.currentTimeMillis();

    double[] enthList = new double[10000];
    for (int i = 0; i < 10000; i++) {
        double enthalpy = specificEnthalpyPT(1 + i / 1000, 274 + i / 100);
        enthList[i] = enthalpy;
    }

    System.out.println(System.currentTimeMillis() - startTime);
}

private static double specificEnthalpyPT(double p, double t) {
    return CoolProp.PropsSI("H", "P", p, "T", t, "nitrogen");
}

}

Expected behavior: Works

Actual behavior: Fails

Versions

CoolProp Version: 6.3.0 Operating System and Version: MacOS Access Method: Java Wrapper

ibell commented 4 years ago

I'm not sure if this would ever be expected to work. I'm not sure how swig handles multi threaded calls, and I suspect some of their code is not threadsafe. On the other hand, CoolProp itself is threadsafe, so doing a similar exercise from C++ or Python should work. I know for sure that works in c++

On Thu, Oct 10, 2019, 11:30 AM mar1u50 notifications@github.com wrote:

Description

When trying to run multiple CoolProp calculations in parallel (from java using threads), one thread works, second one shows:

Unable to load fluid [R125] due to error: Cannot load fluid [R125:354-33-6] because it is already in library; index = [5] of [6]; Consider enabling the config boolean variable OVERWRITE_FLUIDS Exception in thread "Thread-1" java.lang.RuntimeException: Initialize failed for backend: "?", fluid: "nitrogen" fractions "[ 1.0000000000 ]"; error: key [nitrogen] was not found in string_to_index_map in JSONFluidLibrary at CoolPropJNI.PropsSI(Native Method) at CoolProp.PropsSI(CoolProp.java:100) at TestCoolProp.specificEnthalpyPT(TestCoolProp.java:41) at TestCoolProp.calc10000(TestCoolProp.java:33) at TestCoolProp.lambda$main$0(TestCoolProp.java:16) at java.base/java.lang.Thread.run(Thread.java:834)

Basically my code is not doing too much: just spawned 2 threads and doing 10000 calculations on each thread. Steps to Reproduce

Run the code: public class TestCoolProp { static { System.loadLibrary("CoolProp"); }

public static void main(String[] args) throws InterruptedException { long startTime = System.currentTimeMillis();

List<Thread> threads = IntStream.range(0, 2)
        .mapToObj(i -> new Thread(() -> calc10000()))
        .collect(Collectors.toList());

threads.forEach(Thread::start);

for (Thread thread : threads) {
    thread.join();
}

System.out.println("XXXX:" + (System.currentTimeMillis() - startTime));

}

private static void calc10000() { long startTime = System.currentTimeMillis();

double[] enthList = new double[10000];
for (int i = 0; i < 10000; i++) {
    double enthalpy = specificEnthalpyPT(1 + i / 1000, 274 + i / 100);
    enthList[i] = enthalpy;
}

System.out.println(System.currentTimeMillis() - startTime);

}

private static double specificEnthalpyPT(double p, double t) { return CoolProp.PropsSI("H", "P", p, "T", t, "nitrogen"); }

}

Expected behavior: Works

Actual behavior: Fails Versions

CoolProp Version: 6.3.0 Operating System and Version: MacOS Access Method: Java Wrapper

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/CoolProp/CoolProp/issues/1859?email_source=notifications&email_token=AAOGC24ENLJ6GWLUQEZFWDDQN5RB3A5CNFSM4I7P4YJKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HQ7554Q, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOGC23GTEWITUQEWBHQJHTQN5RB3ANCNFSM4I7P4YJA .

mibogi commented 4 years ago

Hi To all, from this link:http://www.swig.org/Doc2.0/SWIGDocumentation.html

Note that some of the STL in Visual C++ 6 is not thread safe, so although code might be linked to the multithread runtime libraries, undefined behaviour might still occur in a single threaded Java program. Similarly some older versions of Sun Studio have bugs in the multi-threaded implementation of the std::string class and so will lead to undefined behaviour in these supposedly single threaded Java applications.

Basically it seams that the issue is with SWIG. Will continue checking it

ibell commented 4 years ago

Note: That's very old documentation, for version 2 of SWIG. Google usually finds the old docs, so make sure you are reading the new ones.

mibogi commented 4 years ago

Hi mar1u50, to me this doesnt look like an multithreading problem. Firstly, you are calling join method that means that the thread needs to wait for previous to finish its execution no matter how long it takes. Secondly, as you are calling :

private static double specificEnthalpyPT(double p, double t) { return CoolProp.PropsSI("H", "P", p, "T", t, "nitrogen"); }

where PropsSI is still an static Method but if we are calling an method that contains only local attributes, there would be no issues with Multi-threading as such methods are thread safe.

Are you always able to reproduce?
BR MiBogi

mibogi commented 4 years ago

Hi to all, i got following issue:

java: malloc.c:2401: sysmalloc: Assertion(old_top == initial_top (av) && old_size == 0) || ((unsigned long) (old_size) >= MINSIZE && prev_inuse (old_top) && ((unsigned long) old_end & (pagesize - 1)) == 0)' failed. Aborted (core dumped) and after that #

A fatal error has been detected by the Java Runtime Environment:

#

SIGSEGV (0xb) at pc=0x00007f6d393e715a, pid=17073, tid=17092

#

JRE version: OpenJDK Runtime Environment (11.0.4+11) (build 11.0.4+11-post-Ubuntu-1ubuntu218.04.3)

Java VM: OpenJDK 64-Bit Server VM (11.0.4+11-post-Ubuntu-1ubuntu218.04.3, mixed mode, sharing, tiered, compressed oops, g1 gc, linux-amd64)

Problematic frame:

C [libstdc++.so.6+0xaa15a] std::_Rb_tree_insert_and_rebalance(bool, std::_Rb_tree_node_base, std::_Rb_tree_node_base, std::_Rb_tree_node_base&)+0xca

#

Core dump will be written. Default location: Core dumps may be processed with "/usr/share/apport/apport %p %s %c %d %P" (or dumping to /media/xadfasdasfasda/LocalPartition/Projects Programming/Refrigeration/adasghdfhdfgdf/core.17073)

#

If you would like to submit a bug report, please visit:

https://bugs.launchpad.net/ubuntu/+source/openjdk-lts

The crash happened outside the Java Virtual Machine in native code.

See problematic frame for where to report the bug.

#

--------------- S U M M A R Y ------------

Command Line: TestCoolProp

Host: `

my code is:

`public class ConcurentCoolProp extends Thread{ static { System.load("/media/milosubuntu/LocalPartition/Projects Programming/Refrigeration/platform-independent(2)/libCoolProp.so"); System.out.println("loaded"); }

@Override
public void run(){

calc10000();

}

private  double specificEnthalpyPT(double p, double t) {
 return CoolProp.PropsSI("H", "P", p, "T", t, "nitrogen");
}

private  void calc10000() {
     long startTime = System.currentTimeMillis();

     double[] enthList = new double[10000];
     for (int i = 0; i < 10000; i++) {
            double enthalpy = specificEnthalpyPT(1 + i / 1000, 274 + i / 100);
            enthList[i] = enthalpy;
        }

        System.out.println(System.currentTimeMillis() - startTime);
}

}

and import static java.util.stream.Collectors.; import java.util.;

public class TestCoolProp { static { System.load("/media/milosubuntu/LocalPartition/Projects Programming/Refrigeration/platform-independent(2)/libCoolProp.so"); System.out.println("loaded"); }

public static void main(String[] args) throws InterruptedException { ConcurentCoolProp cp1=new ConcurentCoolProp(); ConcurentCoolProp cp2=new ConcurentCoolProp(); ConcurentCoolProp cp3=new ConcurentCoolProp(); ConcurentCoolProp cp4=new ConcurentCoolProp(); ConcurentCoolProp cp5=new ConcurentCoolProp(); cp1.start(); cp2.start(); cp3.start(); cp4.start(); cp5.start();

}

}

Well, it seams that it is outside of the Java wrapper, somewhere in C++ code. I will try to rewrite the Complete CoolPropJNI.java file.

However, it would be helpfull if somebody could check out the issue casued in native code BR miBogi

update1: after deleting lines cp2.start(); cp3.start(); cp4.start(); cp5.start();

it worked but that means only one thread did not cause an issue above. What may be interesting is to see what would be an behavior when i synchronize access to CoolProps.

mibogi commented 4 years ago

Workaround

`public class ThreadCoolProp extends Thread{ private ConcurentCoolProp coolProp;

public ThreadCoolProp(ConcurentCoolProp coolProp){
this.coolProp=coolProp; 

}

@Override
public void run(){
synchronized(coolProp){
calc10000();
}

}

private  double specificEnthalpyPT(double p, double t) {
 return CoolProp.PropsSI("H", "P", p, "T", t, "nitrogen");
}

private  void calc10000() {
     long startTime = System.currentTimeMillis();

     double[] enthList = new double[10000];
     for (int i = 0; i < 10000; i++) {
            double enthalpy = specificEnthalpyPT(1 + i / 1000, 274 + i / 100);

            enthList[i] = enthalpy;
        }
    System.out.println(this.getName());

}

}

`public class TestSyncCoolProp {

public static void main(String args[]){
ConcurentCoolProp coolProp=new ConcurentCoolProp();
ThreadCoolProp t1=new ThreadCoolProp(coolProp);
ThreadCoolProp t2=new ThreadCoolProp(coolProp);
ThreadCoolProp t3=new ThreadCoolProp(coolProp);
ThreadCoolProp t4=new ThreadCoolProp(coolProp);
ThreadCoolProp t5=new ThreadCoolProp(coolProp);
ThreadCoolProp t6=new ThreadCoolProp(coolProp);
ThreadCoolProp t7=new ThreadCoolProp(coolProp);
t1.start();
t2.start();
t3.start();
t4.start();
t5.start();
t6.start();
t7.start();

}

}

public class ConcurentCoolProp{ static { System.load("/media/milosubuntu/LocalPartition/Projects Programming/Refrigeration/platform-independent(2)/libCoolProp.so"); System.out.println("loaded"); }

private  double specificEnthalpyPT(double p, double t) {
 return CoolProp.PropsSI("H", "P", p, "T", t, "nitrogen");
}

private  void calc10000() {
     long startTime = System.currentTimeMillis();

     double[] enthList = new double[10000];
     for (int i = 0; i < 10000; i++) {
            double enthalpy = specificEnthalpyPT(1 + i / 1000, 274 + i / 100);
    System.out.println(enthalpy = specificEnthalpyPT(1 + i / 1000, 274 + i / 100));
            enthList[i] = enthalpy;
        }

        System.out.println(System.currentTimeMillis() - startTime);
}

}

However, this really restricts the Concurency where only one thread at the time uses instance of the ConcurentCoolProp.class