This patch enables support for multiple Java threads running different instances of the TornadoVM execution plans.
The core runtime systems needed to be changed due to not thread-safe when it comes to managing command queues (or CUDA Streams, or Level Zero Command Queues and Command Lists).
The key part is that there is at least one command queue associated to an execution plan. A execution plan can have more than 1 Java threads associated, if the concurrent-devices or dynamic reconfiguration is also enabled.
Command Queues are now managed by a double HashMap:
Thread t0;
Thread t1;
final int size = 1024 * 1024 * 32;
FloatArray input = new FloatArray(size);
input.init(1.0f);
FloatArray output = new FloatArray(size);
TaskGraph taskGraph = new TaskGraph("check") //
.transferToDevice(DataTransferMode.EVERY_EXECUTION, input) //
.task("compute01", MultiThreaded::computeForThread2, input, output) //
.transferToHost(DataTransferMode.EVERY_EXECUTION, output);
// Build two execution plans from two different Java threads that share the same Task-Graph
t0 = new Thread(() -> {
ImmutableTaskGraph immutableTaskGraph = taskGraph.snapshot();
TornadoExecutionPlan executionPlan = new TornadoExecutionPlan(immutableTaskGraph);
executionPlan.execute();
});
t1 = new Thread(() -> {
ImmutableTaskGraph immutableTaskGraph = taskGraph.snapshot();
TornadoExecutionPlan executionPlan = new TornadoExecutionPlan(immutableTaskGraph);
executionPlan.execute();
});
t0.start();
t1.start();
t0.join();
t1.join();
Backend/s tested
Mark the backends affected by this PR.
[X] OpenCL
[X] PTX
[X] SPIRV
OS tested
Mark the OS where this PR is tested.
[X] Linux
[ ] OSx
[ ] Windows
Did you check on FPGAs?
If it is applicable, check your changes on FPGAs.
[ ] Yes
[X] No
How to test the new patch?
## Pass Unittests using the OpenCL backend
$ make BACKEND=opencl
$ make tests
## Pass unittests using the PTX backend
$ make BACKEND=ptx
$ make tests
## Pass unittests using the SPIRV backend
$ make BACKEND=spirv
$ make tests
Description
This patch enables support for multiple Java threads running different instances of the TornadoVM execution plans. The core runtime systems needed to be changed due to not thread-safe when it comes to managing command queues (or CUDA Streams, or Level Zero Command Queues and Command Lists).
The key part is that there is at least one command queue associated to an execution plan. A execution plan can have more than 1 Java threads associated, if the concurrent-devices or dynamic reconfiguration is also enabled.
Command Queues are now managed by a double HashMap:
The following use case is now allowed:
Backend/s tested
Mark the backends affected by this PR.
OS tested
Mark the OS where this PR is tested.
Did you check on FPGAs?
If it is applicable, check your changes on FPGAs.
How to test the new patch?
Tests for Multi-threaded:
All supported tests are passing. Pending for testing: A) KFusion B) RayTracer