Support for Multi-Threaded Execution Plans

[jjfumero]

Description

This patch enables support for multiple Java threads running different instances of the TornadoVM execution plans. The core runtime systems needed to be changed due to not thread-safe when it comes to managing command queues (or CUDA Streams, or Level Zero Command Queues and Command Lists).

The key part is that there is at least one command queue associated to an execution plan. A execution plan can have more than 1 Java threads associated, if the concurrent-devices or dynamic reconfiguration is also enabled.

Command Queues are now managed by a double HashMap:

executionPlanId -> Hash<Device, CommandQueTable> -> Hash<ThreadId, CommandQueue> 

The following use case is now allowed:

Thread t0;
Thread t1;

final int size = 1024 * 1024 * 32;
FloatArray input = new FloatArray(size);
input.init(1.0f);
FloatArray output = new FloatArray(size);

TaskGraph taskGraph = new TaskGraph("check") //
       .transferToDevice(DataTransferMode.EVERY_EXECUTION, input) //
       .task("compute01", MultiThreaded::computeForThread2, input, output) //
       .transferToHost(DataTransferMode.EVERY_EXECUTION, output);

// Build two execution plans from two different Java threads that share the same Task-Graph
t0 = new Thread(() -> {
        ImmutableTaskGraph immutableTaskGraph = taskGraph.snapshot();
        TornadoExecutionPlan executionPlan = new TornadoExecutionPlan(immutableTaskGraph);
        executionPlan.execute();
});

t1 = new Thread(() -> {
       ImmutableTaskGraph immutableTaskGraph = taskGraph.snapshot();
       TornadoExecutionPlan executionPlan = new TornadoExecutionPlan(immutableTaskGraph);
       executionPlan.execute();
});

t0.start();
t1.start();

t0.join();
t1.join();

Backend/s tested

Mark the backends affected by this PR.

• [X] OpenCL
• [X] PTX
• [X] SPIRV

OS tested

Mark the OS where this PR is tested.

• [X] Linux
• [ ] OSx
• [ ] Windows

Did you check on FPGAs?

If it is applicable, check your changes on FPGAs.

• [ ] Yes
• [X] No

How to test the new patch?

## Pass Unittests using the OpenCL backend
$ make BACKEND=opencl
$ make tests

## Pass unittests using the PTX backend
$ make BACKEND=ptx
$ make tests 

## Pass unittests using the SPIRV backend
$ make BACKEND=spirv
$ make tests 

Tests for Multi-threaded:

$ make 
$ tornado-test --threadInfo -V --fast uk.ac.manchester.tornado.unittests.multithreaded.MultiThreaded#test01
$ tornado-test --threadInfo -V --fast uk.ac.manchester.tornado.unittests.multithreaded.MultiThreaded#test02

All supported tests are passing. Pending for testing: A) KFusion B) RayTracer

[jjfumero]

TornadoVM RayTracer works

[jjfumero]

KFusion also works

[jjfumero]

All comments addressed