Autoclosable Execution Plans for automatic memory management (free resources)

[jjfumero]

Description

There is a potential problem of having too many execution plans “open” without freeing the resources. The main issue is that TornadoVM cannot close/free the resources until the user explicitly calls freeDeviceMemory using the execution plan. This is by design, since an execution plan can reuse buffers and code cache.

However, there might be cases in which developers want to free resources using the try-with-resources clause from Java. In this case, the execution plan is created within a scope, and as soon that the syntatic scope is finished, we want to free all resources that a specific instance of an execution plan occupied (e.g. , free all memory buffers).

TornadoVM Try-With-Resources

From now on, developers can instance an execution plan using the try-with-resources statement. For example:

TaskGraph taskGraph = new TaskGraph("stress" + dataSizeFactor) //
       .transferToDevice(DataTransferMode.EVERY_EXECUTION, inputArray) //
       .task("moveData", TestStressDeviceMemory::moveData, inputArray, outputArray) //
       .transferToHost(DataTransferMode.EVERY_EXECUTION, outputArray);

ImmutableTaskGraph immutableTaskGraph = taskGraph.snapshot();
try (TornadoExecutionPlan executionPlan = new TornadoExecutionPlan(immutableTaskGraph)) {
   executionPlan.execute();
} catch (TornadoExecutionPlanException e) {}

When resources are freed, the TornadoVM runtime invokes the free device memory function, which calls an internal function to declare all the memory buffers used as free and ready to be used by other execution plans.

Backend/s tested

Mark the backends affected by this PR.

• [X] OpenCL
• [X] PTX
• [X] SPIRV

OS tested

Mark the OS where this PR is tested.

• [X] Linux
• [ ] OSx
• [ ] Windows

Did you check on FPGAs?

If it is applicable, check your changes on FPGAs.

• [ ] Yes
• [X] No

How to test the new patch?

$ tornado-test --fast --threadInfo --jvm="-Xmx12g -Dtornado.device.memory=4GB" -V uk.ac.manchester.tornado.unittests.memory.TestStressDeviceMemory