[feat] Query Memory Consumption and expanded profiler with memory consumption

[jjfumero]

Description

This PR expands the profiler and the TornadoVM Execution Plan API to obtain the current memory consumption per task-graph, total number of bytes transferred to the device back and forth per execution plan, and the total memory usage. This feature is requested from the GAIA project.

a) getTotalBytesTransferred

This expands the profiler to obtain the total number of bytes transferred in every execution plan launch. The number of bytes transferred depends on READ_ONLY, WRITE_ONLY and READ_WRITE data buffers.

 try (TornadoExecutionPlan executionPlan = new TornadoExecutionPlan(immutableTaskGraph)) {
            TornadoExecutionResult executionResult = executionPlan.execute();
            long totalBytesTransferred = executionResult.getProfilerResult().getTotalBytesTransferred();
            long copyInBytes = executionResult.getProfilerResult().getTotalBytesCopyIn();
            long copyOutBytes = executionResult.getProfilerResult().getTotalBytesCopyOut();
            assertEquals(copyInBytes + copyOutBytes, totalBytesTransferred);
        }

b) getTotalDeviceMemoryUsage

The total device memory usage registers all allocations needed to run an execution plan. This value can be queried using the TornadoVM profiler.

try (TornadoExecutionPlan executionPlan = new TornadoExecutionPlan(immutableTaskGraph)) {
            TornadoExecutionResult executionResult = executionPlan.execute();
            long totalMemoryUsedInBytes = executionResult.getProfilerResult().getTotalDeviceMemoryUsage();

            // 3 Arrays
            final long sizeAllocated = a.getNumBytesOfSegmentWithHeader() * 3;
            assertEquals(sizeAllocated, totalMemoryUsedInBytes);

        }

c) getCurrentMemoryUsage

Value to query at the execution plan level, to obtain the actual memory usage at any point during execution.

try (TornadoExecutionPlan executionPlan = new TornadoExecutionPlan(immutableTaskGraph)) {
            executionPlan.execute();
            long currentMemoryUsageInBytes = executionPlan.getCurrentMemoryUsage();
            final long sizeAllocated = a.getNumBytesOfSegmentWithHeader() * 3;
            assertEquals(sizeAllocated, currentMemoryUsageInBytes);
        }

Problem description

n/ a.

Backend/s tested

Mark the backends affected by this PR.

• [X] OpenCL
• [X] PTX
• [X] SPIRV

OS tested

Mark the OS where this PR is tested.

• [X] Linux
• [x] OSx
• [ ] Windows

Did you check on FPGAs?

If it is applicable, check your changes on FPGAs.

• [ ] Yes
• [X] No

How to test the new patch?

make BACKEND=opencl
tornado-test --enableProfiler console -V uk.ac.manchester.tornado.unittests.memory.MemoryConsumptionTest

make BACKEND=ptx
tornado-test --enableProfiler console -V uk.ac.manchester.tornado.unittests.memory.MemoryConsumptionTest

make BACKEND=spirv
tornado-test --enableProfiler console -V uk.ac.manchester.tornado.unittests.memory.MemoryConsumptionTest

[jjfumero]

All comments applied