how to change scheduler?

[Piorosen]

I had a question while analyzing the Scheduler Factory and Scheduler.

Values passed in SchedulerFactory::create are always passed to default values. The default value passed is determined at build time.

What I want to do is to designate a scheduler at runtime. How do I assign a scheduler at runtime?

Apart from the Scheduler Factory, it was analyzed as Scheduler. While analyzing this class, there is no reference or use anywhere. What role does the Scheduler play?

arm_compute/runtime/Scheduler.h arm_compute/runtime/SchedulerFactory.h

#if !ARM_COMPUTE_CPP_SCHEDULER && ARM_COMPUTE_OPENMP_SCHEDULER
const SchedulerFactory::Type SchedulerFactory::_default_type = SchedulerFactory::Type::OMP;
#elif ARM_COMPUTE_CPP_SCHEDULER && !ARM_COMPUTE_OPENMP_SCHEDULER
const SchedulerFactory::Type SchedulerFactory::_default_type = SchedulerFactory::Type::CPP;
#elif ARM_COMPUTE_CPP_SCHEDULER && ARM_COMPUTE_OPENMP_SCHEDULER
const SchedulerFactory::Type SchedulerFactory::_default_type = SchedulerFactory::Type::CPP;
#else  /* ARM_COMPUTE_*_SCHEDULER */
const SchedulerFactory::Type SchedulerFactory::_default_type = SchedulerFactory::Type::ST;
#endif /* ARM_COMPUTE_*_SCHEDULER */

[morgolock]

Hi @Piorosen

What I want to do is to designate a scheduler at runtime. How do I assign a scheduler at runtime?

ACL does not provide the functionality to change the scheduler at runtime. We explored this alternative with the introduction of the RuntimeContext but this is not functional and not planned to be completed. See https://github.com/ARM-software/ComputeLibrary/blob/main/tests/validation/NEON/UNIT/RuntimeContext.cpp#L65

You can however specify the scheduler at compilation time using openmp=0|1 and cppthreads=0|1

If you build with openmp=0 and cppthreads=0 you will get a scheduler using a single thread.

The options openmp and cppthreads are mutually exclusive.

Apart from the Scheduler Factory, it was analyzed as Scheduler. While analyzing this class, there is no reference or use anywhere. What role does the Scheduler play?

The scheduler is used by operators/functions to execute kernels on multiple threads in the Neon backend. See https://github.com/ARM-software/ComputeLibrary/blob/main/src/cpu/operators/CpuSoftmax.cpp#L202

Hope this helps.

[Piorosen]

Thank you! @morgolock

How about providing the function to change the scheduler using system environment information? It is already used by CPP Scheduler, and I think the function is already sufficient.

https://github.com/ARM-software/ComputeLibrary/blob/main/arm_compute/core/utils/misc/Utility.h#L222 https://github.com/ARM-software/ComputeLibrary/blob/main/src/runtime/CPP/CPPScheduler.cpp#L328

[morgolock]

Hi @Piorosen

The scheduler type is set at compilation time, the factory will always instantiate the type specified at compilation time: https://github.com/ARM-software/ComputeLibrary/blob/main/src/runtime/SchedulerFactory.cpp#L40

What's your use-case for changing the scheduler at runtime?

An easy way to go around this is to have multiple binaries and run the one with the scheduler you need from the command line or a script.

Using an environment variable for this would be a good way to allow the user to change the scheduler or even other configuration parameters for the library. But this will require some changes to the library.

Hope this helps.

[Piorosen]

I'm doing a performance analysis of the ACL. I wanted to be able to test N times with one build for performance analysis. I was looking for a way to change it at runtime because I was lazy.