Samsung / ONE

On-device Neural Engine
Other
429 stars 157 forks source link

[onert] Introduce use-def chains into `TrainableGraph` #13317

Closed ragmani closed 1 month ago

ragmani commented 3 months ago

What

Let's introduce use-def chains and new index types for training into TrainableGraph and apply it for memory optimization.

Why

There was no use-def chains that could directly know dependencies between operations and operands in a TrainableGraph. So we were not even verifying whether TrainableGraph is a dag graph, and we were also unable to optimize memory for training.

Ways to support new index types for training

New types : TrainingOperationIndex, TrainingOperandIndex

  1. A way of distinguishing whether an operand or an operation is in forwarding and backwarding by using an existing index number and the information on whether it is in forwarding or in backwarding within the new index types.
    • An example
      
      template <typename Index> class TrainingIndex
      class TrainingIndex
      {
      ...
      private:
      Index _index;
      bool _is_forward;
      }

using TrainingOperationIndex = TrainingIndex; using TrainingOperandIndex = TrainingIndex;


2. A way of distinguishing whether an operand or an operation is in forwarding and backwarding only by using an new index number within the new index types.
- An example
```c++
using TrainingOperationIndex = ::onert::util::Index<uint32_t, TrainingOperationIndexTag>;
using TrainingOperandIndex = ::onert::util::Index<uint32_t, TrainingOperandIndexTag>;

I'm trying with the first way.

Draft : #13305

ragmani commented 1 month ago

I'm closing this issue because memory usage for training reduces as https://github.com/Samsung/ONE/issues/13282#issuecomment-2261836622 on master branch(08cfdf512d87904e791e31d751b5fb9354083cd2) now.

There are still parts that can be further optimized for memory. For example,

However, I won't proceed them unless there is any request since their effects are not expected to reduce memory usage dramatically.