Closed ragmani closed 1 month ago
I'm closing this issue because memory usage for training reduces as https://github.com/Samsung/ONE/issues/13282#issuecomment-2261836622 on master branch(08cfdf512d87904e791e31d751b5fb9354083cd2) now.
There are still parts that can be further optimized for memory. For example,
However, I won't proceed them unless there is any request since their effects are not expected to reduce memory usage dramatically.
What
Let's introduce use-def chains and new index types for training into
TrainableGraph
and apply it for memory optimization.Why
There was no use-def chains that could directly know dependencies between operations and operands in a
TrainableGraph
. So we were not even verifying whetherTrainableGraph
is a dag graph, and we were also unable to optimize memory for training.Ways to support new index types for training
New types :
TrainingOperationIndex
,TrainingOperandIndex
using TrainingOperationIndex = TrainingIndex;
using TrainingOperandIndex = TrainingIndex;
I'm trying with the first way.
13318
13338
13353
13372
13373
13374
13429
13452
13453
13454
13455
13456
13457
13458
13459
13460
13461
13462
13547
13352
13375
13379
13428
13432
13433
13470
13471
13476
13522
13523
13552
13553
13570
13571
13600
13601
13351
13376
13377
13378
Draft : #13305