[onert/training] Add a feature of setting trainable op to circle_plus_gen

[zetwhite]

background

The fine-tuning feature is being introduced to the onert based on this draft ( https://github.com/Samsung/ONE/pull/12951. thanks to @mbencer). Since circle+ file is a model file for training, It is natural that circle+ holds a fine-tuning configuration. And already, metadata schema holds a trainable_ops for fine-tuning. https://github.com/Samsung/ONE/blob/2deec3926ac4989723fa0a78866844d7793e565c/runtime/libs/circle-schema/include/circle_traininfo.fbs#L110-L122

what to do

In order to make easy use of fine-tuning feature, Let's support generating a circle+ file with a trainable_ops.

reference

• https://github.com/Samsung/ONE/pull/12984

/cc @jyoungyun @mbencer

[zetwhite]

how to do

I'm still thinking about its' interface. How about this kind of interface?

# set last 2 layers are trainable for input.circle file 
circle_plus_generator/main.py input.circle --fine-tune -2

# set last 2 layers are trainable for input.circle file 
circle_plus_generator/main.py input.circle --train-only -2

[zetwhite]

how to do

# set full training (default, if nothing is given)
circle_plus_generator/main.py input.circle --train all 

# set last 2 layers are trainable for input.circle file 
circle_plus_generator/main.py input.circle --train last2

# not training (might be used for inference) 
circle_plus_generator/main.py input.circle --train none

# select a specific layer to be trained
# hidden feature for developer, not going to write a document about this
circle_plus_generator/main.py input.circle --train 1-5, 7, 9

[chunseoklee]

@zetwhite circle_plus_generator/main.py input.circle --train-last 2 generates trainable_ops : [8,9] in metadata if 10 layers in seq ?

[zetwhite]

@zetwhite circle_plus_generator/main.py input.circle --train-last 2 generates trainable_ops : [8,9] in metadata if 10 layers in seq ?

I thought so.

[zetwhite]

better to have similar interface with https://github.com/Samsung/ONE/issues/13140#issuecomment-2175944483

+) So, updated interface a bit https://github.com/Samsung/ONE/pull/13261#discussion_r1647574923