search

Samsung / ONE

On-device Neural Engine
Other
414 stars 145 forks source link

[one-cmds] define optimizations #9191

Open seanshpark opened 2 years ago

seanshpark commented 2 years ago

There was an old issue about one-optimize group options, but as I remember without concrete conclusion about what is with what.

Now, it's time to define them.

seanshpark commented 2 years ago

9190 is already fired but

we need to (1) define group of optimization and (2) switchs of pass for each group.

YongseopKim commented 2 years ago

Related issue: https://github.com/Samsung/ONE/issues/5784

YongseopKim commented 2 years ago

Very sorry for late.

Why did I do like https://github.com/Samsung/ONE/pull/9190?

YongseopKim commented 2 years ago

[one-cmds] define optimizations There was an old issue about one-optimize group options, but as I remember without concrete conclusion about what is with what.

Is it for just one-optimize? or all passes(eg. from one-import, ... to one-optimize)?

(1) define group of optimization

group means by model type(tf, tflite, ...)? I guess so.

(2) switchs of pass for each group

I first planned like that

However, now I fix my idea after discussion with @jyoungyun on 5/27. The maintain for the files can be very expensive. So now I think the optimization group can be done by one-quantize.

seanshpark commented 2 years ago

Is it for just one-optimize? or all passes?

I don't understand the question. one-import just imports source model (tf, tflite, onnx, ...) to circle model. I don't know what related you want to ask with one-optimize.

group means by model type(tf, tflite, ...)? I guess so.

Maybe so, but not from model source type but optimization purpose. This is currently fuzzy for me too.

ODEFAULT_GROUP_A, ODEFAULT_GROUP_B

I'm against using word DEFAULT and adding suffix to word DEFAULT makes more confusion.

Pass the parameter of one-quantize with DEFAULT_GROUP_A So now I think the optimization group can be done by one-quantize.

Why does one-quantize come here? Please explain more about this.

So, it doesn't switchs of pass for each group

This too. I can't catch this.

seanshpark commented 2 years ago

I thought that almost options are okay So I think it's okay to turn on all options If so, the default option supports all true options

ONNX related NCHW to NHWC and maybe (not sure) with transpose, reshape, may give unexpected results from TF models. This is the main issue from your draft.

And as I wrote above, I'm against using word DEFAULT.

seanshpark commented 2 years ago

The word group I mentioned needs to be changed to another word for approrpriate purpose.

YongseopKim commented 2 years ago

I don't understand the question. one-import just imports source model (tf, tflite, onnx, ...) to circle model. I don't know what related you want to ask with one-optimize.

I asked this because of https://github.com/Samsung/ONE/pull/9190#issuecomment-1139398914 . So I thought optimization could be included from one-import to one-quantizaiton if possible. (Of course, I could misunderstand 😢 )

seanshpark commented 2 years ago

I asked this because of https://github.com/Samsung/ONE/pull/9190#issuecomment-1139398914 .

@mhs4670go , can you clarify what you mean in the comment? and about So I thought optimization could be included from one-import to one-quantizaiton if possible. ?

mhs4670go commented 2 years ago

@YongseopKim The old issue that @seanshpark is referring to is https://github.com/Samsung/ONE/issues/5784. And, this is for one-optimize and technically, it is for circle2circle optimization pass.

"optimization" will be able to be applied to other one-cmds tools as well as one-optimize.

@YongseopKim 's thought is from this. "optimization" would be able to have broader meaning. For example, we could need some optimization or preprocessing in import step for domain specific import. But, we've never had a discussion on this. This is a feature for future.

YongseopKim commented 2 years ago

@YongseopKim 's thought is from this. "optimization" would be able to have broader meaning. For example, we could need some optimization or preprocessing in import step for domain specific import. But, we've never had a discussion on this. This is a feature for future.

I thought there was a consensus on it. So, now I don't think of optimization as a broader meaning. I focus on optimization for circle-circle, one-optimize.

YongseopKim commented 2 years ago

Maybe so, but not from model source type but optimization purpose. This is currently fuzzy for me too. I'm against using word DEFAULT and adding suffix to word DEFAULT makes more confusion. ONNX related NCHW to NHWC and maybe (not sure) with transpose, reshape, may give unexpected results from TF models. This is the main issue from your draft.

While I'm reading these comments, I realize that I'm using mixed meaning of default. I use it as 'basic options(meaning as it is default)' and 'optimized options as default'.

YongseopKim commented 2 years ago

And this issue is for defining an optimization list for each group(eg. onnx_with_transpose, ..., not yet defined).

'basic options(meaning as it is the default)' and 'optimized options as default' should be handled by different issues.

YongseopKim commented 2 years ago

Before suggesting my idea for Optimization Options, let's define when the optimization option groups(?) are necessary.

When?

  1. User doesn't have any cfg file so that uses onecc init with its own model. ONE wants to support possible optimization options. ONE knows the only model type. So onecc init wants to fill an optimization options group for the model type.
  2. User has a cfg file and can make the final compiled output by the backend. The user wants to optimize it simply/easily. So user uses onecc optimize with some option groups (or by fixing cfg file).
  3. User has a cfg file and is good at ONE. The user wants to optimize at most. The user knows the main operations of the model and other compiled options.

I suggest

I'd like to get your opinions on the suggestion.

seanshpark commented 2 years ago

for the beginning, +1 to No.1