Samsung / ONE

On-device Neural Engine
Other
429 stars 152 forks source link

[one-cmds] Reduce test time #12190

Open seanshpark opened 9 months ago

seanshpark commented 9 months ago

Like current luci-value-test takes long time to test and #12071 is to provide alternative method to reduce time, it would be wonderful if we could reduce test duration somehow.

mhs4670go commented 9 months ago

I think we have some options for this.

  1. Skip one-build related test as one-build is going to be deprecated. (#10345)
  2. Replace inceptionV3 model with lighter one.
  3. Introduce core(? or essential?) test and make it default one. And full test only runs in day-test.

For example, onecc_001.cfg and onecc_002.cfg look like below.

### onecc_001.cfg
[onecc]
one-import-tf=True
one-import-tflite=False
one-import-bcq=False
one-optimize=True
one-quantize=False
one-pack=False
one-codegen=False

[one-import-tf]
input_path=inception_v3.pb
output_path=inception_v3.onecc_001.circle
input_arrays=input
input_shapes=1,299,299,3
output_arrays=InceptionV3/Predictions/Reshape_1
converter_version=v2

[one-optimize]
input_path=inception_v3.onecc_001.circle
output_path=inception_v3.opt.circle

### onecc_002.cfg
[onecc]
one-import-tf=True
one-import-tflite=False
one-import-bcq=False
one-optimize=True
one-quantize=False
one-pack=True
one-codegen=False

[one-import-tf]
input_path=inception_v3.pb
output_path=inception_v3.onecc_002.circle
input_arrays=input
input_shapes=1,299,299,3
output_arrays=InceptionV3/Predictions/Reshape_1
converter_version=v2

[one-optimize]
input_path=inception_v3.onecc_002.circle
output_path=inception_v3.opt.circle

[one-pack]
input_path=inception_v3.opt.circle
output_path=inception_v3_pkg

Actually, these are very similar and have duplicates. we can just remove one of them.


Moreover, if we can think of it as heavy burden, we might as well focus on the command line test rather than the integration test. Seems that the most of the test duration is part of one-import-tf (not sure, need to check again) where tensorflow is imported. We can make a dummy for tf2tfliteV2 and just test if the given arguments are successfully passed into one-cmds tools. And testing tf2tfliteV2 can be done in its own test.

mhs4670go commented 9 months ago

Current status

Test duration: 1096.77s

ALL TESTS PASSED!
./runtestall.sh  1096.77s user 771.64s system 191% cpu 16:13.31 total