Compiler FE: Adding U8 recipies

seanshpark commented 4 years ago

As you can see in https://github.com/Samsung/ONE/tree/master/res/TensorFlowLiteRecipes folder, there are sub folders of recipe that we support(some are work in progress).

recipe is a protocol buffer file, a soure file to generate TF lite file
tflchef is a tool that actually generates the file
we use human readable text source so that editing is easy only with a text editor

There are two data types that we support in major: Float32 and Uint8. All the Ops have Float32. But some have Uint8 recipe.

This task is to create Uint8 recipes that are missing

https://www.tensorflow.org/lite/performance/quantization_spec shows list of Ops that TensorFlow defined.
This document provides specifications for Int8 but we use Uint8.
Related paper with Uint8: https://arxiv.org/abs/1712.05877
- Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference

Things to do

Write test.recipe file. You can copy-paste from existing ones
Add item to https://github.com/Samsung/ONE/blob/master/compiler/luci/tests/test.lst file
- configure - build - test should pass
Fix problems if anything goes wrong
One recipe at a time, that is one recipe per PR

You can also fire an issue (optional), Add for example

[res] Add U8 recipe: ADD

Let's enable U8 recipe for Add Op

This will let others, including me, know that one is already working on it.

Steps (not mandatory but usually this may happen)

Create an issue with title like [res] Add U8 recipe: ADD for ADD Op, and some descriptions
It would be nice if you work on one Op at a time to concentrate on the operator
I'll update the table of this issue that others know you are working on like #1606 (I'll be notified by github from the new issue)
create a PR and work on this topic; update PR with Work In Progress codes and this may help getting faster feedbacks
get a review from guys working on this topic, like @jinevening
apply feedback changes if any
land the PR and done !

How to test

./nncc configure
./nncc build
./nncc test

seanshpark commented 4 years ago

Operators in https://www.tensorflow.org/lite/performance/quantization_spec

please add a comment if this table is incorrect and needs update

Operator	status
ADD	(exist)
AVERAGE_POOL_2D	@underflow101 (#3429) done
CONCATENATION	(exist)
CONV_2D	(exist)
DEPTHWISE_CONV_2D	(exist)
FULLY_CONNECTED	(exist)
L2_NORMALIZATION	@underflow101 (#3444) done
LOGISTIC	@underflow101 (#3461) done
MAX_POOL_2D	(exist)
MUL	(exist)
RESHAPE	(exist)
RESIZE_BILINEAR	@underflow101 done
SOFTMAX	(exist)
SPACE_TO_DEPTH	@underflow101 (#3583)
TANH	@underflow101 (#3592)
PAD	(exist)
GATHER
BATCH_TO_SPACE_ND
SPACE_TO_BATCH_ND
TRANSPOSE	done
MEAN	(exist)
SUB	(exist)
SUM
SQUEEZE
LOG_SOFTMAX	@underflow101 done
MAXIMUM	done
ARG_MAX	(exist)
MINIMUM	done
LESS	done
PADV2	(float32 not ready yet)
GREATER	done
GREATER_EQUAL	done
LESS_EQUAL	done
SLICE
EQUAL	done
NOT_EQUAL	done
SHAPE
QUANTIZE	(float32 not ready yet)

vasantteja commented 4 years ago

Would like to work on this if this is open.

seanshpark commented 4 years ago

@vasantteja Thank you for your interest! Sure, please pick an Op in the list and fire an issue:) I think I need to prepare a template for this :) For the mean time, this can be used; for an example;

[res] Add U8 recipe: ADD

Let's enable U8 recipe for Add Op

We can discuss within this issue if you have any questions.

vasantteja commented 4 years ago

@seanshpark Sure. I will start working on this. Can I please connect with you if I have any doubts. Thanks in advance.

seanshpark commented 4 years ago

Can I please connect with you if I have any doubts.

Sure :) feel free to ask any questions related.

underflow101 commented 4 years ago

@seanshpark, I would like to solve this issue. Is there any place I could join in this problem solving?

seanshpark commented 4 years ago

Is there any place I could join in this problem solving?

@underflow101 , thank your for your interest! I think you've already joined :)

I'm not sure what place you mentioned could be... but I think the steps would be like

Create an issue with title like [res] Add U8 recipe: ADD for ADD Op, and some descriptions
It would be nice if you work ~one~ on one Op at a time to concentrate on the operator
I'll update the table of this issue that others know you are working on like #1606 (I'll be notified by github from the new issue)
create a PR and work on this topic; update PR with Work In Progress codes and this may help getting faster feedbacks
get a review from guys working on this topic, like @jinevening
apply feedback changes if any
land the PR and done !

underflow101 commented 4 years ago

@seanshpark , then is it okay for me to work on QUANTIZE op recipe? If that is okay for you, I will write an issue as you have guided above and start working on it.

seanshpark commented 4 years ago

then is it okay for me to work on QUANTIZE op recipe? If that is okay for you, I will write an issue as you have guided above and start working on it.

Sure :) please go ahead

underflow101 commented 4 years ago

@seanshpark , Thank you for your kind reply :)

underflow101 commented 4 years ago

@seanshpark , I would like to solve AveragePooling2D issue as you have advised me on #3426

Thank you.

underflow101 commented 4 years ago

@seanshpark , I have finished my work on AveragePool2D_U8_000 (#3439), waiting for merge. I would like to solve L2_NORMALIZATION for my next task, if it is okay.

Thank you.

underflow101 commented 4 years ago

@seanshpark , I would like to solve LOGISTIC for my next task, if it is okay.

Thank you.

underflow101 commented 4 years ago

@seanshpark , I would like to solve RESIZE_BILINEAR while Logistic U8 compiler fix is ongoing, if it is okay.

Thank you.

underflow101 commented 4 years ago

@seanshpark , I have a question: Things like SUM operation has Sum_000 & Sum_001. Does this mean I have to make U8 recipe for each one of them? Or do I have to make U8 recipe for only Sum_000?

seanshpark commented 4 years ago

Does this mean I have to make U8 recipe for each one of them? Or do I have to make U8 recipe for only Sum_000?

These multiple recipes are to check diffent parameters. For U8, I think only 000 is OK.

underflow101 commented 4 years ago

These multiple recipes are to check diffent parameters. For U8, I think only 000 is OK.

Got it! :+1: Thank you @seanshpark !

underflow101 commented 4 years ago

@seanshpark , I'm having problems with Resize_Bilinear.

Related Issue: #3497 , circle2circle: /home/bear/data/opensource/ONE/compiler/luci/service/src/CircleTypeInferenceRule.cpp:521: virtual loco::DataType {anonymous}::TypeInferenceAlgorithm::visit(const luci::CircleOutput*): Assertion output_dtype == loco::dtype_get(node->from())' failed.

I have put 4 days of effort in this, (manipulating parameters, etc.) but it doesn't work well. I think it would be better for me to put my effort in another recipe first, and then come back to this.

I will work on generating U8 recipe for SPACE_TO_DEPTH.

Thank you.

underflow101 commented 4 years ago

RESIZE_BILINEAR & LOGSOFTMAX done. Please update table if you have spare time.

Thank you!

Samsung / ONE

Compiler FE: Adding U8 recipies #1880