This is a minor optimization: in my TensorFlow graphs I can see that the clip_by_value lambda layer is converted as two layers (a clip with a maximum and a clip with a minimum). When the minimum value is of the clip is 0, the clip is equivalent to a ReLU layer with the max argument set. The resulting output has just a single node in the final graph, if the max value is 6 it actually turns in to a ReLU6 layer automatically. In my experience the clip nodes often come from ReLU6 in the original graph, so this makes for the best conversion.
gmalivenko
commented
5 years ago
This is a minor optimization: in my TensorFlow graphs I can see that the
clip_by_valuelambda layer is converted as two layers (a clip with a maximum and a clip with a minimum). When the minimum value is of the clip is 0, the clip is equivalent to a ReLU layer with the max argument set. The resulting output has just a single node in the final graph, if the max value is 6 it actually turns in to aReLU6layer automatically. In my experience the clip nodes often come fromReLU6in the original graph, so this makes for the best conversion.