Open bayesian-mind opened 2 years ago
Any suggestions for this problem?
My problem in version TF2.7.
class TNN(tfk.Model):
def __init__(self, input_dim):
super(TNN, self).__init__(name='TNN')
self.fn_dense_input = tfk.layers.Dense(units=256, input_dim=input_dim)
self.fn_dense_hidden = tfk.layers.Dense(units=128)
self.fn_dense_output = tfk.layers.Dense(units=1, activation='sigmoid')
self.fn_reg_1 = tfk.layers.ActivityRegularization(l1=0.01, l2=0.01, input_dim=256)
self.fn_reg_2 = tfk.layers.ActivityRegularization(l1=0, l2=0.01, input_dim=128)
self.fn_bn_1 = tfk.layers.BatchNormalization()
self.fn_bn_2 = tfk.layers.BatchNormalization() # not repeat
self.fn_af = tfk.layers.Activation(activation='relu')
self.fn_dropout = tfk.layers.AlphaDropout(rate=0.85)
@tf.function
def call(self, inputs):
x = self.fn_dense_input(inputs)
x = self.fn_reg_1(x)
x = self.fn_bn_1(x)
x = self.fn_af(x)
# x = self.fn_dropout(x)
x = self.fn_dense_hidden(x)
x = self.fn_reg_2(x)
x = self.fn_bn_2(x)
x = self.fn_af(x)
# x = self.fn_dropout(x)
outputs = self.fn_dense_output(x)
return outputs
Error output:
InaccessibleTensorError: tf.Graph captured an external symbolic tensor. The symbolic tensor <tf.Tensor 'activity_regularization/ActivityRegularizer/truediv:0' shape=() dtype=float32> is captured by FuncGraph(name=train_function, id=139760322273232), but it is defined at FuncGraph(name=call, id=139760322164240). A tf.Graph is not allowed to capture symoblic tensors from another graph. Use return values, explicit Python locals or TensorFlow collections to access it. Please see https://www.tensorflow.org/guide/function#all_outputs_of_a_tffunction_must_be_return_values for more information.
same problem in TF2.4.2. any suggestion?
My problem in version TF2.7.
class TNN(tfk.Model): def __init__(self, input_dim): super(TNN, self).__init__(name='TNN') self.fn_dense_input = tfk.layers.Dense(units=256, input_dim=input_dim) self.fn_dense_hidden = tfk.layers.Dense(units=128) self.fn_dense_output = tfk.layers.Dense(units=1, activation='sigmoid') self.fn_reg_1 = tfk.layers.ActivityRegularization(l1=0.01, l2=0.01, input_dim=256) self.fn_reg_2 = tfk.layers.ActivityRegularization(l1=0, l2=0.01, input_dim=128) self.fn_bn_1 = tfk.layers.BatchNormalization() self.fn_bn_2 = tfk.layers.BatchNormalization() # not repeat self.fn_af = tfk.layers.Activation(activation='relu') self.fn_dropout = tfk.layers.AlphaDropout(rate=0.85) @tf.function def call(self, inputs): x = self.fn_dense_input(inputs) x = self.fn_reg_1(x) x = self.fn_bn_1(x) x = self.fn_af(x) # x = self.fn_dropout(x) x = self.fn_dense_hidden(x) x = self.fn_reg_2(x) x = self.fn_bn_2(x) x = self.fn_af(x) # x = self.fn_dropout(x) outputs = self.fn_dense_output(x) return outputs
Error output:
InaccessibleTensorError: tf.Graph captured an external symbolic tensor. The symbolic tensor <tf.Tensor 'activity_regularization/ActivityRegularizer/truediv:0' shape=() dtype=float32> is captured by FuncGraph(name=train_function, id=139760322273232), but it is defined at FuncGraph(name=call, id=139760322164240). A tf.Graph is not allowed to capture symoblic tensors from another graph. Use return values, explicit Python locals or TensorFlow collections to access it. Please see https://www.tensorflow.org/guide/function#all_outputs_of_a_tffunction_must_be_return_values for more information.
I see the same issue, TF2.7, model_optimization v0.7.3. I'm just removing the ActivityRegularization for now.
Describe the Issue Activity Regularizer not working with quantization aware training (QAT). TypeError: An op outside of the function building code is being passed a "Graph" tensor.
System information
TensorFlow version (installed from source or binary): TF 2.6
TensorFlow Model Optimization version (installed from source or binary): pip install
Python version: 3.6.9
Describe the expected behavior This happens only on quantized models, non-quantized model have numpy value for activity regularization loss. The model losses i.e. regularization losses should be accumulated without empty tensors/placeholders. And should be able to operate on them outside the graph when executing in eager mode.
Describe the current behavior Currently the activity regularization loss tensor is of type
<class 'tensorflow.python.framework.ops.Tensor'>
with no numpy values, the bias regularization tensor is also tf.tensor but that does carry a numpy value. But I am not sure if that is throwing the error, where I am unable to operate on the the tf op.Code to reproduce the issue
Error Output