search

qubvel / efficientnet

Implementation of EfficientNet model. Keras and TensorFlow Keras.
https://arxiv.org/abs/1905.11946
Apache License 2.0
2.07k stars 472 forks source link

ValueError: rate must be a scalar tensor or a float in the range [0, 1), got 1 #142

Open nkowlagi opened 3 years ago

nkowlagi commented 3 years ago

import efficientnet.keras as efn base_model = efn.EfficientNetB7(weights=None, include_top=False,drop_connect_rate=0.4)

model.fit fails for the error "ValueError: rate must be a scalar tensor or a float in the range [0, 1), got 1" though a valid value is given for drop_connect_rate. If I remove and run it, works as expected

ValueError: in user code:

/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/training.py:805 train_function  *
    return step_function(self, iterator)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/training.py:795 step_function  **
    outputs = model.distribute_strategy.run(run_step, args=(data,))
/usr/local/lib/python3.7/dist-packages/tensorflow/python/distribute/distribute_lib.py:1259 run
    return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/distribute/distribute_lib.py:2730 call_for_each_replica
    return self._call_for_each_replica(fn, args, kwargs)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/distribute/distribute_lib.py:3417 _call_for_each_replica
    return fn(*args, **kwargs)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/training.py:788 run_step  **
    outputs = model.train_step(data)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/training.py:754 train_step
    y_pred = self(x, training=True)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/base_layer.py:1012 __call__
    outputs = call_fn(inputs, *args, **kwargs)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/functional.py:425 call
    inputs, training=training, mask=mask)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/functional.py:560 _run_internal_graph
    outputs = node.layer(*args, **kwargs)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/base_layer.py:1012 __call__
    outputs = call_fn(inputs, *args, **kwargs)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/layers/core.py:231 call  **
    lambda: array_ops.identity(inputs))
/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/utils/control_flow_util.py:115 smart_cond
    pred, true_fn=true_fn, false_fn=false_fn, name=name)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/smart_cond.py:54 smart_cond
    return true_fn()
/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/layers/core.py:228 dropped_inputs
    rate=self.rate)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/dispatch.py:201 wrapper
    return target(*args, **kwargs)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/deprecation.py:538 new_func
    return func(*args, **kwargs)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/nn_ops.py:5059 dropout
    return dropout_v2(x, rate, noise_shape=noise_shape, seed=seed, name=name)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/dispatch.py:201 wrapper
    return target(*args, **kwargs)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/nn_ops.py:5140 dropout_v2
    "range [0, 1), got %g" % rate)

ValueError: rate must be a scalar tensor or a float in the range [0, 1), got 1
tirear commented 3 years ago

I faced this problem when I tried to use default EfficientNetL2. The dropout rate was gradually increased from 0.0125 to 1.0 as the number of blocks was increasing. I think maybe it is a good idea to modify the model.py file as follow,

--- a/efficientnet/model.py 2021-07-07 10:52:39.000000000 +0900
+++ b/efficientnet/model.py 2021-07-07 10:43:06.000000000 +0900
@@ -366,6 +366,8 @@

         # The first block needs to take care of stride and filter size increase.
         drop_rate = drop_connect_rate * float(block_num) / num_blocks_total
+        if drop_rate > 0.5:
+          drop_rate = 0.5
         x = mb_conv_block(x, block_args,
                           activation=activation,
                           drop_rate=drop_rate,
@@ -378,6 +380,9 @@
             # pylint: enable=protected-access
             for bidx in xrange(block_args.num_repeat - 1):
                 drop_rate = drop_connect_rate * float(block_num) / num_blocks_total
+                if drop_rate > 0.5:
+                  drop_rate = 0.5
                 block_prefix = 'block{}{}_'.format(
                     idx + 1,
                     string.ascii_lowercase[bidx + 1]