galeone / dynamic-training-bench

Simplify the training and tuning of Tensorflow models
Mozilla Public License 2.0
213 stars 31 forks source link

At least two variables have the same name: global_step #20

Closed pierlui92 closed 6 years ago

pierlui92 commented 6 years ago

System information TensorFlow version: 1.5 Python version: 3.5 CUDA/cuDNN version: CUDA 9.0, CuDNN 7 Command to reproduce:

    info = train(
        model=model,
        dataset=dataset,
        hyperparameters={
            "batch_size": 128,
            "epochs": 20,
            "regularizations": {
                "l2": 1e-4,
                "augmentation": {
                    "name": "noise_brightness",
                    "fn": lambda image: tf.image.random_brightness(aug(image), max_delta=32./255.),
                    "factor": 100
                }
            },
            "gd": {
                "optimizer": tf.train.AdamOptimizer,
                    "args": {
                        "learning_rate": 1e-3
                }
            },
            "seed": None,        },
        comment='test',
        force_restart=False)

After upgrading to Tensorflow 1.5 I get this error in train function:

At least two variables have the same name: global_step <

galeone commented 6 years ago

Thank you for reporting this. From tensorfloe 1.5 the global step variable is considered a variable to save by tensorflow itself (finally!). Hence I had to remove my fix in order to solve the problem