the-deep-learners / deep-learning-illustrated

Deep Learning Illustrated (2020)
https://www.deeplearningillustrated.com
MIT License
708 stars 355 forks source link

Update notebook for Python 3 #1

Closed tromgy closed 5 years ago

tromgy commented 5 years ago

With Python 3.x (and the docker image shows version 3.6.7) we need to explcitly use integer (aka floor) division, otherwise Keras throws:

TypeError: Value passed to parameter 'shape' has DataType float32 not in list of allowed values: int32, int64

for instance:

>>> 256/4
64.0
>>> 256//4
64
jonkrohn commented 5 years ago

Hi @tromgy -- I like this idea, however I'm unable to replicate the error. Are you using the Dockerfile dependencies provided here including Keras 2.2.0?

tromgy commented 5 years ago

Good point, I have compared and my docker file is identical (I have recently re-cloned from this new repo and rebuilt the docker image).

However when I did

print(keras.__version__)

right in that same notebook I got this 2.0.8

The Docker file says 2.2.0 and the base image doesn't list Keras, so I'm at a loss where that version mismatch might be coming from.

tromgy commented 5 years ago

OK, so it might have been running an old docker image somehow, as I have restarted everything and made sure it got docker from the latest directory, and lo and behold, keras.__version__ reports 2.2.0

But, I still see the same error from Keras if I use standard division n_dense/4

Here's the complete error stack:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-9-eaf9fd2c6473> in <module>()
     30 
     31 dense_2 = Dense(n_dense/4, 
---> 32                 activation='relu', name='dense_2')(drop_dense_layer)
     33 dropout_2 = Dropout(dropout, name='drop_dense_2')(dense_2)
     34 

/opt/conda/lib/python3.6/site-packages/keras/engine/base_layer.py in __call__(self, inputs, **kwargs)
    430                                          '`layer.build(batch_input_shape)`')
    431                 if len(input_shapes) == 1:
--> 432                     self.build(input_shapes[0])
    433                 else:
    434                     self.build(input_shapes)

/opt/conda/lib/python3.6/site-packages/keras/layers/core.py in build(self, input_shape)
    870                                       name='kernel',
    871                                       regularizer=self.kernel_regularizer,
--> 872                                       constraint=self.kernel_constraint)
    873         if self.use_bias:
    874             self.bias = self.add_weight(shape=(self.units,),

/opt/conda/lib/python3.6/site-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs)
     89                 warnings.warn('Update your `' + object_name +
     90                               '` call to the Keras 2 API: ' + signature, stacklevel=2)
---> 91             return func(*args, **kwargs)
     92         wrapper._original_function = func
     93         return wrapper

/opt/conda/lib/python3.6/site-packages/keras/engine/base_layer.py in add_weight(self, name, shape, dtype, initializer, regularizer, trainable, constraint)
    247         if dtype is None:
    248             dtype = K.floatx()
--> 249         weight = K.variable(initializer(shape),
    250                             dtype=dtype,
    251                             name=name,

/opt/conda/lib/python3.6/site-packages/keras/initializers.py in __call__(self, shape, dtype)
    216             limit = np.sqrt(3. * scale)
    217             return K.random_uniform(shape, -limit, limit,
--> 218                                     dtype=dtype, seed=self.seed)
    219 
    220     def get_config(self):

/opt/conda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py in random_uniform(shape, minval, maxval, dtype, seed)
   4075         seed = np.random.randint(10e6)
   4076     return tf.random_uniform(shape, minval=minval, maxval=maxval,
-> 4077                              dtype=dtype, seed=seed)
   4078 
   4079 

/opt/conda/lib/python3.6/site-packages/tensorflow/python/ops/random_ops.py in random_uniform(shape, minval, maxval, dtype, seed, name)
    240           shape, minval, maxval, seed=seed1, seed2=seed2, name=name)
    241     else:
--> 242       rnd = gen_random_ops.random_uniform(shape, dtype, seed=seed1, seed2=seed2)
    243       return math_ops.add(rnd * (maxval - minval), minval, name=name)
    244 

/opt/conda/lib/python3.6/site-packages/tensorflow/python/ops/gen_random_ops.py in random_uniform(shape, dtype, seed, seed2, name)
    672     _, _, _op = _op_def_lib._apply_op_helper(
    673         "RandomUniform", shape=shape, dtype=dtype, seed=seed, seed2=seed2,
--> 674         name=name)
    675     _result = _op.outputs[:]
    676     _inputs_flat = _op.inputs

/opt/conda/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py in _apply_op_helper(self, op_type_name, name, **keywords)
    607               _SatisfiesTypeConstraint(base_type,
    608                                        _Attr(op_def, input_arg.type_attr),
--> 609                                        param_name=input_name)
    610             attrs[input_arg.type_attr] = attr_value
    611             inferred_from[input_arg.type_attr] = input_name

/opt/conda/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py in _SatisfiesTypeConstraint(dtype, attr_def, param_name)
     58           "allowed values: %s" %
     59           (param_name, dtypes.as_dtype(dtype).name,
---> 60            ", ".join(dtypes.as_dtype(x).name for x in allowed_list)))
     61 
     62 

TypeError: Value passed to parameter 'shape' has DataType float32 not in list of allowed values: int32, int64

Now looking at it carefully it seems that the error originates in Tensorflow. I checked that version (the notebooks don't directly import Tensorflow) and it shows 1.8.0 just as listed in the docker file

jonkrohn commented 5 years ago

Huh, that's weird. Thank you very much for bringing this to my attention, @tromgy -- I can't take the time to resolve this today, but will ASAP.

jonkrohn commented 5 years ago

I've figured out the discrepancy between our docker images @tromgy ... jupyter/scipy-notebook:e7000ca1416d specified in the Dockerfile corresponds to Python 3.6.5. Any insight into how you ended up with 3.6.7?

tromgy commented 5 years ago

It might have been that "old" docker image, although I don't know where it could have came from. Right now I'm sure running the latest Docker image that I built from the docker file in this repo on May 20.

I added this code to the notebook:

# report versions
import sys
print('Python:', sys.version)

import tensorflow
print('Tensorflow:', tensorflow.__version__)

print('Keras:', keras.__version__)

dense_2 = Dense(n_dense/4, 
                activation='relu', name='dense_2')(drop_dense_layer)
dropout_2 = Dropout(dropout, name='drop_dense_2')(dense_2)

# sigmoid output layer: 
predictions = Dense(1, activation='sigmoid', name='output')(dropout_2)

# create model: 
model = Model(input_layer, predictions)

and that's what I get when I run this cell:

Python: 3.6.5 | packaged by conda-forge | (default, Apr  6 2018, 13:39:56) 
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)]
Tensorflow: 1.8.0
Keras: 2.2.0
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-13-3ff1d14e3d09> in <module>()
     40 
     41 dense_2 = Dense(n_dense/4, 
---> 42                 activation='relu', name='dense_2')(drop_dense_layer)
     43 dropout_2 = Dropout(dropout, name='drop_dense_2')(dense_2)
     44 

/opt/conda/lib/python3.6/site-packages/keras/engine/base_layer.py in __call__(self, inputs, **kwargs)
    430                                          '`layer.build(batch_input_shape)`')
    431                 if len(input_shapes) == 1:
--> 432                     self.build(input_shapes[0])
    433                 else:
    434                     self.build(input_shapes)

/opt/conda/lib/python3.6/site-packages/keras/layers/core.py in build(self, input_shape)
    870                                       name='kernel',
    871                                       regularizer=self.kernel_regularizer,
--> 872                                       constraint=self.kernel_constraint)
    873         if self.use_bias:
    874             self.bias = self.add_weight(shape=(self.units,),

/opt/conda/lib/python3.6/site-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs)
     89                 warnings.warn('Update your `' + object_name +
     90                               '` call to the Keras 2 API: ' + signature, stacklevel=2)
---> 91             return func(*args, **kwargs)
     92         wrapper._original_function = func
     93         return wrapper

/opt/conda/lib/python3.6/site-packages/keras/engine/base_layer.py in add_weight(self, name, shape, dtype, initializer, regularizer, trainable, constraint)
    247         if dtype is None:
    248             dtype = K.floatx()
--> 249         weight = K.variable(initializer(shape),
    250                             dtype=dtype,
    251                             name=name,

/opt/conda/lib/python3.6/site-packages/keras/initializers.py in __call__(self, shape, dtype)
    216             limit = np.sqrt(3. * scale)
    217             return K.random_uniform(shape, -limit, limit,
--> 218                                     dtype=dtype, seed=self.seed)
    219 
    220     def get_config(self):

/opt/conda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py in random_uniform(shape, minval, maxval, dtype, seed)
   4075         seed = np.random.randint(10e6)
   4076     return tf.random_uniform(shape, minval=minval, maxval=maxval,
-> 4077                              dtype=dtype, seed=seed)
   4078 
   4079 

/opt/conda/lib/python3.6/site-packages/tensorflow/python/ops/random_ops.py in random_uniform(shape, minval, maxval, dtype, seed, name)
    240           shape, minval, maxval, seed=seed1, seed2=seed2, name=name)
    241     else:
--> 242       rnd = gen_random_ops.random_uniform(shape, dtype, seed=seed1, seed2=seed2)
    243       return math_ops.add(rnd * (maxval - minval), minval, name=name)
    244 

/opt/conda/lib/python3.6/site-packages/tensorflow/python/ops/gen_random_ops.py in random_uniform(shape, dtype, seed, seed2, name)
    672     _, _, _op = _op_def_lib._apply_op_helper(
    673         "RandomUniform", shape=shape, dtype=dtype, seed=seed, seed2=seed2,
--> 674         name=name)
    675     _result = _op.outputs[:]
    676     _inputs_flat = _op.inputs

/opt/conda/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py in _apply_op_helper(self, op_type_name, name, **keywords)
    607               _SatisfiesTypeConstraint(base_type,
    608                                        _Attr(op_def, input_arg.type_attr),
--> 609                                        param_name=input_name)
    610             attrs[input_arg.type_attr] = attr_value
    611             inferred_from[input_arg.type_attr] = input_name

/opt/conda/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py in _SatisfiesTypeConstraint(dtype, attr_def, param_name)
     58           "allowed values: %s" %
     59           (param_name, dtypes.as_dtype(dtype).name,
---> 60            ", ".join(dtypes.as_dtype(x).name for x in allowed_list)))
     61 
     62 

TypeError: Value passed to parameter 'shape' has DataType float32 not in list of allowed values: int32, int64
jonkrohn commented 5 years ago

You were correct, @tromgy -- I replicated your error, and have merged your change. Thank you very much for your pull request! 🙏

jonkrohn commented 5 years ago

Hey @tromgy ... out of curiosity, I don't suppose you're testing every Jupyter notebook in this repo (perhaps as you work through the book)? It would be tremendously helpful if you were because we're in the final round of book edits right now. This error you identified, for example, will be corrected within the text.

tromgy commented 5 years ago

Yes, @jonkrohn in fact I do go through every notebook mentioned in the text as I progress through the book. I'm now on chapter 12. Although for the notebooks from chapters prior to 10 I used code from the "old" github repository: https://github.com/illustrated-series/deep-learning-illustrated

jonkrohn commented 5 years ago

That's perfect, thanks 😃

tromgy commented 5 years ago

@jonkrohn, are issues disabled for this repo?

I'm seeing a problem in the generative_adversarial_network notebook. What is the best way to communicate it?

jonkrohn commented 5 years ago

Hi @tromgy ! I certainly haven't deliberately disabled issues. Indeed, I'm rather sure I haven't made any changes to the settings to this repo since the issue you filed last week.

tromgy commented 5 years ago

Last week it was a pull request, not an "issue". Issues are for problems that people see, but don't know how to solve. And that's exactly what I have now with the GAN notebook.

Here's how this repository looks: new-repo

the red arrow shows where the Issues tab would be.

And this is how the original repository looks:

old-repo

so somehow along the migration path the "issues" setting got lost...

In my own repos, when I click the Settings tab, I see this:

issues

so I think it's just the matter of checking this box.

jonkrohn commented 5 years ago

Awesome, @tromgy -- you are correct, as usual! The Issues tab should now be available to you :)