how to run your code?

[eeric]

Hi, I'm glad to appreciate your code, but I don't know that how to implement it. Such as benchmark_test.py, it occurs error, that is no module named 'models'. SRGAN model isn't provided, is it?

[titu1994]

@eeric models.py is there in the root folder. It contains the methods needed to create the SRResNet and SRGAN networks.

I am able to execute the benchmark test in PyCharm without any problem. Try adding the path of the root directory to the Python path.

[eeric]

What does it include to val_images folder? In addition, there is a error when making the order, python models.py, detail following,

Traceback (most recent call last): File "models.py", line 696, in srgan_model = srgan_network.build_srgan_model(use_small_srgan=False, use_small_discriminator=False) File "models.py", line 327, in build_srgan_model sr_output = self.generative_network.create_sr_model(ip) File "models.py", line 249, in create_sr_model x = self._residual_block(x, i + 1) File "models.py", line 270, in _residual_block x = BatchNormalization(axis=1, mode=self.mode, name="sr_resbatchnorm" + str(id) + "_1")(x) File "/usr/local/lib/python2.7/dist-packages/Keras-1.1.1-py2.7.egg/keras/engine/topology.py", line 514, in call self.add_inbound_node(inbound_layers, node_indices, tensor_indices) File "/usr/local/lib/python2.7/dist-packages/Keras-1.1.1-py2.7.egg/keras/engine/topology.py", line 572, in add_inbound_node Node.create_node(self, inbound_layers, node_indices, tensor_indices) File "/usr/local/lib/python2.7/dist-packages/Keras-1.1.1-py2.7.egg/keras/engine/topology.py", line 149, in create_node output_tensors = to_list(outbound_layer.call(input_tensors[0], mask=input_masks[0])) File "/usr/local/lib/python2.7/dist-packages/Keras-1.1.1-py2.7.egg/keras/layers/normalization.py", line 122, in call epsilon=self.epsilon) File "/usr/local/lib/python2.7/dist-packages/Keras-1.1.1-py2.7.egg/keras/backend/tensorflow_backend.py", line 700, in normalize_batch_in_training shift=None, name=None, keep_dims=False) TypeError: moments() got an unexpected keyword argument 'shift'

[titu1994]

@eeric This code supports only Theano backend at the moment with Theano dim ordering. That's why it's showing that error.

For valimages folder, run the three scripts called download*.py in each of the 3 dataset directories. It will download the validation images of set 5, set 14, and BSD 100

[eeric]

models.py(323) build_srgan_model() -> ip = Input(shape=(3, self.img_width, self.img_height), name='x_generator') <tf.Tensor 'x_generator:0' shape=(?, 3, 32, 32) dtype=float32>

models.py(327) build_srgan_model() -> sr_output = self.generative_network.create_sr_model(ip)

TypeError: "moments() got an unexpected keyword argument 'shift'"

[eeric]

It isn't provided to SRGAN model, so benchmark_test.py occur error. Then how to run models.py? How to load training data?

[titu1994]

@eeric As I said, this code is only for Theano backend with Python 3.4, tested on Keras 1.1.0. It's a work in progress. Calling python benchmark_test.py should work to run the code.

Instructions to run model.py are given in the Usage portion of the readme. As to training data, simply providing the path to the image data set will be enough.

[eeric]

python benchmark_test.py error following

python benchmark_test.py File "benchmark_test.py", line 32 def test_set5(model : Model, img_width=32, img_height=32, batch_size=1): ^ SyntaxError: invalid syntax

As to training data, simply providing the path to the image data set will be enough.

where is it to providing the path of training data in model.py code?

[titu1994]

@eeric inside model.py main declaration at the end of the file, there is a coco_path variable with an empty string. Put the path there. Also in the benchmark_test.py put the coco path at the end of the file in the coco_path variable.

For the invalid syntax, it may be due to the method definition. Remove : Model from the line def test_set5(model : Model, img_width=32, img_height=32, batch_size=1):

[eeric]

Thanks, according your advice, coco_path variable has new path, but it occur a error, following

1.python model.py

Using TensorFlow backend. I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcublas.so locally I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcudnn.so locally I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcufft.so locally I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcuda.so.1 locally I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcurand.so locally Traceback (most recent call last): File "models.py", line 709, in discriminator_network = srgan_network.build_discriminator_pretrain_model(use_small_discriminator=False) File "models.py", line 361, in build_discriminator_pretrain_model sr_output = self.generative_network.create_sr_model(ip) File "models.py", line 249, in create_sr_model x = self._residual_block(x, i + 1) File "models.py", line 270, in _residual_block x = BatchNormalization(axis=1, mode=self.mode, name="sr_resbatchnorm" + str(id) + "_1")(x) File "/usr/local/lib/python2.7/dist-packages/Keras-1.1.1-py2.7.egg/keras/engine/topology.py", line 514, in call self.add_inbound_node(inbound_layers, node_indices, tensor_indices) File "/usr/local/lib/python2.7/dist-packages/Keras-1.1.1-py2.7.egg/keras/engine/topology.py", line 572, in add_inbound_node Node.create_node(self, inbound_layers, node_indices, tensor_indices) File "/usr/local/lib/python2.7/dist-packages/Keras-1.1.1-py2.7.egg/keras/engine/topology.py", line 149, in create_node output_tensors = to_list(outbound_layer.call(input_tensors[0], mask=input_masks[0])) File "/usr/local/lib/python2.7/dist-packages/Keras-1.1.1-py2.7.egg/keras/layers/normalization.py", line 122, in call epsilon=self.epsilon) File "/usr/local/lib/python2.7/dist-packages/Keras-1.1.1-py2.7.egg/keras/backend/tensorflow_backend.py", line 700, in normalize_batch_in_training shift=None, name=None, keep_dims=False)

TypeError: moments() got an unexpected keyword argument 'shift'

2.> Remove : Model from the line def test_set5(model : Model, img_width=32, img_height=32, batch_size=1):

python benchmark_test.py File "benchmark_test.py", line 145 def build_model(self, load_weights=False) -> Model: ^ SyntaxError: invalid syntax

[eeric]

----python models.py

Using Theano backend. Loaded Generator weights Training SRGAN network () Epoch : 1 Found 0 images belonging to 0 classes. Traceback (most recent call last): File "models.py", line 710, in srgan_network.pre_train_discriminator(coco_path, nb_images=1895, nb_epochs=1) File "models.py", line 418, in pre_train_discriminator load_generative_weights=True) File "models.py", line 474, in _train_model target_size=(img_width, img_height)): File "/usr/local/lib/python2.7/dist-packages/Keras-1.1.1-py2.7.egg/keras/preprocessing/image.py", line 596, in next index_array, current_index, current_batch_size = next(self.index_generator) File "/usr/local/lib/python2.7/dist-packages/Keras-1.1.1-py2.7.egg/keras/preprocessing/image.py", line 444, in _flow_index current_index = (self.batch_index * batch_size) % N

ZeroDivisionError: integer division or modulo by zero

[titu1994]

@eeric You need to have a sub folder containing the images.

Say the folders are at /home/data/images/*.jpg, then provide the path as : /home/data

[titu1994]

@eeric Are you using image_dim_ordering="th" and not image_dim_ordering="th" in the ~/keras/keras.json file? As I said, this code does not work with Tensorflow and Tensorflow dim ordering.

[eeric]

It debugs successfully to model.py. Through readme is not very detailed, such as library is not clear, path of training data can't be know, thanks for your kind help.

[eeric]

----Remove : Model ----python benchmark_test.py

Traceback (most recent call last): File "benchmark_test.py", line 1, in import models ImportError: No module named models

[titu1994]

@eeric Well this is by no means a finished project so readme is incomplete.

I updated the benchmark scripts to correctly setup the path and import the modules. Try with the latest version. It was a problem with the way python loads modules if they don't exist in the current working directory.

[eeric]

--Remove 'model :’ from the line def test_set5(model : Model, img_width=32, img_height=32, batch_size=1): ----Remove '-> Model’ from the line def build_model(self, load_weights=False) -> Model:

-- python benchmark_test.py

Using Theano backend. Training SR ResNet network () Epoch : 1 Found 1895 images belonging to 1 classes. Iter : 1 / 1895 | Improvement : -0.00 percent | Time required : 329.73 seconds | PSNR : -43.567 Iter : 2 / 1895 | Improvement : -2.43 percent | Time required : 24.05 seconds | PSNR : -44.628 Iter : 3 / 1895 | Improvement : 3.35 percent | Time required : 23.41 seconds | PSNR : -43.132

[titu1994]

Run this code on the gpu. The SE ResNet is too large to train on the cpu efficiently.

Also, 2000 training images is far too small to train or pretrain the SE ResNet or SRGAN. It requires at least 5000 images to reach near +10 PSNR range, and a further 70000 images to hit the +20 PSNR range. This is mentioned in the readme. It's caused by blurred checkerboard artifacts.

I have already provided SR ResNet weights which have been pre trained on the MS COCO dataset, so you can just run validation on that.

[eeric]

In benchmark_test.py, As for 'coco_path' (= r"D:\Yue\Documents\Dataset\coco2014\train2014"), did it be use to train SR ResNet network? so if no training, how to used sr_resnet_weights.h5(that is SR ResNet weights provided ) to load 'sr_resnet_test.model' in line test_set5(sr_resnet_test.model, img_width=img_width, img_height=img_height)?

[titu1994]

In benchnark_test.py, in the if main part, comment out the line sr_resnet_test.train_model(coco_path, nb_images=50000, nb_epochs=1).

The path that is there is the path I used to train the model. You don't need Coco path anymore.

To load the pre trained model, make load_weights=True in the sr_resnet_test.build_model(load_weights=False) line

[eeric]

Thank you,

Through your above advice, it occured below, --python benchmark_test.py Using Theano backend. SR ResNet model weights loaded. Testing model on Set 5 Validation images Traceback (most recent call last): File "benchmark_test.py", line 299, in test_set5(sr_resnet_test.model, img_width=img_width, img_height=img_height) File "benchmark_test.py", line 45, in test_set5 model, total_psnr, "set5", 5) NameError: global name 'model' is not defined

( from: img_width = img_height = 96 sr_resnet_test = SRResNetTest(img_width=img_width, img_height=img_height, batch_size=1) sr_resnet_test.build_model(load_weights=True)

plot(sr_resnet_test.model, to_file='sr_resnet.png', show_shapes=True)

#sr_resnet_test.train_model(coco_path, nb_images=1895, nb_epochs=1)
test_set5(sr_resnet_test.model, img_width=img_width, img_height=img_height)

)

In paper: Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network, the bicubic kernel is for bicubic interpolation when making the training data, does it?

[eeric]

python benchmark_test.py Using Theano backend. Using gpu device 0: GeForce GTX 980 Ti (CNMeM is disabled, cuDNN 4007) SR ResNet model weights loaded. Testing model on Set 5 Validation images Traceback (most recent call last): File "benchmark_test.py", line 299, in test_set5(sr_resnet_test.model, img_width=img_width, img_height=img_height) File "benchmark_test.py", line 45, in test_set5 model, total_psnr, "set5", 5) NameError: global name 'model' is not defined