search

deepfakes / faceswap-playground

User dedicated repo for the faceswap project
305 stars 194 forks source link

train crash #272

Closed xiepengchong closed 5 years ago

xiepengchong commented 5 years ago

Expected behavior

Training process crash.

Actual behavior

root@deepfakes-cpu:/srv# python faceswap.py train -A liudehua_dest -B chenglong_dest -m ./models/ 04/01/2019 13:05:12 INFO Log level set to: INFO Using TensorFlow backend. 04/01/2019 13:05:15 INFO Model A Directory: /srv/liudehua_dest 04/01/2019 13:05:15 INFO Model B Directory: /srv/chenglong_dest 04/01/2019 13:05:15 INFO Training data directory: /srv/models 04/01/2019 13:05:15 INFO =============================================== 04/01/2019 13:05:15 INFO - Starting - 04/01/2019 13:05:15 INFO - Press 'ENTER' to save and quit - 04/01/2019 13:05:15 INFO - Press 'S' to save model weights immediately - 04/01/2019 13:05:15 INFO =============================================== 04/01/2019 13:05:16 INFO Loading data, this may take a while... 04/01/2019 13:05:16 INFO Loading Model from Original plugin... 04/01/2019 13:05:23 INFO Loading config: '/srv/config/train.ini' 04/01/2019 13:05:23 WARNING No existing state file found. Generating. 04/01/2019 13:05:24 INFO Creating new 'original' model in folder: '/srv/models' 04/01/2019 13:05:25 INFO Loading Trainer from Original plugin... 04/01/2019 13:05:26 INFO Enabled TensorBoard Logging Exception in thread Thread-1: Traceback (most recent call last): File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner self.run() File "/usr/lib/python3.5/threading.py", line 862, in run self._target(*self._args, **self._kwargs) File "/usr/lib/python3.5/logging/handlers.py", line 1431, in _monitor record = self.dequeue(True) File "/usr/lib/python3.5/logging/handlers.py", line 1380, in dequeue return self.queue.get(block) File "", line 2, in get File "/usr/lib/python3.5/multiprocessing/managers.py", line 717, in _callmethod kind, result = conn.recv() File "/usr/lib/python3.5/multiprocessing/connection.py", line 250, in recv buf = self._recv_bytes() File "/usr/lib/python3.5/multiprocessing/connection.py", line 407, in _recv_bytes buf = self._recv(4) File "/usr/lib/python3.5/multiprocessing/connection.py", line 383, in _recv raise EOFError EOFError

Killed

Steps to reproduce

this is my Mac os,when I run python faceswap.py train -A liudehua_dest -B chenglong_dest -m ./models/ happened this error

Other relevant information

torzdf commented 5 years ago

Crash_report from you faceswap folder

xiepengchong commented 5 years ago

there is no crash_report in my faceswap folder,but I add command of trace,I found mabey it is my memory is low, but I don't know how to resolve it still

python3 faceswap.py train -A liudehua_dest/ -B zhoujielun_dest -m ./models/ -L TRACE --trainer villain --batch-size 2 --save-interval 20

root@deepfakes-cpu:/srv# python3 faceswap.py train -A liudehua_dest/ -B zhoujielun_dest -m ./models/ -L TRACE --trainer villain --save-interval 20 04/05/2019 04:33:06 INFO Log level set to: TRACE Using TensorFlow backend. 04/05/2019 04:33:08 INFO Model A Directory: /srv/liudehua_dest 04/05/2019 04:33:08 INFO Model B Directory: /srv/zhoujielun_dest 04/05/2019 04:33:08 INFO Training data directory: /srv/models 04/05/2019 04:33:08 INFO =============================================== 04/05/2019 04:33:08 INFO - Starting - 04/05/2019 04:33:08 INFO - Press 'ENTER' to save and quit - 04/05/2019 04:33:08 INFO - Press 'S' to save model weights immediately - 04/05/2019 04:33:08 INFO =============================================== 04/05/2019 04:33:09 INFO Loading data, this may take a while... 04/05/2019 04:33:09 INFO Loading Model from Villain plugin... 04/05/2019 04:33:15 INFO Loading config: '/srv/config/train.ini' 04/05/2019 04:33:15 WARNING No existing state file found. Generating. 2019-04-05 04:33:15.832424: I tensorflow/core/platform/cpu_featureguard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2019-04-05 04:33:17.608853: W tensorflow/core/framework/allocator.cc:122] Allocation of 268435456 exceeds 10% of system memory. 04/05/2019 04:33:18 INFO Creating new 'villain' model in folder: '/srv/models' 04/05/2019 04:33:18 VERBOSE [Villain A Summary]: 04/05/2019 04:33:18 VERBOSE ____ 04/05/2019 04:33:18 VERBOSE Layer (type) Output Shape Param #
04/05/2019 04:33:18 VERBOSE ================================================================= 04/05/2019 04:33:18 VERBOSE face (InputLayer) (None, 128, 128, 3) 0
04/05/2019 04:33:18 VERBOSE 04/05/2019 04:33:18 VERBOSE encoder (Model) (None, 16, 16, 512) 112027904 04/05/2019 04:33:18 VERBOSE 04/05/2019 04:33:18 VERBOSE decodera (Model) (None, 128, 128, 3) 21543555
04/05/2019 04:33:18 VERBOSE ================================================================= 04/05/2019 04:33:18 VERBOSE Total params: 133,571,459 04/05/2019 04:33:18 VERBOSE Trainable params: 133,571,459 04/05/2019 04:33:18 VERBOSE Non-trainable params: 0 04/05/2019 04:33:18 VERBOSE ____ 04/05/2019 04:33:18 VERBOSE Encoder: 04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE Layer (type) Output Shape Param # Connected to
04/05/2019 04:33:18 VERBOSE ================================================================================================== 04/05/2019 04:33:18 VERBOSE input_3 (InputLayer) (None, 128, 128, 3) 0
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_21 (Conv2D) (None, 64, 64, 128) 9728 input_3[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_19 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_21[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_22 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_19[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_20 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_22[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_23 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_20[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_7 (Add) (None, 64, 64, 128) 0 conv2d_23[0][0]
04/05/2019 04:33:18 VERBOSE conv2d_21[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_21 (LeakyReLU) (None, 64, 64, 128) 0 add_7[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_22 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_21[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_24 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_22[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_23 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_24[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_25 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_23[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_8 (Add) (None, 64, 64, 128) 0 conv2d_25[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_21[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_24 (LeakyReLU) (None, 64, 64, 128) 0 add_8[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_25 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_24[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_26 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_25[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_26 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_26[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_27 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_26[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_9 (Add) (None, 64, 64, 128) 0 conv2d_27[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_24[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_27 (LeakyReLU) (None, 64, 64, 128) 0 add_9[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_28 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_27[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_28 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_28[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_29 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_28[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_29 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_29[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_10 (Add) (None, 64, 64, 128) 0 conv2d_29[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_27[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_30 (LeakyReLU) (None, 64, 64, 128) 0 add_10[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_31 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_30[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_30 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_31[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_32 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_30[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_31 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_32[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_11 (Add) (None, 64, 64, 128) 0 conv2d_31[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_30[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_33 (LeakyReLU) (None, 64, 64, 128) 0 add_11[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_34 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_33[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_32 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_34[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_35 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_32[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_33 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_35[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_12 (Add) (None, 64, 64, 128) 0 conv2d_33[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_33[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_36 (LeakyReLU) (None, 64, 64, 128) 0 add_12[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_37 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_36[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_34 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_37[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_38 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_34[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_35 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_38[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_13 (Add) (None, 64, 64, 128) 0 conv2d_35[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_36[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_39 (LeakyReLU) (None, 64, 64, 128) 0 add_13[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_40 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_39[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_36 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_40[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_41 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_36[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_37 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_41[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_14 (Add) (None, 64, 64, 128) 0 conv2d_37[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_39[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_42 (LeakyReLU) (None, 64, 64, 128) 0 add_14[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_43 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_42[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_38 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_43[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_44 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_38[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_39 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_44[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_15 (Add) (None, 64, 64, 128) 0 conv2d_39[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_42[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_45 (LeakyReLU) (None, 64, 64, 128) 0 add_15[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_46 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_45[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_40 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_46[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_47 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_40[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_41 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_47[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_16 (Add) (None, 64, 64, 128) 0 conv2d_41[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_45[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_48 (LeakyReLU) (None, 64, 64, 128) 0 add_16[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_49 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_48[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_42 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_49[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_50 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_42[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_43 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_50[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_17 (Add) (None, 64, 64, 128) 0 conv2d_43[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_48[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_51 (LeakyReLU) (None, 64, 64, 128) 0 add_17[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_52 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_51[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_44 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_52[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_53 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_44[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_45 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_53[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_18 (Add) (None, 64, 64, 128) 0 conv2d_45[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_51[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_54 (LeakyReLU) (None, 64, 64, 128) 0 add_18[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_55 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_54[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_46 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_55[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_56 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_46[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_47 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_56[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_19 (Add) (None, 64, 64, 128) 0 conv2d_47[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_54[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_57 (LeakyReLU) (None, 64, 64, 128) 0 add_19[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_58 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_57[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_48 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_58[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_59 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_48[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_49 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_59[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_20 (Add) (None, 64, 64, 128) 0 conv2d_49[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_57[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_60 (LeakyReLU) (None, 64, 64, 128) 0 add_20[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_61 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_60[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_50 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_61[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_62 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_50[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_51 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_62[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_21 (Add) (None, 64, 64, 128) 0 conv2d_51[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_60[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_63 (LeakyReLU) (None, 64, 64, 128) 0 add_21[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_64 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_63[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_52 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_64[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_65 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_52[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_53 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_65[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_22 (Add) (None, 64, 64, 128) 0 conv2d_53[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_63[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_66 (LeakyReLU) (None, 64, 64, 128) 0 add_22[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_23 (Add) (None, 64, 64, 128) 0 leaky_re_lu_66[0][0]
04/05/2019 04:33:18 VERBOSE conv2d_21[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_54 (Conv2D) (None, 32, 32, 128) 409728 add_23[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_67 (LeakyReLU) (None, 32, 32, 128) 0 conv2d_54[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE pixel_shuffler_7 (PixelShuffler (None, 64, 64, 32) 0 leaky_re_lu_67[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_55 (Conv2D) (None, 32, 32, 128) 102528 pixel_shuffler_7[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_68 (LeakyReLU) (None, 32, 32, 128) 0 conv2d_55[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE pixel_shuffler_8 (PixelShuffler (None, 64, 64, 32) 0 leaky_re_lu_68[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_56 (Conv2D) (None, 32, 32, 128) 102528 pixel_shuffler_8[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_69 (LeakyReLU) (None, 32, 32, 128) 0 conv2d_56[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE separable_conv2d_1 (SeparableCo (None, 16, 16, 256) 36224 leaky_re_lu_69[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE activation_1 (Activation) (None, 16, 16, 256) 0 separable_conv2d_1[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_57 (Conv2D) (None, 8, 8, 512) 3277312 activation_1[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_70 (LeakyReLU) (None, 8, 8, 512) 0 conv2d_57[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE separable_conv2d_2 (SeparableCo (None, 4, 4, 1024) 538112 leaky_re_lu_70[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE activation_2 (Activation) (None, 4, 4, 1024) 0 separable_conv2d_2[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE flatten_1 (Flatten) (None, 16384) 0 activation_2[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE dense_1 (Dense) (None, 1024) 16778240 flatten_1[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE dense_2 (Dense) (None, 65536) 67174400 dense_1[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE reshape_1 (Reshape) (None, 8, 8, 1024) 0 dense_2[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_58 (Conv2D) (None, 8, 8, 2048) 18876416 reshape_1[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_71 (LeakyReLU) (None, 8, 8, 2048) 0 conv2d_58[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE pixel_shuffler_9 (PixelShuffler (None, 16, 16, 512) 0 leaky_re_lu_71[0][0]
04/05/2019 04:33:18 VERBOSE ================================================================================================== 04/05/2019 04:33:18 VERBOSE Total params: 112,027,904 04/05/2019 04:33:18 VERBOSE Trainable params: 112,027,904 04/05/2019 04:33:18 VERBOSE Non-trainable params: 0 04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE Decoder_A: 04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE Layer (type) Output Shape Param # Connected to
04/05/2019 04:33:18 VERBOSE ================================================================================================== 04/05/2019 04:33:18 VERBOSE input_1 (InputLayer) (None, 16, 16, 512) 0
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_1 (Conv2D) (None, 16, 16, 2048) 9439232 input_1[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE pixel_shuffler_1 (PixelShuffler (None, 32, 32, 512) 0 conv2d_1[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_1 (LeakyReLU) (None, 32, 32, 512) 0 pixel_shuffler_1[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_2 (Conv2D) (None, 32, 32, 512) 2359808 leaky_re_lu_1[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_2 (LeakyReLU) (None, 32, 32, 512) 0 conv2d_2[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_3 (Conv2D) (None, 32, 32, 512) 2359808 leaky_re_lu_2[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_1 (Add) (None, 32, 32, 512) 0 conv2d_3[0][0]
04/05/2019 04:33:18 VERBOSE pixel_shuffler_1[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_3 (LeakyReLU) (None, 32, 32, 512) 0 add_1[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_4 (Conv2D) (None, 32, 32, 1024) 4719616 leaky_re_lu_3[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE pixel_shuffler_2 (PixelShuffler (None, 64, 64, 256) 0 conv2d_4[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_4 (LeakyReLU) (None, 64, 64, 256) 0 pixel_shuffler_2[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_5 (Conv2D) (None, 64, 64, 256) 590080 leaky_re_lu_4[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_5 (LeakyReLU) (None, 64, 64, 256) 0 conv2d_5[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_6 (Conv2D) (None, 64, 64, 256) 590080 leaky_re_lu_5[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_2 (Add) (None, 64, 64, 256) 0 conv2d_6[0][0]
04/05/2019 04:33:18 VERBOSE pixel_shuffler_2[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_6 (LeakyReLU) (None, 64, 64, 256) 0 add_2[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_7 (Conv2D) (None, 64, 64, 512) 1180160 leaky_re_lu_6[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE pixel_shuffler_3 (PixelShuffler (None, 128, 128, 128 0 conv2d_7[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_7 (LeakyReLU) (None, 128, 128, 128 0 pixel_shuffler_3[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_8 (Conv2D) (None, 128, 128, 128 147584 leaky_re_lu_7[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_8 (LeakyReLU) (None, 128, 128, 128 0 conv2d_8[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_9 (Conv2D) (None, 128, 128, 128 147584 leaky_re_lu_8[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_3 (Add) (None, 128, 128, 128 0 conv2d_9[0][0]
04/05/2019 04:33:18 VERBOSE pixel_shuffler_3[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_9 (LeakyReLU) (None, 128, 128, 128 0 add_3[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_10 (Conv2D) (None, 128, 128, 3) 9603 leaky_re_lu_9[0][0]
04/05/2019 04:33:18 VERBOSE ================================================================================================== 04/05/2019 04:33:18 VERBOSE Total params: 21,543,555 04/05/2019 04:33:18 VERBOSE Trainable params: 21,543,555 04/05/2019 04:33:18 VERBOSE Non-trainable params: 0 04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE [Villain B Summary]: 04/05/2019 04:33:18 VERBOSE 04/05/2019 04:33:18 VERBOSE Layer (type) Output Shape Param #
04/05/2019 04:33:18 VERBOSE ================================================================= 04/05/2019 04:33:18 VERBOSE face (InputLayer) (None, 128, 128, 3) 0
04/05/2019 04:33:18 VERBOSE 04/05/2019 04:33:18 VERBOSE encoder (Model) (None, 16, 16, 512) 112027904 04/05/2019 04:33:18 VERBOSE _ 04/05/2019 04:33:18 VERBOSE decoderb (Model) (None, 128, 128, 3) 21543555
04/05/2019 04:33:18 VERBOSE ================================================================= 04/05/2019 04:33:18 VERBOSE Total params: 133,571,459 04/05/2019 04:33:18 VERBOSE Trainable params: 133,571,459 04/05/2019 04:33:18 VERBOSE Non-trainable params: 0 04/05/2019 04:33:18 VERBOSE ____ 04/05/2019 04:33:18 VERBOSE Encoder: 04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE Layer (type) Output Shape Param # Connected to
04/05/2019 04:33:18 VERBOSE ================================================================================================== 04/05/2019 04:33:18 VERBOSE input_3 (InputLayer) (None, 128, 128, 3) 0
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_21 (Conv2D) (None, 64, 64, 128) 9728 input_3[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_19 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_21[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_22 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_19[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_20 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_22[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_23 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_20[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_7 (Add) (None, 64, 64, 128) 0 conv2d_23[0][0]
04/05/2019 04:33:18 VERBOSE conv2d_21[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_21 (LeakyReLU) (None, 64, 64, 128) 0 add_7[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_22 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_21[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_24 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_22[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_23 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_24[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_25 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_23[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_8 (Add) (None, 64, 64, 128) 0 conv2d_25[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_21[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_24 (LeakyReLU) (None, 64, 64, 128) 0 add_8[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_25 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_24[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_26 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_25[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_26 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_26[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_27 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_26[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_9 (Add) (None, 64, 64, 128) 0 conv2d_27[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_24[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_27 (LeakyReLU) (None, 64, 64, 128) 0 add_9[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_28 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_27[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_28 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_28[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_29 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_28[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_29 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_29[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_10 (Add) (None, 64, 64, 128) 0 conv2d_29[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_27[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_30 (LeakyReLU) (None, 64, 64, 128) 0 add_10[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_31 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_30[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_30 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_31[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_32 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_30[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_31 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_32[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_11 (Add) (None, 64, 64, 128) 0 conv2d_31[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_30[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_33 (LeakyReLU) (None, 64, 64, 128) 0 add_11[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_34 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_33[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_32 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_34[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_35 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_32[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_33 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_35[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_12 (Add) (None, 64, 64, 128) 0 conv2d_33[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_33[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_36 (LeakyReLU) (None, 64, 64, 128) 0 add_12[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_37 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_36[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_34 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_37[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_38 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_34[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_35 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_38[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_13 (Add) (None, 64, 64, 128) 0 conv2d_35[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_36[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_39 (LeakyReLU) (None, 64, 64, 128) 0 add_13[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_40 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_39[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_36 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_40[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_41 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_36[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_37 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_41[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_14 (Add) (None, 64, 64, 128) 0 conv2d_37[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_39[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_42 (LeakyReLU) (None, 64, 64, 128) 0 add_14[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_43 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_42[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_38 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_43[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_44 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_38[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_39 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_44[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_15 (Add) (None, 64, 64, 128) 0 conv2d_39[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_42[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_45 (LeakyReLU) (None, 64, 64, 128) 0 add_15[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_46 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_45[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_40 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_46[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_47 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_40[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_41 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_47[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_16 (Add) (None, 64, 64, 128) 0 conv2d_41[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_45[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_48 (LeakyReLU) (None, 64, 64, 128) 0 add_16[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_49 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_48[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_42 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_49[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_50 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_42[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_43 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_50[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_17 (Add) (None, 64, 64, 128) 0 conv2d_43[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_48[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_51 (LeakyReLU) (None, 64, 64, 128) 0 add_17[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_52 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_51[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_44 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_52[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_53 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_44[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_45 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_53[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_18 (Add) (None, 64, 64, 128) 0 conv2d_45[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_51[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_54 (LeakyReLU) (None, 64, 64, 128) 0 add_18[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_55 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_54[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_46 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_55[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_56 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_46[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_47 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_56[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_19 (Add) (None, 64, 64, 128) 0 conv2d_47[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_54[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_57 (LeakyReLU) (None, 64, 64, 128) 0 add_19[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_58 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_57[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_48 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_58[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_59 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_48[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_49 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_59[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_20 (Add) (None, 64, 64, 128) 0 conv2d_49[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_57[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_60 (LeakyReLU) (None, 64, 64, 128) 0 add_20[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_61 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_60[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_50 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_61[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_62 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_50[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_51 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_62[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_21 (Add) (None, 64, 64, 128) 0 conv2d_51[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_60[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_63 (LeakyReLU) (None, 64, 64, 128) 0 add_21[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_64 (LeakyReLU) (None, 64, 64, 128) 0 leaky_re_lu_63[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_52 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_64[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_65 (LeakyReLU) (None, 64, 64, 128) 0 conv2d_52[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_53 (Conv2D) (None, 64, 64, 128) 147584 leaky_re_lu_65[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_22 (Add) (None, 64, 64, 128) 0 conv2d_53[0][0]
04/05/2019 04:33:18 VERBOSE leaky_re_lu_63[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_66 (LeakyReLU) (None, 64, 64, 128) 0 add_22[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE add_23 (Add) (None, 64, 64, 128) 0 leaky_re_lu_66[0][0]
04/05/2019 04:33:18 VERBOSE conv2d_21[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_54 (Conv2D) (None, 32, 32, 128) 409728 add_23[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_67 (LeakyReLU) (None, 32, 32, 128) 0 conv2d_54[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE pixel_shuffler_7 (PixelShuffler (None, 64, 64, 32) 0 leaky_re_lu_67[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_55 (Conv2D) (None, 32, 32, 128) 102528 pixel_shuffler_7[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_68 (LeakyReLU) (None, 32, 32, 128) 0 conv2d_55[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE pixel_shuffler_8 (PixelShuffler (None, 64, 64, 32) 0 leaky_re_lu_68[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_56 (Conv2D) (None, 32, 32, 128) 102528 pixel_shuffler_8[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_69 (LeakyReLU) (None, 32, 32, 128) 0 conv2d_56[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE separable_conv2d_1 (SeparableCo (None, 16, 16, 256) 36224 leaky_re_lu_69[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE activation_1 (Activation) (None, 16, 16, 256) 0 separable_conv2d_1[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE conv2d_57 (Conv2D) (None, 8, 8, 512) 3277312 activation_1[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE leaky_re_lu_70 (LeakyReLU) (None, 8, 8, 512) 0 conv2d_57[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE separable_conv2d_2 (SeparableCo (None, 4, 4, 1024) 538112 leaky_re_lu_70[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE activation_2 (Activation) (None, 4, 4, 1024) 0 separable_conv2d_2[0][0]
04/05/2019 04:33:18 VERBOSE __ 04/05/2019 04:33:18 VERBOSE flatten_1 (Flatten) (None, 16384) 0 activation_2[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE dense_1 (Dense) (None, 1024) 16778240 flatten_1[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE dense_2 (Dense) (None, 65536) 67174400 dense_1[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE reshape_1 (Reshape) (None, 8, 8, 1024) 0 dense_2[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE conv2d_58 (Conv2D) (None, 8, 8, 2048) 18876416 reshape_1[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE leaky_re_lu_71 (LeakyReLU) (None, 8, 8, 2048) 0 conv2d_58[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE pixel_shuffler_9 (PixelShuffler (None, 16, 16, 512) 0 leaky_re_lu_71[0][0]
04/05/2019 04:33:19 VERBOSE ================================================================================================== 04/05/2019 04:33:19 VERBOSE Total params: 112,027,904 04/05/2019 04:33:19 VERBOSE Trainable params: 112,027,904 04/05/2019 04:33:19 VERBOSE Non-trainable params: 0 04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE Decoder_B: 04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE Layer (type) Output Shape Param # Connected to
04/05/2019 04:33:19 VERBOSE ================================================================================================== 04/05/2019 04:33:19 VERBOSE input_2 (InputLayer) (None, 16, 16, 512) 0
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE conv2d_11 (Conv2D) (None, 16, 16, 2048) 9439232 input_2[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE pixel_shuffler_4 (PixelShuffler (None, 32, 32, 512) 0 conv2d_11[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE leaky_re_lu_10 (LeakyReLU) (None, 32, 32, 512) 0 pixel_shuffler_4[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE conv2d_12 (Conv2D) (None, 32, 32, 512) 2359808 leaky_re_lu_10[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE leaky_re_lu_11 (LeakyReLU) (None, 32, 32, 512) 0 conv2d_12[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE conv2d_13 (Conv2D) (None, 32, 32, 512) 2359808 leaky_re_lu_11[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE add_4 (Add) (None, 32, 32, 512) 0 conv2d_13[0][0]
04/05/2019 04:33:19 VERBOSE pixel_shuffler_4[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE leaky_re_lu_12 (LeakyReLU) (None, 32, 32, 512) 0 add_4[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE conv2d_14 (Conv2D) (None, 32, 32, 1024) 4719616 leaky_re_lu_12[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE pixel_shuffler_5 (PixelShuffler (None, 64, 64, 256) 0 conv2d_14[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE leaky_re_lu_13 (LeakyReLU) (None, 64, 64, 256) 0 pixel_shuffler_5[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE conv2d_15 (Conv2D) (None, 64, 64, 256) 590080 leaky_re_lu_13[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE leaky_re_lu_14 (LeakyReLU) (None, 64, 64, 256) 0 conv2d_15[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE conv2d_16 (Conv2D) (None, 64, 64, 256) 590080 leaky_re_lu_14[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE add_5 (Add) (None, 64, 64, 256) 0 conv2d_16[0][0]
04/05/2019 04:33:19 VERBOSE pixel_shuffler_5[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE leaky_re_lu_15 (LeakyReLU) (None, 64, 64, 256) 0 add_5[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE conv2d_17 (Conv2D) (None, 64, 64, 512) 1180160 leaky_re_lu_15[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE pixel_shuffler_6 (PixelShuffler (None, 128, 128, 128 0 conv2d_17[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE leaky_re_lu_16 (LeakyReLU) (None, 128, 128, 128 0 pixel_shuffler_6[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE conv2d_18 (Conv2D) (None, 128, 128, 128 147584 leaky_re_lu_16[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE leaky_re_lu_17 (LeakyReLU) (None, 128, 128, 128 0 conv2d_18[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE conv2d_19 (Conv2D) (None, 128, 128, 128 147584 leaky_re_lu_17[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE add_6 (Add) (None, 128, 128, 128 0 conv2d_19[0][0]
04/05/2019 04:33:19 VERBOSE pixel_shuffler_6[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE leaky_re_lu_18 (LeakyReLU) (None, 128, 128, 128 0 add_6[0][0]
04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE conv2d_20 (Conv2D) (None, 128, 128, 3) 9603 leaky_re_lu_18[0][0]
04/05/2019 04:33:19 VERBOSE ================================================================================================== 04/05/2019 04:33:19 VERBOSE Total params: 21,543,555 04/05/2019 04:33:19 VERBOSE Trainable params: 21,543,555 04/05/2019 04:33:19 VERBOSE Non-trainable params: 0 04/05/2019 04:33:19 VERBOSE __ 04/05/2019 04:33:19 VERBOSE Using Mean Absolute Error Loss 04/05/2019 04:33:19 INFO Loading Trainer from Original plugin... 04/05/2019 04:33:21 INFO Enabled TensorBoard Logging 2019-04-05 04:33:31.044569: W tensorflow/core/framework/allocator.cc:122] Allocation of 268435456 exceeds 10% of system memory. 2019-04-05 04:33:31.353580: W tensorflow/core/framework/allocator.cc:122] Allocation of 268435456 exceeds 10% of system memory. 2019-04-05 04:33:31.621899: W tensorflow/core/framework/allocator.cc:122] Allocation of 268435456 exceeds 10% of system memory. 2019-04-05 04:33:31.868991: W tensorflow/core/framework/allocator.cc:122] Allocation of 268435456 exceeds 10% of system memory. Killed

xiepengchong commented 5 years ago

even I use the command option of "memory saving gradients",it can not work still

torzdf commented 5 years ago

Villain is a very heavy model and may not train on CPU. You should try another model, or try lowering the batchsize right down

torzdf commented 5 years ago

memory saving gradients is only a GPU feature.

xiepengchong commented 5 years ago

Villain is a very heavy model and may not train on CPU. You should try another model, or try lowering the batchsize right down

thanks for your reply. I change the model to dfaker and I got error like below, my command is: python3 faceswap.py train -A liudehua_dest/ -B zhoujielun_dest -m ./models/ -p --trainer dfaker --batch-size 2 --save-interval 20

04/06/2019 12:09:13 MainProcess training_0 _base VERBOSE __ 04/06/2019 12:09:13 MainProcess training_0 _base VERBOSE pixel_shuffler_16 (PixelShuffle (None, 128, 128, 64) 0 leaky_re_lu_28[0][0] 04/06/2019 12:09:13 MainProcess training_0 _base VERBOSE __ 04/06/2019 12:09:13 MainProcess training_0 _base VERBOSE conv2d_27 (Conv2D) (None, 128, 128, 3) 4803 pixel_shuffler_12[0][0] 04/06/2019 12:09:13 MainProcess training_0 _base VERBOSE __ 04/06/2019 12:09:13 MainProcess training_0 _base VERBOSE conv2d_32 (Conv2D) (None, 128, 128, 1) 1601 pixel_shuffler_16[0][0] 04/06/2019 12:09:13 MainProcess training_0 _base VERBOSE ================================================================================================== 04/06/2019 12:09:13 MainProcess training_0 _base VERBOSE Total params: 37,469,700 04/06/2019 12:09:13 MainProcess training_0 _base VERBOSE Trainable params: 37,469,700 04/06/2019 12:09:13 MainProcess training_0 _base VERBOSE Non-trainable params: 0 04/06/2019 12:09:13 MainProcess training_0 _base VERBOSE __ 04/06/2019 12:09:13 MainProcess training_0 _base compile_predictors DEBUG Compiling Predictors 04/06/2019 12:09:13 MainProcess training_0 _base get_optimizer DEBUG Optimizer kwargs: {'beta_2': 0.999, 'beta_1': 0.5, 'lr': 5e-05} 04/06/2019 12:09:13 MainProcess training_0 _base loss_function VERBOSE Using DSSIM Loss 04/06/2019 12:09:13 MainProcess training_0 _base loss_function VERBOSE Penalizing mask for Loss 04/06/2019 12:09:13 MainProcess training_0 _base mask_loss_function VERBOSE Using Mean Squared Error Loss for mask 04/06/2019 12:09:13 MainProcess training_0 _base add_session_loss_names DEBUG Adding session loss_names. (side: 'a', loss_names: ['total_loss', 'loss', 'mask_loss'] 04/06/2019 12:09:14 MainProcess training_0 _base add_session_loss_names DEBUG Adding session loss_names. (side: 'b', loss_names: ['total_loss', 'loss', 'mask_loss'] 04/06/2019 12:09:14 MainProcess training_0 _base compile_predictors DEBUG Compiled Predictors. Losses: ['total_loss', 'loss', 'mask_loss'] 04/06/2019 12:09:14 MainProcess training_0 _base set_training_data DEBUG Setting training data 04/06/2019 12:09:14 MainProcess training_0 _base calculate_coverage_ratio DEBUG Requested coverage_ratio: 1.0 04/06/2019 12:09:14 MainProcess training_0 _base calculate_coverage_ratio DEBUG Final coverage_ratio: 1.0 04/06/2019 12:09:14 MainProcess training_0 _base set_training_data DEBUG Set training data: {'coverage_ratio': 1.0, 'no_flip': False, 'preview_images': 14, 'no_logs': False, 'pingpong': False, 'warp_to_landmarks': False, 'training_size': 256, 'mask_type': 'dfaker', 'preview_scaling': 0.5, 'alignments': {'a': '/srv/liudehua_dest/alignments.json', 'b': '/srv/zhoujielun_dest/alignments.json'}} 04/06/2019 12:09:14 MainProcess training_0 _base init DEBUG Initialized ModelBase (Model) 04/06/2019 12:09:14 MainProcess training_0 original init DEBUG Initialized Model 04/06/2019 12:09:14 MainProcess training_0 dfaker init DEBUG Initialized Model 04/06/2019 12:09:14 MainProcess training_0 train load_model DEBUG Loaded Model 04/06/2019 12:09:14 MainProcess training_0 train load_trainer DEBUG Loading Trainer 04/06/2019 12:09:14 MainProcess training_0 plugin_loader _import INFO Loading Trainer from Original plugin... 04/06/2019 12:09:14 MainProcess training_0 _base init DEBUG Initializing TrainerBase: (model: '<plugins.train.model.dfaker.Model object at 0x7f48ac13ca90>', batch_size: 2) 04/06/2019 12:09:14 MainProcess training_0 _base add_session_batchsize DEBUG Adding session batchsize: 2 04/06/2019 12:09:14 MainProcess training_0 _base process_training_opts DEBUG {'coverage_ratio': 1.0, 'no_flip': False, 'preview_images': 14, 'no_logs': False, 'pingpong': False, 'warp_to_landmarks': False, 'training_size': 256, 'mask_type': 'dfaker', 'preview_scaling': 0.5, 'alignments': {'a': '/srv/liudehua_dest/alignments.json', 'b': '/srv/zhoujielun_dest/alignments.json'}} 04/06/2019 12:09:14 MainProcess training_0 _base landmarks_required DEBUG True 04/06/2019 12:09:14 MainProcess training_0 _base init DEBUG Initializing Landmarks: (training_opts: '{'coverage_ratio': 1.0, 'no_flip': False, 'preview_images': 14, 'no_logs': False, 'pingpong': False, 'warp_to_landmarks': False, 'training_size': 256, 'mask_type': 'dfaker', 'preview_scaling': 0.5, 'alignments': {'a': '/srv/liudehua_dest/alignments.json', 'b': '/srv/zhoujielun_dest/alignments.json'}}') 04/06/2019 12:09:14 MainProcess training_0 alignments init DEBUG Initializing Alignments: (folder: '/srv/liudehua_dest', filename: 'alignments', serializer: 'json') 04/06/2019 12:09:14 MainProcess training_0 alignments get_serializer DEBUG Getting serializer: (filename: 'alignments', serializer: 'json') 04/06/2019 12:09:14 MainProcess training_0 alignments get_serializer DEBUG Serializer set from argument: 'json' 04/06/2019 12:09:14 MainProcess training_0 alignments get_serializer VERBOSE Using 'json' serializer for alignments 04/06/2019 12:09:14 MainProcess training_0 alignments get_location DEBUG Getting location: (folder: '/srv/liudehua_dest', filename: 'alignments') 04/06/2019 12:09:14 MainProcess training_0 alignments get_location DEBUG File extension set from serializer: 'json' 04/06/2019 12:09:14 MainProcess training_0 alignments get_location VERBOSE Alignments filepath: '/srv/liudehua_dest/alignments.json' 04/06/2019 12:09:14 MainProcess training_0 alignments load DEBUG Loading alignments 04/06/2019 12:09:14 MainProcess training_0 multithreading run DEBUG Error in thread (training_0): Error: Alignments file not found at /srv/liudehua_dest/alignments.json 04/06/2019 12:09:14 MainProcess MainThread train monitor DEBUG Thread error detected 04/06/2019 12:09:14 MainProcess MainThread train monitor DEBUG Closed Monitor 04/06/2019 12:09:14 MainProcess MainThread train end_thread DEBUG Ending Training thread 04/06/2019 12:09:14 MainProcess MainThread train end_thread CRITICAL Error caught! Exiting... 04/06/2019 12:09:14 MainProcess MainThread multithreading join DEBUG Joining Threads: 'training' 04/06/2019 12:09:14 MainProcess MainThread multithreading join DEBUG Joining Thread: 'training_0' 04/06/2019 12:09:14 MainProcess MainThread multithreading join ERROR Caught exception in thread: 'training_0' Traceback (most recent call last): File "/srv/lib/cli.py", line 107, in execute_script process.process() File "/srv/scripts/train.py", line 98, in process self.end_thread(thread, err) File "/srv/scripts/train.py", line 123, in end_thread thread.join() File "/srv/lib/multithreading.py", line 443, in join raise thread.err[1].with_traceback(thread.err[2]) File "/srv/lib/multithreading.py", line 381, in run self._target(*self._args, **self._kwargs) File "/srv/scripts/train.py", line 149, in training raise err File "/srv/scripts/train.py", line 138, in training trainer = self.load_trainer(model) File "/srv/scripts/train.py", line 197, in load_trainer self.args.batch_size) File "/srv/plugins/train/trainer/_base.py", line 51, in init self.process_training_opts() File "/srv/plugins/train/trainer/_base.py", line 96, in process_training_opts landmarks = Landmarks(self.model.training_opts).landmarks File "/srv/plugins/train/trainer/_base.py", line 604, in init self.landmarks = self.get_alignments() File "/srv/plugins/train/trainer/_base.py", line 617, in get_alignments serializer=serializer) File "/srv/lib/alignments.py", line 36, in init self.data = self.load() File "/srv/lib/alignments.py", line 121, in load "{}".format(self.file)) ValueError: Error: Alignments file not found at /srv/liudehua_dest/alignments.json

============ System Information ============ encoding: ANSI_X3.4-1968 git_branch: Not Found git_commits: Not Found gpu_cuda: Not Found gpu_cudnn: Not Found gpu_devices:
gpu_driver: No Nvidia driver found gpu_vram:
os_machine: x86_64 os_platform: Linux-4.9.125-linuxkit-x86_64-with-Ubuntu-16.04-xenial os_release: 4.9.125-linuxkit py_command: faceswap.py train -A liudehua_dest/ -B zhoujielun_dest -m ./models/ -p --trainer dfaker --batch-size 2 --save-interval 20 py_conda_version: N/A py_implementation: CPython py_version: 3.5.2 py_virtual_env: False sys_cores: 4 sys_processor: x86_64 sys_ram: Total: 1998MB, Available: 472MB, Used: 1382MB, Free: 312MB

=============== Pip Packages =============== absl-py==0.6.1 astor==0.7.1 backcall==0.1.0 bleach==3.0.2 Click==7.0 cmake==3.13.3 cycler==0.10.0 decorator==4.3.0 defusedxml==0.5.0 dlib==19.17.0 entrypoints==0.2.3 face-recognition==1.2.3 face-recognition-models==0.3.0 ffmpy==0.2.2 gast==0.2.0 grpcio==1.16.0 h5py==2.9.0 imageio==2.5.0 ipykernel==5.1.0 ipython==7.1.1 ipython-genutils==0.2.0 ipywidgets==7.4.2 jedi==0.13.1 Jinja2==2.10 jsonschema==2.6.0 jupyter==1.0.0 jupyter-client==5.2.3 jupyter-console==6.0.0 jupyter-core==4.4.0 Keras==2.2.4 Keras-Applications==1.0.6 Keras-Preprocessing==1.0.5 kiwisolver==1.0.1 Markdown==3.0.1 MarkupSafe==1.1.0 matplotlib==2.2.2 mistune==0.8.4 nbconvert==5.4.0 nbformat==4.4.0 networkx==2.2 notebook==5.7.0 numpy==1.15.4 nvidia-ml-py3==7.352.0 opencv-python==4.0.0.21 pandas==0.23.4 pandocfilters==1.4.2 parso==0.3.1 pathlib==1.0.1 pexpect==4.6.0 pickleshare==0.7.5 Pillow==5.3.0 prometheus-client==0.4.2 prompt-toolkit==2.0.7 protobuf==3.6.1 psutil==5.6.1 ptyprocess==0.6.0 pycurl==7.43.0 Pygments==2.2.0 pygobject==3.20.0 pyparsing==2.3.0 python-apt==1.1.0b1+ubuntu0.16.4.2 python-dateutil==2.7.5 pytz==2018.7 PyWavelets==1.0.2 PyYAML==5.1 pyzmq==17.1.2 qtconsole==4.4.2 scikit-image==0.15.0 scikit-learn==0.20.0 scipy==1.1.0 Send2Trash==1.5.0 six==1.11.0 sklearn==0.0 tensorboard==1.12.0 tensorflow==1.12.0 termcolor==1.1.0 terminado==0.8.1 testpath==0.4.2 toposort==1.5 tornado==5.1.1 tqdm==4.31.1 traitlets==4.3.2 wcwidth==0.1.7 webencodings==0.5.1 Werkzeug==0.14.1 widgetsnbextension==3.4.2

xiepengchong commented 5 years ago

I extract like belows python faceswap.py extract -i ./chenglong/ -o ./chenglong_dest but I generate the alignments.json in chenglong folder,there is no alignments.json file in chenglong_dest, and I move the alignments.json from chenglong to chenglong_dest,it still error

torzdf commented 5 years ago

ValueError: Error: Alignments file not found at /srv/liudehua_dest/alignments.json The alignments file must be in /srv/liudehua_dest/alignments.json

If you're just starting out, start with the original model until you learn what you're doing.

xiepengchong commented 5 years ago

ValueError: Error: Alignments file not found at /srv/liudehua_dest/alignments.json The alignments file must be in /srv/liudehua_dest/alignments.json

If you're just starting out, start with the original model until you learn what you're doing.

I am sorry,I hadn't mv the alignments.json to dest folder before,after I copy it ,the problem resolved, but I have the "Cannot create group in read only mode" problem still,I will try to resolve it, thank you very much

xiepengchong commented 5 years ago

ValueError: Error: Alignments file not found at /srv/liudehua_dest/alignments.json The alignments file must be in /srv/liudehua_dest/alignments.json

If you're just starting out, start with the original model until you learn what you're doing.

thanks for your help,I trained success just now,I use lightweight and batch 2.