USTC-JialunPeng
commented
2 years ago Hi! I didn't fully understand your question. Could you please be more specific？
Open chandniagarwal opened 2 years ago
USTC-JialunPeng
commented
2 years ago Hi! I didn't fully understand your question. Could you please be more specific？
chandniagarwal
commented
2 years ago Thanks Peng!!
I am research scholar and using your model in regenerating faces on my own dataset with masked faces (created on celebA and celebAHQ). I have used your pre-trained model given on git for the face reconstruction using test.py. The created faces are not at all good for celebA whereas faces created for CelebAHQ are somewhat better as your model is trained on that. My Issues are :
All the issues are urgent as my paper submission is due and my lot of time is going in troubleshoot. I am planning to make it base paper and to be used as transfer learning model.
TIA
chandniagarwal
commented
2 years ago done with tensorflow error now in another notebook. please guide about training .
USTC-JialunPeng
commented
2 years ago Sorry, I just saw your issues.
If you want to train a new model, you need to input the masked image and the corresponding binary mask in the training. In our training code, the masked image and the corresponding binary mask are automatically generated. So you just need to collect the ground truth dataset and leave the masking procedure to the code. You can set the random_mask argument to True to use random masks, otherwise the default center masks will be used.
Best wishes, Jialun
chandniagarwal
commented
2 years ago Thanks USTC-JialunPeng/Diverse-Structure-Inpainting So I need to send ground truth and corresponding binary mask image in training the new model. Thanks for reply.
Regards Chandni Agarwal
On Tue, Jul 19, 2022 at 6:42 PM USTC-JialunPeng @.***> wrote:
Sorry, I just saw your issues.
If you want to train a new model, you need to input the masked image and the corresponding binary mask in the training. In our training code, the masked image and the corresponding binary mask are automatically generated. So you just need to collect the ground truth dataset and leave the masking procedure to the code. You can set the random_mask argument to True to use random masks, otherwise the default center masks will be used.
Best wishes, Jialun
— Reply to this email directly, view it on GitHub https://github.com/USTC-JialunPeng/Diverse-Structure-Inpainting/issues/25#issuecomment-1189036713, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACJSE45WZUCGGJHPGHLPEVTVU2ST5ANCNFSM53YL6XAA . You are receiving this because you authored the thread.Message ID: @.*** com>
USTC-JialunPeng
commented
2 years ago Right. You just need to collect your ground truth dataset and make a path list. Please refer to our training instructions for more details.
chandniagarwal
commented
2 years ago Dear Sir
Facing issue with train_structure generator and train texture generator code. sharing .txt file containing error while running individual codes. Not able to train data. It is urgent,
PFA file.
Kindly reply asap.
Regards
On Tue, Jul 19, 2022 at 10:26 PM Chandni Agarwal @.***> wrote:
Thanks!!
Following same instructions. One more question : Should I put binary mask and ground truth in same folder.? giving images pl tell which one to load?
ground truth
- only mask map or mask covered with overlay map.
Regards
On Tue, Jul 19, 2022 at 9:44 PM USTC-JialunPeng @.***> wrote:
Right. You just need to collect your ground truth dataset and make a path list. Please refer to our training instructions https://github.com/USTC-JialunPeng/Diverse-Structure-Inpainting#training for more details.
— Reply to this email directly, view it on GitHub https://github.com/USTC-JialunPeng/Diverse-Structure-Inpainting/issues/25#issuecomment-1189277876, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACJSE43HM3GWHOBEB7WLQN3VU3H7ZANCNFSM53YL6XAA . You are receiving this because you authored the thread.Message ID: @.*** .com>
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/dtypes.py:528: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/dtypes.py:529: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/dtypes.py:530: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/dtypes.py:535: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
------------ Options -------------
batch_size: 8
checkpoints_dir: ./gdata/vqvae-inpainting
commitment_cost: 0.25
dataset: celeba
decay: 0.99
dropout_s: 0.1
ema_decay: 0.9997
embedding_dim: 64
image_size: 256
learning_rate: 0.18
load_size: 266
margins: 0
mask_size: 128
max_delta: 0
max_steps: 1000000
nr_attention_s: 4
nr_channel_cond_s: 32
nr_channel_s: 128
nr_channel_vq: 128
nr_gpu: 2
nr_head_s: 8
nr_res_block_vq: 2
nr_res_channel_cond_s: 32
nr_res_channel_s: 128
nr_res_channel_vq: 64
nr_resnet_out_s: 20
nr_resnet_s: 20
num_embeddings: 512
random_mask: True
resnet_nonlinearity: concat_elu
train_flist: ./gdata/train.flist
train_spe: 10000
val_steps: 10000
valid_flist: ./gdata/val.flist
vqvae_network_dir: /content/gdrive/MyDrive/Diverse-Structure-Inpainting/gdata/vqvae-inpainting/20220719-184610_CelebAHQ_celebahq
-------------- End ----------------
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/control_flow_ops.py:423: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From /content/gdrive/MyDrive/Diverse-Structure-Inpainting/net/nn.py:128: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version.
Instructions for updating:
tf.py_func is deprecated in TF V2. Instead, use
tf.py_function, which takes a python function which manipulates tf eager
tensors instead of numpy arrays. It's easy to convert a tf eager tensor to
an ndarray (just call tensor.numpy()) but having access to eager tensors
means tf.py_functions can use accelerators such as GPUs as well as
being differentiable using a gradient tape.
WARNING:tensorflow:From train_structure_generator.py:225: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version. Instructions for updating:
Future major versions of TensorFlow will allow gradients to flow into the labels input on backprop by default.
See tf.nn.softmax_cross_entropy_with_logits_v2.
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/math_grad.py:102: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Deprecated in favor of operator or tf.math.divide.
WARNING:tensorflow:From train_structure_generator.py:267: multinomial (from tensorflow.python.ops.random_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.random.categorical instead.
2022-07-21 17:17:20.484784: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 AVX512F FMA
2022-07-21 17:17:20.487942: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2000175000 Hz
2022-07-21 17:17:20.488233: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x3111c80 executing computations on platform Host. Devices:
2022-07-21 17:17:20.488270: I tensorflow/compiler/xla/service/service.cc:158] StreamExecutor device (0):
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "train_structure_generator.py", line 322, in
Caused by op 'gradients_1/structure_condition_3/wnconv2d_22/Conv2D_grad/Conv2DBackpropInput', defined at:
File "train_structure_generator.py", line 229, in
...which was originally created as op 'structure_condition_3/wnconv2d_22/Conv2D', defined at:
File "train_structure_generator.py", line 223, in
InvalidArgumentError (see above for traceback): Current libxsmm and customized CPU implementations do not yet support dilation rates larger than 1. [[node gradients_1/structure_condition_3/wnconv2d_22/Conv2D_grad/Conv2DBackpropInput (defined at train_structure_generator.py:229) ]]
USTC-JialunPeng
commented
2 years ago Are you training the model on GPU? BTW, which TensorFlow version are you using？
chandniagarwal
commented
2 years ago is it possible instead of generating a mask , I give a binary image of the mask for training from the file list.
Kindly reply.
Regards Chandni Agarwal
On Tue, Jul 19, 2022 at 6:42 PM USTC-JialunPeng @.***> wrote:
Sorry, I just saw your issues.
If you want to train a new model, you need to input the masked image and the corresponding binary mask in the training. In our training code, the masked image and the corresponding binary mask are automatically generated. So you just need to collect the ground truth dataset and leave the masking procedure to the code. You can set the random_mask argument to True to use random masks, otherwise the default center masks will be used.
Best wishes, Jialun
— Reply to this email directly, view it on GitHub https://github.com/USTC-JialunPeng/Diverse-Structure-Inpainting/issues/25#issuecomment-1189036713, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACJSE45WZUCGGJHPGHLPEVTVU2ST5ANCNFSM53YL6XAA . You are receiving this because you authored the thread.Message ID: @.*** com>
USTC-JialunPeng
commented
2 years ago Of course you can use your own masks instead of generated masks. But it may be challenging for you to modify the corresponding code. You can take a try and I wish you good luck.
chandniagarwal
commented
2 years ago Can I use your regular mask code instead as I can give it a try later . As the faces are not correctly created so trying with training with new dataset. checked nn.py too for own mask. Thanks for quick reply.
On Sun, Jul 24, 2022 at 2:07 PM USTC-JialunPeng @.***> wrote:
Of course you can use your own masks instead of generated masks. But it may be challenging for you to modify the corresponding code. You can take a try and I wish you good luck.
— Reply to this email directly, view it on GitHub https://github.com/USTC-JialunPeng/Diverse-Structure-Inpainting/issues/25#issuecomment-1193273028, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACJSE47VFIDOGGU3A6NNNKTVVT6FTANCNFSM53YL6XAA . You are receiving this because you authored the thread.Message ID: @.*** com>
chandniagarwal
commented
2 years ago 2022-07-24 12:53:22.183438: I tensorflow/core/kernels/data/shuffle_dataset_op.cc:195] Shuffle buffer filled. 2022-07-24 12:53:24.173695: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 100 iterations, time: 427s, train NLL: nan. 200 iterations, time: 332s, train NLL: nan. 300 iterations, time: 333s, train NLL: nan.
Not able to remove this error.
On Sun, Jul 24, 2022 at 6:09 PM Chandni Agarwal @.***> wrote:
Can I use your regular mask code instead as I can give it a try later . As the faces are not correctly created so trying with training with new dataset. checked nn.py too for own mask. Thanks for quick reply.
On Sun, Jul 24, 2022 at 2:07 PM USTC-JialunPeng @.***> wrote:
Of course you can use your own masks instead of generated masks. But it may be challenging for you to modify the corresponding code. You can take a try and I wish you good luck.
— Reply to this email directly, view it on GitHub https://github.com/USTC-JialunPeng/Diverse-Structure-Inpainting/issues/25#issuecomment-1193273028, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACJSE47VFIDOGGU3A6NNNKTVVT6FTANCNFSM53YL6XAA . You are receiving this because you authored the thread.Message ID: @.*** .com>
chandniagarwal
commented
2 years ago For train_structure_generator
2022-07-24 12:53:22.183438: I tensorflow/core/kernels/data/shuffle_dataset_op.cc:195] Shuffle buffer filled. 2022-07-24 12:53:24.173695: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 100 iterations, time: 427s, train NLL: nan. 200 iterations, time: 332s, train NLL: nan. 300 iterations, time: 333s, train NLL: nan. How to solve
On Sun, Jul 24, 2022 at 6:42 PM Chandni Agarwal @.***> wrote:
2022-07-24 12:53:22.183438: I tensorflow/core/kernels/data/shuffle_dataset_op.cc:195] Shuffle buffer filled. 2022-07-24 12:53:24.173695: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 100 iterations, time: 427s, train NLL: nan. 200 iterations, time: 332s, train NLL: nan. 300 iterations, time: 333s, train NLL: nan.
Not able to remove this error.
On Sun, Jul 24, 2022 at 6:09 PM Chandni Agarwal @.***> wrote:
Can I use your regular mask code instead as I can give it a try later . As the faces are not correctly created so trying with training with new dataset. checked nn.py too for own mask. Thanks for quick reply.
On Sun, Jul 24, 2022 at 2:07 PM USTC-JialunPeng @.***> wrote:
Of course you can use your own masks instead of generated masks. But it may be challenging for you to modify the corresponding code. You can take a try and I wish you good luck.
— Reply to this email directly, view it on GitHub https://github.com/USTC-JialunPeng/Diverse-Structure-Inpainting/issues/25#issuecomment-1193273028, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACJSE47VFIDOGGU3A6NNNKTVVT6FTANCNFSM53YL6XAA . You are receiving this because you authored the thread.Message ID: <USTC-JialunPeng/Diverse-Structure-Inpainting/issues/25/1193273028@ github.com>
USTC-JialunPeng
commented
2 years ago It seems that the negative log-likelihood (NLL) diverges at the beginning of training. Could you provide more details about your code modifications?
chandniagarwal
commented
2 years ago Can we have meeting on Google meet so that I can share my code with you? Pl tell suitable time. Regards
On Thu, 28 Jul 2022, 12:09 USTC-JialunPeng, @.***> wrote:
It seems that the negative log-likelihood (NLL) diverges at the beginning of training. Could you provide more details about your code modifications?
— Reply to this email directly, view it on GitHub https://github.com/USTC-JialunPeng/Diverse-Structure-Inpainting/issues/25#issuecomment-1197727352, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACJSE47BG6DGJHGNZQO5RTTVWITJDANCNFSM53YL6XAA . You are receiving this because you authored the thread.Message ID: @.*** com>
chandniagarwal
commented
2 years ago Hello sir
Stucked in research and not able figure out problem.
Regards
On Thu, 28 Jul 2022, 18:18 Chandni Agarwal, @.***> wrote:
Can we have meeting on Google meet so that I can share my code with you? Pl tell suitable time. Regards
On Thu, 28 Jul 2022, 12:09 USTC-JialunPeng, @.***> wrote:
It seems that the negative log-likelihood (NLL) diverges at the beginning of training. Could you provide more details about your code modifications?
— Reply to this email directly, view it on GitHub https://github.com/USTC-JialunPeng/Diverse-Structure-Inpainting/issues/25#issuecomment-1197727352, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACJSE47BG6DGJHGNZQO5RTTVWITJDANCNFSM53YL6XAA . You are receiving this because you authored the thread.Message ID: @.*** .com>
USTC-JialunPeng
commented
2 years ago I'm afraid it's not convenient for me to attend google meeting. I also don't know why the NLL diverges. Can you briefly describe what changes were made?
chandniagarwal
commented
2 years ago Thanks for reply, could the size 178x218 a reason of failing , just checked the size of image from dataset
On Sat, Jul 30, 2022 at 8:16 PM USTC-JialunPeng @.***> wrote:
I'm afraid it's not convenient for me to attend google meeting. I also don't know why the NLL diverges. Can you briefly describe what changes were made?
— Reply to this email directly, view it on GitHub https://github.com/USTC-JialunPeng/Diverse-Structure-Inpainting/issues/25#issuecomment-1200174501, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACJSE4ZYLFI35N26USIVC63VWU52RANCNFSM53YL6XAA . You are receiving this because you authored the thread.Message ID: @.*** com>
USTC-JialunPeng
commented
2 years ago I think it may be the reason. I have only evaluated our method on 256x256 image size.
chandniagarwal
commented
2 years ago ok doing in that way. will revert soon
On Sat, Jul 30, 2022 at 9:21 PM USTC-JialunPeng @.***> wrote:
I think is may be the reason. I have only evaluated our method on 256x256 image size.
— Reply to this email directly, view it on GitHub https://github.com/USTC-JialunPeng/Diverse-Structure-Inpainting/issues/25#issuecomment-1200238132, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACJSE47IIXVPTLKQLPZEDVLVWVFRXANCNFSM53YL6XAA . You are receiving this because you authored the thread.Message ID: @.*** com>
chandniagarwal
commented
2 years ago done train-vqvae using 256x256 ground truth images and still getting nan for train structure generator. pl reply
On Sat, Jul 30, 2022 at 10:33 PM Chandni Agarwal @.***> wrote:
ok doing in that way. will revert soon
On Sat, Jul 30, 2022 at 9:21 PM USTC-JialunPeng @.***> wrote:
I think is may be the reason. I have only evaluated our method on 256x256 image size.
— Reply to this email directly, view it on GitHub https://github.com/USTC-JialunPeng/Diverse-Structure-Inpainting/issues/25#issuecomment-1200238132, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACJSE47IIXVPTLKQLPZEDVLVWVFRXANCNFSM53YL6XAA . You are receiving this because you authored the thread.Message ID: @.*** .com>
chandniagarwal
commented
2 years ago Thanks!!
Following same instructions. One more question : Should I put binary mask and ground truth in same folder.? giving images pl tell which one to load?
ground truth
Regards
On Tue, Jul 19, 2022 at 9:44 PM USTC-JialunPeng @.***> wrote:
Right. You just need to collect your ground truth dataset and make a path list. Please refer to our training instructions https://github.com/USTC-JialunPeng/Diverse-Structure-Inpainting#training for more details.
— Reply to this email directly, view it on GitHub https://github.com/USTC-JialunPeng/Diverse-Structure-Inpainting/issues/25#issuecomment-1189277876, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACJSE43HM3GWHOBEB7WLQN3VU3H7ZANCNFSM53YL6XAA . You are receiving this because you authored the thread.Message ID: @.*** com>
USTC-JialunPeng
commented
2 years ago We didn't put masks and images in the same folder. One random mask and one random image are loaded to generate a masked image in the training. The generated masked image and the corresponding mask are send into the model while the image is used as ground truth.
Hi! I am using the model for generating new faces from masked faces. I have run test.py and modify the code a little for removing face mask.
The model is not performing on CelebA masked faces , kindly guide for training for masked face, do i need to pass masks ?