The model is not successfully trained under windows

[Songxinlei]

E:\BGRD\RealTimeDenoisingNeuralBilateralGrid-master\image_utils.py:72: UserWarning: Inputs have mismatched dtype. Setting data_range based on im_true. img_true[idx,:,:,:], img_test[idx,:,:,:]) Traceback (most recent call last): File "C:\ProgramData\Anaconda3\envs\MCDenoising\lib\site-packages\tensorflow\python\client\session.py", line 1334, in _do_call return fn(*args) File "C:\ProgramData\Anaconda3\envs\MCDenoising\lib\site-packages\tensorflow\python\client\session.py", line 1319, in _run_fn options, feed_dict, fetch_list, target_list, run_metadata) File "C:\ProgramData\Anaconda3\envs\MCDenoising\lib\site-packages\tensorflow\python\client\session.py", line 1407, in _call_tf_sessionrun run_metadata) tensorflow.python.framework.errors_impl.InvalidArgumentError: Nan in summary histogram for: GuideNet/final_conv/conv/bias [[{{node GuideNet/final_conv/conv/bias}}]] During handling of the above exception, another exception occurred: Traceback (most recent call last): File "E:/BGRD/RealTimeDenoisingNeuralBilateralGrid-master/network_train.py", line 184, in [summary_merge, guide_net['denoised_hdr'], loss_all_L1, train_step1], feed_dict) File "C:\ProgramData\Anaconda3\envs\MCDenoising\lib\site-packages\tensorflow\python\client\session.py", line 929, in run run_metadata_ptr) File "C:\ProgramData\Anaconda3\envs\MCDenoising\lib\site-packages\tensorflow\python\client\session.py", line 1152, in _run feed_dict_tensor, options, run_metadata) File "C:\ProgramData\Anaconda3\envs\MCDenoising\lib\site-packages\tensorflow\python\client\session.py", line 1328, in _do_run run_metadata) File "C:\ProgramData\Anaconda3\envs\MCDenoising\lib\site-packages\tensorflow\python\client\session.py", line 1348, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.InvalidArgumentError: Nan in summary histogram for: GuideNet/final_conv/conv/bias [[node GuideNet/final_conv/conv/bias (defined at E:\BGRD\RealTimeDenoisingNeuralBilateralGrid-master\network_units.py:25) ]]

Caused by op 'GuideNet/final_conv/conv/bias', defined at: File "E:/BGRD/RealTimeDenoisingNeuralBilateralGrid-master/network_train.py", line 134, in guide_net = model.inference() File "E:\BGRD\RealTimeDenoisingNeuralBilateralGrid-master\network.py", line 31, in inference net_out = self._denseconnect_conv(ae_input, net_name="GuideNet") File "E:\BGRD\RealTimeDenoisingNeuralBilateralGrid-master\network.py", line 132, in _denseconnect_conv output = conv_layer(current_input, final_filter_size, final_layer_stride, final_layer_out_ch) File "E:\BGRD\RealTimeDenoisingNeuralBilateralGrid-master\network_units.py", line 25, in conv_layer tf.summary.histogram('bias', b) File "C:\ProgramData\Anaconda3\envs\MCDenoising\lib\site-packages\tensorflow\python\summary\summary.py", line 177, in histogram tag=tag, values=values, name=scope) File "C:\ProgramData\Anaconda3\envs\MCDenoising\lib\site-packages\tensorflow\python\ops\gen_logging_ops.py", line 312, in histogram_summary "HistogramSummary", tag=tag, values=values, name=name) File "C:\ProgramData\Anaconda3\envs\MCDenoising\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 788, in _apply_op_helper op_def=op_def) File "C:\ProgramData\Anaconda3\envs\MCDenoising\lib\site-packages\tensorflow\python\util\deprecation.py", line 507, in new_func return func(*args, **kwargs) File "C:\ProgramData\Anaconda3\envs\MCDenoising\lib\site-packages\tensorflow\python\framework\ops.py", line 3300, in create_op op_def=op_def) File "C:\ProgramData\Anaconda3\envs\MCDenoising\lib\site-packages\tensorflow\python\framework\ops.py", line 1801, in init self._traceback = tf_stack.extract_stack()

InvalidArgumentError (see above for traceback): Nan in summary histogram for: GuideNet/final_conv/conv/bias [[node GuideNet/final_conv/conv/bias (defined at E:\BGRD\RealTimeDenoisingNeuralBilateralGrid-master\network_units.py:25) ]] Process finished with exit code 1 The model is not successfully trained under windows, and the above error will be reported at the beginning of training. Many methods have been tried, for example, to see if the data is complete, adjust the learning rate, batch-size, and recompile windows to change the value of maxrregcount, but unfortunately none of them have been successful. Do you have a good solution?

[xmeng525]

It seems that there are NAN values in GuideNet/final_conv/conv/bias. Can you run network_test.py with pertrained models?

[Songxinlei]

Yes, network_test works, but training throws such an error. Now I have solved it. I annotated tf.summary.histogram(b), tf.summary.histogram(w), and tf.summary.histogram(pb_out). Then they can train. But I don't understand why this works. Isn't this just for variable visualization in tensorboard?

------------------ 原始邮件 ------------------ 发件人: "xmeng525/RealTimeDenoisingNeuralBilateralGrid" @.>; 发送时间: 2022年9月10日(星期六) 凌晨1:51 @.>; @.**@.>; 主题: Re: [xmeng525/RealTimeDenoisingNeuralBilateralGrid] The model is not successfully trained under windows (Issue #12)

It seems that there are NAN values in GuideNet/final_conv/conv/bias. Can you run network_test.py with pertrained models?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[xmeng525]

Yes. It was for visualization. I'm glad you solved it! Let me know if you have any other questions.