search

jakeret / tf_unet

Generic U-Net Tensorflow implementation for image segmentation
GNU General Public License v3.0
1.9k stars 748 forks source link

Possible Errors in Current Version - some variables are not trained #176

Open XiaoYunZhou27 opened 6 years ago

XiaoYunZhou27 commented 6 years ago

Hi,

Thank you for contributing this fantastic code. It helps me a lot.

However, as I am learning deeper and deeper, I found that the "wd, bd, weight, and bias" in the function "create_conv_net" is not appended into the trainable variable - "variables". I make these variables trainable and the result improves a lot.

Could you please kindly discuss and tell me whether my concern makes sense, please? Thanks a lot.

jakeret commented 6 years ago

Hi thanks for this input. Which variables do mean exactly, could you point me to the relevant code line?

The variables list is only used when regularization is used. Therefore I would assume that it should only affect the generalization error. Do you see a difference during training?

XiaoYunZhou27 commented 6 years ago

Hi,

Thanks for the replying.

Could you please kindly go to function "def create_conv_net" - "up layers" - wd and bd are not appended into the "variables", also "def create_conv_net" - "Output Map" - weight and bias are not appended into the "variables".

I think the "variables" is the variable which will be updated by the SGD. If I misunderstood, please let me know. I am learning.

Thank you very much for this contribution.

jakeret commented 6 years ago

Ok is see. Yes you're correct. Thank you for pointing this out. Those variables should be appended to the list. Weights and biases are indeed the ones updated by SGD. However, this happens internally in tensorflow and does not depend on the variables list. Therefore, training should not be affected.

If you have time, it would be great if you could fix the bug and send me a git Pull Request.

meijie0401 commented 6 years ago

So even if you don't append 'wd, bd, weight, and bias' to varilables list, they are still can be updated by tensorflow ? Only when we want to use regularization, we should append them to the varilables list. Am I right?

jakeret commented 6 years ago

Yes. However I don't know how big the effect is if they are in the list or not