Unable to run training on BiSeNet using pre-trained MobileNetV2

[anzisheng]

When I run train.py , I set the frontend is MobileNetV2, and download the mobilenet_v2.ckpt in models folder. But, the below code cannot read the weight from checkpoint. init_fn = slim.assign_from_checkpoint_fn(model_path=os.path.join(pretrained_dir, 'mobilenet_v2.ckpt'), var_list=slim.get_model_variables('mobilenet_v2'), ignore_missing_vars=True)

errors:

WARNING:tensorflow:Variable mobilenet_v2/Conv/weights missing in checkpoint models/mobilenet_v2.ckpt WARNING:tensorflow:Variable mobilenet_v2/Conv/BatchNorm/gamma missing in checkpoint models/mobilenet_v2.ckpt WARNING:tensorflow:Variable mobilenet_v2/Conv/BatchNorm/beta missing in checkpoint models/mobilenet_v2.ckpt WARNING:tensorflow:Variable mobilenet_v2/Conv/BatchNorm/moving_mean missing in checkpoint models/mobilenet_v2.ckpt WARNING:tensorflow:Variable mobilenet_v2/Conv/BatchNorm/moving_variance missing in checkpoint models/mobilenet_v2.ckpt WARNING:tensorflow:Variable mobilenet_v2/expanded_conv/depthwise/depthwise_weights missing in checkpoint models/mobilenet_v2.ckpt WARNING:tensorflow:Variable mobilenet_v2/expanded_conv/depthwise/BatchNorm/gamma missing in checkpoint models/mobilenet_v2.ckpt WARNING:tensorflow:Variable mobilenet_v2/expanded_conv/depthwise/BatchNorm/beta missing in checkpoint models/mobilenet_v2.ckpt WARNING:tensorflow:Variable mobilenet_v2/expanded_conv/depthwise/BatchNorm/moving_mean missing in checkpoint models/mobilenet_v2.ckpt WARNING:tensorflow:Variable mobilenet_v2/expanded_conv/depthwise/BatchNorm/moving_variance missing in checkpoint models/mobilenet_v2.ckpt WARNING:tensorflow:Variable mobilenet_v2/expanded_conv/project/weights missing in checkpoint models/mobilenet_v2.ckpt

Source code / logs

Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached. Try to provide a reproducible test case that is the bare minimum necessary to generate the problem.

FAQs

• Question: I got an InvalidArgumentError saying that Dimensions of inputs should match Answer: See issue #17

• Question: Can you upload pre-trained weights for these networks? Answer: See issue #57

• Question: Do I need a GPU to train these models? Answer: Technically no, but I'd highly recommend it. I was able to train the models pretty well in about a day using a 1080Ti GPU. Training on CPU would take much longer than that.

• Question: Will you be adding the FCN or U-Net models? Answer: No I won't be adding those simply because they're a few years old and state-of-the-art has moved past that.

• Question: I got an invalid argument error when using the InceptionV4 model. Am I doing something wrong? Answer: No you're not! Due to the design of the InceptiveV4 model, when you end up upsampling you do some rounding which creates a shape mismatch. _This only happens when you end up having to use the end_points['pool5']_. See the code for some of the models if you want to check whether the model will use end_points['pool5'].

[Dreamgang]

I have got the same problem!

[anzisheng]

I have resolved the problem: change the scope "mobilenet_v2" to "MobilenetV2"

[Dreamgang]

I have resolved the problem: change the scope "mobilenet_v2" to "MobilenetV2"

方便加个微信吗？

[Dreamgang]

I have resolved the problem: change the scope "mobilenet_v2" to "MobilenetV2" This problem has resolved,but I have got another problem Assign requires shapes of both tensors to match. lhs shape= [1,1,32,192] rhs shape= [1,1,48,288]

[anzisheng]

Hi, @Dreamgang
先加qq吧： 181888908. 是不是在 AttentionRefinementModule(end_points['pool4'], n_filters=512) 遇到了类似上面的问题？我在看代码决定如何调整。

[orshi]

@Dreamgang @anzisheng Meet the same issue.And have solved.There comes 2 issues:

1.AttentionRefinementModule(end_points['pool4'], n_filters=512), the n_filters value should be modified according to front end network ,but it's hard coded.

2.For BiSeNet ,the front end is build with MobileNetV2,but the project downloaded pre-trained model is for MobileNetV2_140,so you can either use "mobilenet_v2.mobilenet_v2_140" in front end or download the correct pre-trained model from google model zoo.

May help.

[FredHaa]

Can you tell how you changed the code to use "mobilenet_v2.mobilenet_v2_140"?

[RobinHan24]

@Dreamgang @anzisheng Meet the same issue.And have solved.There comes 2 issues:

1.AttentionRefinementModule(end_points['pool4'], n_filters=512), the n_filters value should be modified according to front end network ,but it's hard coded.

2.For BiSeNet ,the front end is build with MobileNetV2,but the project downloaded pre-trained model is for MobileNetV2_140,so you can either use "mobilenet_v2.mobilenet_v2_140" in front end or download the correct pre-trained model from google model zoo.

May help.

So how to modify the n_filters? Thanks a lot.

[qweawq]

I have resolved the problem: change the scope "mobilenet_v2" to "MobilenetV2" Excuse me, which scope is the revision? Is the scope in frontend_builder.py? Which var_list, frontend_scope, etc. need to be modified? Thank you for your help

[qweawq]

Hi, @Dreamgang 先加qq吧： 181888908. 是不是在 AttentionRefinementModule(end_points['pool4'], n_filters=512) 遇到了类似上面的问题？我在看代码决定如何调整。

Excuse me, which scope is the revision? Is the scope in frontend_builder.py? Which var_list, frontend_scope, etc. need to be modified? Thank you for your help