search

hpi-xnor / BMXNet

(New version is out: https://github.com/hpi-xnor/BMXNet-v2) BMXNet: An Open-Source Binary Neural Network Implementation Based on MXNet
Apache License 2.0
349 stars 95 forks source link

Core dump during fine-tuning with bin-resnet18 #21

Closed ehsanmok closed 5 years ago

ehsanmok commented 6 years ago

Environment info

Operating System: Ubuntu 16.04

Compiler: gcc, g++ 5.4.0

Package used (Python/R/Scala/Julia): Python

MXNet version: BMXNet built from master. USE_CUDA=0, USE_CUDNN=0 USE_DIST_KVSTORE=0

If you are using python package, please provide

Python version and distribution: 3.6 Anaconda

Error Message:

Core dump in ipython/jupyter notebook. Running with NaiveEngine gives me 

./tensor_blob.h:198: Check failed: mshadow::DataType<DType>::kFlag == type_flag_ TBlob.get_with_shape: data type do not match specified type.Expected: 1 v.s. given 0

which seems to be a known issue but not sure if it was resolved the BMXNet version of MXNet 0.10.1!

Minimum reproducible example

/smd_hpi/examples/binary-imagenet1k/fine-tune.py works for regular pretrained resnet downloaded from modelzoo, but get the core dump using bin-resnet18 in smd_hpi/binary_models/binarized_imagenet-resnet18-64bit-0040

Looking at the git history, it seems that you haven't tried the fine-tuning and the paper only mentions training. How can I correctly load the already converted (binarized) weights in smd_hpi/binary_models/ in mx.module and send batch of data to mod.forward(batch)?

yanghaojin commented 6 years ago

hi, so since this implementation is based on the image classification example of mxnet, so the fine-tune.py was just left in the repository. We do the fine tuning just used the same training script e.g. "train_imagenet.py, train_cifar.py..." and the additional option "--pretrained" like:

python train_imagenet.py --gpus=0 --log=binary-resnet.log --network='resnet-binary' --model-prefix=/...../binary-resnet-cudnn --data-train=/data/mxnet_data/imagenet1k/imagenet1k-train.rec --data-val=/data/mxnet_data/imagenet1k/imagenet1k-val.rec --optimizer=sgd --lr=0.1 --batch-size=256 --pretrained=/home/haojin/workspace/mxnet-HPI/smd_hpi/examples/binary-imagenet1k/binary-resnet-cudnn-0022.params

ehsanmok commented 6 years ago

@yanghaojin yes, the problem is only converted binarized_* params are provided in smd_hpi/binary-models/ so have to first train with binary then I can use them!

yanghaojin commented 6 years ago

I can send you a pretrained model with binary weights (before binary-model conversion) so that you can use it for fine-tuning.

huanyii commented 6 years ago

can you send me the pretrained model with binary weights on Imagenet datasets (before binary-model conversion) too? I want to fine-tuning with my datasets, and then convert it .Thank you very much. @yanghaojin

ghost commented 6 years ago

yes, pls let me know your email.

2018-04-03 8:00 GMT+02:00 huanyii notifications@github.com:

can you send me the pretrained model with binary weights on Imagenet datasets (before binary-model conversion) too? I want to fine-tuning with my datasets, and then convert it .Thank you very much. @yanghaojin https://github.com/yanghaojin

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/hpi-xnor/BMXNet/issues/21#issuecomment-378137345, or mute the thread https://github.com/notifications/unsubscribe-auth/AHsYmOXR47PEaA95GWWeGDoqA7GgvQdrks5tkw_0gaJpZM4Q-Cas .

huanyii commented 6 years ago

Thank you!!! This is my email: 2014301220051@whu.edu.cn

2018-04-03 22:40 GMT+08:00 SemaMediaData notifications@github.com:

yes, pls let me know your email.

2018-04-03 8:00 GMT+02:00 huanyii notifications@github.com:

can you send me the pretrained model with binary weights on Imagenet datasets (before binary-model conversion) too? I want to fine-tuning with my datasets, and then convert it .Thank you very much. @yanghaojin https://github.com/yanghaojin

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/hpi-xnor/BMXNet/issues/21#issuecomment-378137345, or mute the thread https://github.com/notifications/unsubscribe-auth/ AHsYmOXR47PEaA95GWWeGDoqA7GgvQdrks5tkw_0gaJpZM4Q-Cas .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/hpi-xnor/BMXNet/issues/21#issuecomment-378274101, or mute the thread https://github.com/notifications/unsubscribe-auth/AgrH9j-WOuJXn_P9Jq1sMJGeVtpzqXo5ks5tk4nigaJpZM4Q-Cas .