ModuleNotFoundError: No module named 'utils'

[xinyuwang1126]

Hi Luowei,

When I tried to run the scripts, it tells me it needs a util module that is not available in the repo. Is there anything specific I need to do? Besides, I am trying to extract features for my own dataset. Could you provide a sample folder structure I should follow? Thank you very much and look forward to your reply!

[LuoweiZhou]

@xinyuwang1126 Make sure you add your root dir and lib dir to PYTHONPATH: export PYTHONPATH="[Your dir]/detectron-release:$PYTHONPATH" export PYTHONPATH="[Your dir]/detectron-release/lib:$PYTHONPATH" There is no specific data structure to follow (we place all the image jpg under $DATA_ROOT/images). You can customize the code to read according to your folder structure using glob.iglob.

[xinyuwang1126]

Hi Luowei,

Thank you for your help! I have done so and this problem got fixed. However, I am facing an error OSError: $myroot/detectron-vlp/lib/libcaffe2_detectron_ops_gpu.so: undefined symbol Do you have any clue how it would happen? I have installed caffe2 using miniconda, and tried to rename the libcaffe2_detectron_ops_gpu.so file under the conda site-packages but still got the same error. Thanks!

[LuoweiZhou]

@xinyuwang1126 This file appears to be generated by Detectron. How you tried replacing the libcaffe2_detectron_ops_gpu.so file we placed under lib with yours?

[xinyuwang1126]

Hi Luowei, Thanks for your help and it works! I have an additional question (sorry for so many questions): When I extracted the features, it output vlp_checkpoint_trainval_cls100.h5, vlp_checkpoint_trainval_feat100.h5 and vlp_checkpoint_trainval_bbox100.h5. Is there script that can map these files to the feature files used in Flickr30k task? Or are those files of the same format as the ones used in COCO task? (I could not unzip the files extracted from cat COCO0 > COCO.tar.gz so that I could not browse how the coco features look like) Thank you very much for your patience!

[LuoweiZhou]

@xinyuwang1126 Depending on how you config --proc_split, you will get a bunch of .h5 per category (either _cls, _feat, or _bbox). For example, if you set --proc_split from 000 to 999 (each indicating the last three digit of the image ID), you will end up having 1000 .h5 files. Since the keys of the .h5 files are image IDs, you can easily merge them as you want. In our case, for example, in the CC dataset, each _cls and _feat stores feature for 3M/1000=3k images (2G in size) while _bbox stores the coordinates for all 3M images as the size is relatively small. Same for COCO. For legacy reasons, we stored Flickr30k features as individual .npy files but you can try merging them into one or a few .h5 files.

[xinyuwang1126]

So in other words, if I want to run the fine-tuning on another captioning dataset, what I could do is to merge all the extracted _bbox.h5 files into a single one, leave _cls and _feat files as it is and provide the relevant annotations? And the COCO annotations are of the same format as the Flickr30k ones, am I understanding it correctly? Thanks!

[LuoweiZhou]

Yes or simply adapt the code on your own to the feature files: https://github.com/LuoweiZhou/VLP/blob/master/vlp/seq2seq_loader.py#L466

[Htruth]

Hi Luowei,

When I tried to run the scripts, it tells me it needs a util module that is not available in the repo. Is there anything specific I need to do? Besides, I am trying to extract features for my own dataset. Could you provide a sample folder structure I should follow? Thank you very much and look forward to your reply!

So in other words, if I want to run the fine-tuning on another captioning dataset, what I could do is to merge all the extracted _bbox.h5 files into a single one, leave _cls and _feat files as it is and provide the relevant annotations? And the COCO annotations are of the same format as the Flickr30k ones, am I understanding it correctly? Thanks!

Hello, xinyu:

Can you share your specific configuration information or DockerFile? Thank you very much.

[Htruth]

/home/zht/detectron-vlp/datasets//flickr30k_images/2333816000.jpg [E net_async_base.cc:377] [enforce fail at conv_op_cudnn.cc:807] status == CUDNN_STATUS_SUCCESS. 7 vs 0. , Error at: /opt/conda/conda-bld/pytorch_1544080996887/work/caffe2/operators/conv_op_cudnn.cc:807: CUDNN_STATUS_MAPPING_ERROR Error from operator: input: "gpu_0/data" input: "gpu_0/conv1_w" output: "gpu_0/conv1" name: "" type: "Conv" arg { name: "kernel" i: 7 } arg { name: "exhaustive_search" i: 0 } arg { name: "pad" i: 3 } arg { name: "order" s: "NCHW" } arg { name: "stride" i: 2 } device_option { device_type: 1 device_id: 0 } engine: "CUDNN"frame #0: c10::ThrowEnforceNotMet(char const*, int, char const*, std::string const&, void const*) + 0x59 (0x7fa5ce8b6309 in /home/zht/anaconda3/envs/py2.7torch1.0cuda8/lib/python2.7/site-packages/caffe2/python/../../torch/lib/libc10.so) I meet this problem, It looks like CUDNN has problems.

[MarcusNerva]

/home/zht/detectron-vlp/datasets//flickr30k_images/2333816000.jpg [E net_async_base.cc:377] [enforce fail at conv_op_cudnn.cc:807] status == CUDNN_STATUS_SUCCESS. 7 vs 0. , Error at: /opt/conda/conda-bld/pytorch_1544080996887/work/caffe2/operators/conv_op_cudnn.cc:807: CUDNN_STATUS_MAPPING_ERROR Error from operator: input: "gpu_0/data" input: "gpu_0/conv1_w" output: "gpu_0/conv1" name: "" type: "Conv" arg { name: "kernel" i: 7 } arg { name: "exhaustive_search" i: 0 } arg { name: "pad" i: 3 } arg { name: "order" s: "NCHW" } arg { name: "stride" i: 2 } device_option { device_type: 1 device_id: 0 } engine: "CUDNN"frame #0: c10::ThrowEnforceNotMet(char const*, int, char const*, std::string const&, void const*) + 0x59 (0x7fa5ce8b6309 in /home/zht/anaconda3/envs/py2.7torch1.0cuda8/lib/python2.7/site-packages/caffe2/python/../../torch/lib/libc10.so) I meet this problem, It looks like CUDNN has problems.

Hi, there! Have you solve this problem? I meet this problem too! Wondering what happened.