search

airsplay / lxmert

PyTorch code for EMNLP 2019 paper "LXMERT: Learning Cross-Modality Encoder Representations from Transformers".
MIT License
923 stars 157 forks source link

cpu-only caffe version of extract_nlvr2_image.py #89

Closed yezhengli-Mr9 closed 3 years ago

yezhengli-Mr9 commented 3 years ago

Will a cpu-only caffe version of extract_nlvr2_image.py available? I think install an environment with cpu-only caffe is not difficult anyway.

yezhengli-Mr9 commented 3 years ago

While I am using Docker airsplay/bottom-up-attention, I try to update a version but with python extract_nlvr2_image.py --split test, I get 

Called with args:
Namespace(caffemodel='./resnet101_faster_rcnn_final_iter_320000.caffemodel', cfg_file='/opt/butd/experiments/cfgs/faster_rcnn_end2end_resnet.yml', imgroot='/workspace/images/', outfile='test_obj36.tsv', prototxt='/opt/butd/models/vg/ResNet-101/faster_rcnn_end2end_final/test.prototxt', set_cfgs=None, split='test')
/opt/butd//tools/../lib/fast_rcnn/config.py:288: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  yaml_cfg = edict(yaml.load(f))
Using config:
{'DATA_DIR': '/opt/butd/data',
 'DEDUP_BOXES': 0.0625,
 'EPS': 1e-14,
 'EXP_DIR': 'faster_rcnn_resnet',
 'GPU_ID': 0,
 'MATLAB': 'matlab',
 'MODELS_DIR': '/opt/butd/models/pascal_voc',
 'PIXEL_MEANS': array([[[102.9801, 115.9465, 122.7717]]]),
 'RNG_SEED': 3,
 'ROOT_DIR': '/opt/butd',
 'TEST': {'AGNOSTIC': False,
          'BBOX_REG': True,
          'HAS_ATTRIBUTES': True,
          'HAS_RELATIONS': False,
          'HAS_RPN': True,
          'MAX_SIZE': 1000,
          'NMS': 0.3,
          'PROPOSAL_METHOD': 'selective_search',
          'RPN_MIN_SIZE': 16,
          'RPN_NMS_THRESH': 0.7,
          'RPN_POST_NMS_TOP_N': 300,
          'RPN_PRE_NMS_TOP_N': 6000,
          'SCALES': [600],
          'SOFT_NMS': 0,
          'SVM': False},
 'TRAIN': {'AGNOSTIC': False,
           'ASPECT_GROUPING': True,
           'BATCH_SIZE': 64,
           'BBOX_INSIDE_WEIGHTS': [1.0, 1.0, 1.0, 1.0],
           'BBOX_NORMALIZE_MEANS': [0.0, 0.0, 0.0, 0.0],
           'BBOX_NORMALIZE_STDS': [0.1, 0.1, 0.2, 0.2],
           'BBOX_NORMALIZE_TARGETS': True,
           'BBOX_NORMALIZE_TARGETS_PRECOMPUTED': True,
           'BBOX_REG': True,
           'BBOX_THRESH': 0.5,
           'BG_THRESH_HI': 0.5,
           'BG_THRESH_LO': 0.0,
           'FG_FRACTION': 0.5,
           'FG_THRESH': 0.5,
           'HAS_ATTRIBUTES': True,
           'HAS_RELATIONS': False,
           'HAS_RPN': True,
           'IMS_PER_BATCH': 1,
           'MAX_SIZE': 1000,
           'MIN_RELATION_FRACTION': 0.25,
           'PROPOSAL_METHOD': 'gt',
           'RPN_BATCHSIZE': 64,
           'RPN_BBOX_INSIDE_WEIGHTS': [1.0, 1.0, 1.0, 1.0],
           'RPN_CLOBBER_POSITIVES': False,
           'RPN_FG_FRACTION': 0.5,
           'RPN_MIN_SIZE': 16,
           'RPN_NEGATIVE_OVERLAP': 0.3,
           'RPN_NMS_THRESH': 0.7,
           'RPN_NORMALIZE_MEANS': [0.0, 0.0, 0.0, 0.0],
           'RPN_NORMALIZE_STDS': [0.1, 0.1, 0.2, 0.2],
           'RPN_NORMALIZE_TARGETS': False,
           'RPN_POSITIVE_OVERLAP': 0.7,
           'RPN_POSITIVE_WEIGHT': -1.0,
           'RPN_POST_NMS_TOP_N': 2000,
           'RPN_PRE_NMS_TOP_N': 12000,
           'SCALES': [600],
           'SNAPSHOT_INFIX': '',
           'SNAPSHOT_ITERS': 10000,
           'USE_FLIPPED': True,
           'USE_PREFETCH': False},
 'USE_GPU_NMS': False}
missing 9/9
WARNING: Logging before InitGoogleLogging() is written to STDERR
E0106 20:31:51.758689   920 common.cpp:114] Cannot create Cublas handle. Cublas won't be available.
E0106 20:31:51.758805   920 common.cpp:121] Cannot create Curand generator. Curand won't be available.
F0106 20:31:51.762009   920 cudnn_conv_layer.cpp:52] Check failed: error == cudaSuccess (35 vs. 0)  CUDA driver version is insufficient for CUDA runtime version
*** Check failure stack trace: ***
Aborted (core dumped)
yezhengli-Mr9 commented 3 years ago

I installed caffe of version bottom-up-attention and I adjusted Makefile.config and now python extract_nlvr2_image.py --split test outputs

Called with args:
Namespace(caffemodel='./resnet101_faster_rcnn_final_iter_320000.caffemodel', cfg_file='/opt/butd/experiments/cfgs/faster_rcnn_end2end_resnet.yml', imgroot='/workspace/images/', outfile='test_obj36.tsv', prototxt='/opt/butd/models/vg/ResNet-101/faster_rcnn_end2end_final/test.prototxt', set_cfgs=None, split='test')
/opt/butd//tools/../lib/fast_rcnn/config.py:288: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  yaml_cfg = edict(yaml.load(f))
Using config:
{'DATA_DIR': '/opt/butd/data',
 'DEDUP_BOXES': 0.0625,
 'EPS': 1e-14,
 'EXP_DIR': 'faster_rcnn_resnet',
 'GPU_ID': 0,
 'MATLAB': 'matlab',
 'MODELS_DIR': '/opt/butd/models/pascal_voc',
 'PIXEL_MEANS': array([[[102.9801, 115.9465, 122.7717]]]),
 'RNG_SEED': 3,
 'ROOT_DIR': '/opt/butd',
 'TEST': {'AGNOSTIC': False,
          'BBOX_REG': True,
          'HAS_ATTRIBUTES': True,
          'HAS_RELATIONS': False,
          'HAS_RPN': True,
          'MAX_SIZE': 1000,
          'NMS': 0.3,
          'PROPOSAL_METHOD': 'selective_search',
          'RPN_MIN_SIZE': 16,
          'RPN_NMS_THRESH': 0.7,
          'RPN_POST_NMS_TOP_N': 300,
          'RPN_PRE_NMS_TOP_N': 6000,
          'SCALES': [600],
          'SOFT_NMS': 0,
          'SVM': False},
 'TRAIN': {'AGNOSTIC': False,
           'ASPECT_GROUPING': True,
           'BATCH_SIZE': 64,
           'BBOX_INSIDE_WEIGHTS': [1.0, 1.0, 1.0, 1.0],
           'BBOX_NORMALIZE_MEANS': [0.0, 0.0, 0.0, 0.0],
           'BBOX_NORMALIZE_STDS': [0.1, 0.1, 0.2, 0.2],
           'BBOX_NORMALIZE_TARGETS': True,
           'BBOX_NORMALIZE_TARGETS_PRECOMPUTED': True,
           'BBOX_REG': True,
           'BBOX_THRESH': 0.5,
           'BG_THRESH_HI': 0.5,
           'BG_THRESH_LO': 0.0,
           'FG_FRACTION': 0.5,
           'FG_THRESH': 0.5,
           'HAS_ATTRIBUTES': True,
           'HAS_RELATIONS': False,
           'HAS_RPN': True,
           'IMS_PER_BATCH': 1,
           'MAX_SIZE': 1000,
           'MIN_RELATION_FRACTION': 0.25,
           'PROPOSAL_METHOD': 'gt',
           'RPN_BATCHSIZE': 64,
           'RPN_BBOX_INSIDE_WEIGHTS': [1.0, 1.0, 1.0, 1.0],
           'RPN_CLOBBER_POSITIVES': False,
           'RPN_FG_FRACTION': 0.5,
           'RPN_MIN_SIZE': 16,
           'RPN_NEGATIVE_OVERLAP': 0.3,
           'RPN_NMS_THRESH': 0.7,
           'RPN_NORMALIZE_MEANS': [0.0, 0.0, 0.0, 0.0],
           'RPN_NORMALIZE_STDS': [0.1, 0.1, 0.2, 0.2],
           'RPN_NORMALIZE_TARGETS': False,
           'RPN_POSITIVE_OVERLAP': 0.7,
           'RPN_POSITIVE_WEIGHT': -1.0,
           'RPN_POST_NMS_TOP_N': 2000,
           'RPN_PRE_NMS_TOP_N': 12000,
           'SCALES': [600],
           'SNAPSHOT_INFIX': '',
           'SNAPSHOT_ITERS': 10000,
           'USE_FLIPPED': True,
           'USE_PREFETCH': False},
 'USE_GPU_NMS': False}
missing 9/9
START caffe.set_mode_cpu()
WARNING: Logging before InitGoogleLogging() is written to STDERR
F0107 06:00:52.336226 25839 layer_factory.hpp:81] Check failed: registry.count(type) == 1 (0 vs. 1) Unknown layer type: Python (known types: AbsVal, Accuracy, ArgMax, BNLL, BatchNorm, BatchReindex, Bias, BoxAnnotatorOHEM, Concat, ContrastiveLoss, Convolution, Crop, Data, Deconvolution, Dropout, DummyData, ELU, Eltwise, Embed, EuclideanLoss, Exp, Filter, Flatten, HDF5Data, HDF5Output, HingeLoss, Im2col, ImageData, InfogainLoss, InnerProduct, InnerProductBlob, Input, LRN, LSTM, LSTMUnit, Log, MVN, MemoryData, MultinomialLogisticLoss, PReLU, PSROIPooling, Parameter, Pooling, Power, RNN, ROIPooling, ReLU, Reduction, Reshape, SPP, Scale, Sigmoid, SigmoidCrossEntropyLoss, Silence, Slice, SmoothL1Loss, SmoothL1LossOHEM, Softmax, SoftmaxWithLoss, SoftmaxWithLossOHEM, Split, TanH, Threshold, Tile, WindowData)
*** Check failure stack trace: ***
yezhengli-Mr9 commented 3 years ago

CPU-only seems very slow. Merely 9 images might take about 5 minutes on my virtual machine. 100%|##########| 9/9 [04:58<00:00, 24.64s/it]