Closed zhangzhuoran997997 closed 3 years ago
@zhangzhuoran997997 Thanks for your interest in our work!
max_h_as_o > 0
, only the top max_h_as_o humans with the highest object detection scores are concatenated with objects. This argument is set to -1
for the training set and 3
for the test set to reduce the number of pair proposals.Really appreciate your answer!
I'm a beginner, so the problem will be a little simple.
I have other questions that:
When use the pre-trained detector gt_blobs, dt_blobs = model(return_loss=False, **data)
,I know the dt_blobs is consist of a tensor shaped ($$N_0$$ , 1110)
And I know the gt_blobs should be 1024 dimension features of the human and objects without confidence score. But where did it from and what's the meaning of it ?
In addition,how can I view the detail of the model after build_detector function?
Please feel free to ask me again if you have any other questions :)
Thank you very much for the timely answers. Now, I have a comprehensive understanding of the code and each module, but there are still some questions:
In the semantic.py file , the init and train both use the _use_cache and _cache , when will this variable _use_cache become true? And how the train function work here?
def __init__(self, stream, dims, graph=None, **kwargs):
super(SemanticBlock, self).__init__()
self._stream = stream
self._use_cache = False
self._cache = None
def train(self, mode=True):
self._use_cache = not mode
self._cache = None
return super(SemanticBlock, self).train(mode=mode)
Which parameters will be updated during training ?
Is the affect of the convert_annotation fine-tuning the detector pre-trained on COCO?
self._use_cache
is False
in training mode and turns True
in test mode (after you explicitly invoke model.eval()
). This is because the input of SemanticBlock (GATs) are fixed (word embeddings from pretrained ELMo) so that its output can be cached for more efficient inference.convert_annotation
is used to convert the annotations of HICO-DET to COCO format. We indeed used these converted annotations to finetune the detector using mmdet.Really appreciate your answer!
I have a question that do I need to change a lot if I want training with multiple GPUs?
the function build_model in the builder.py
def build_model(cfg, **kwargs):
model = build_object(cfg, MODELS, **kwargs)
model = NNDataParallel(model, device_ids=[0])
return model
I try to change the device_ids to device_ids=[0,1,2] only , but it raises an error when running launch.py
Traceback (most recent call last):
File "/S1/MIPL/zhangzhuoran/code/present_work/ConsNet/ConsNet-main/tools/launch.py", line 72, in <module>
main()
File "/S1/MIPL/zhangzhuoran/code/present_work/ConsNet/ConsNet-main/tools/launch.py", line 68, in main
engine.launch(eval_mode=args.eval)
File "/home/zhangzr/anaconda3/envs/consnet_4/lib/python3.9/site-packages/nncore/engine/engine.py", line 514, in launch
self.run_stage()
File "/home/zhangzr/anaconda3/envs/consnet_4/lib/python3.9/site-packages/nncore/engine/engine.py", line 458, in run_stage
self.train_epoch()
File "/home/zhangzr/anaconda3/envs/consnet_4/lib/python3.9/site-packages/nncore/engine/engine.py", line 409, in train_epoch
self.train_iter(data)
File "/home/zhangzr/anaconda3/envs/consnet_4/lib/python3.9/site-packages/nncore/engine/engine.py", line 359, in train_iter
output = self.model(data, mode=self._mode, **self._kwargs)
File "/home/zhangzr/anaconda3/envs/consnet_4/lib/python3.9/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/zhangzr/anaconda3/envs/consnet_4/lib/python3.9/site-packages/torch/nn/parallel/data_parallel.py", line 153, in forward
raise RuntimeError("module must have its parameters and buffers "
RuntimeError: module must have its parameters and buffers on device cuda:0 (device_ids[0]) but found one of them on device: cpu
But I'm sure that I have four GPUs
And when I added the code
def build_model(cfg, **kwargs):
model = build_object(cfg, MODELS, **kwargs)
model = NNDataParallel(model, device_ids=[0,1,2])
device = torch.device("cuda:0" )
model.to(device)
return model
It raises another error when running launch.py
Traceback (most recent call last):
File "/S1/MIPL/zhangzhuoran/code/present_work/ConsNet/ConsNet-main/tools/launch.py", line 72, in <module>
main()
File "/S1/MIPL/zhangzhuoran/code/present_work/ConsNet/ConsNet-main/tools/launch.py", line 68, in main
engine.launch(eval_mode=args.eval)
File "/home/zhangzr/anaconda3/envs/consnet_4/lib/python3.9/site-packages/nncore/engine/engine.py", line 514, in launch
self.run_stage()
File "/home/zhangzr/anaconda3/envs/consnet_4/lib/python3.9/site-packages/nncore/engine/engine.py", line 458, in run_stage
self.train_epoch()
File "/home/zhangzr/anaconda3/envs/consnet_4/lib/python3.9/site-packages/nncore/engine/engine.py", line 409, in train_epoch
self.train_iter(data)
File "/home/zhangzr/anaconda3/envs/consnet_4/lib/python3.9/site-packages/nncore/engine/engine.py", line 359, in train_iter
output = self.model(data, mode=self._mode, **self._kwargs)
File "/home/zhangzr/anaconda3/envs/consnet_4/lib/python3.9/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/zhangzr/anaconda3/envs/consnet_4/lib/python3.9/site-packages/torch/nn/parallel/data_parallel.py", line 168, in forward
return self.gather(outputs, self.output_device)
File "/home/zhangzr/anaconda3/envs/consnet_4/lib/python3.9/site-packages/torch/nn/parallel/data_parallel.py", line 180, in gather
return gather(outputs, output_device, dim=self.dim)
File "/home/zhangzr/anaconda3/envs/consnet_4/lib/python3.9/site-packages/torch/nn/parallel/scatter_gather.py", line 76, in gather
res = gather_map(outputs)
File "/home/zhangzr/anaconda3/envs/consnet_4/lib/python3.9/site-packages/torch/nn/parallel/scatter_gather.py", line 69, in gather_map
return type(out)(((k, gather_map([d[k] for d in outputs]))
File "/home/zhangzr/anaconda3/envs/consnet_4/lib/python3.9/site-packages/torch/nn/parallel/scatter_gather.py", line 69, in <genexpr>
return type(out)(((k, gather_map([d[k] for d in outputs]))
File "/home/zhangzr/anaconda3/envs/consnet_4/lib/python3.9/site-packages/torch/nn/parallel/scatter_gather.py", line 71, in gather_map
return type(out)(map(gather_map, zip(*outputs)))
TypeError: 'int' object is not iterable
Sorry for my late reply. Currently, NNDataParallel
only supports single GPU training. If you would like to train on multiple GPUs, you may use NNDistributedDataParallel
, which is similar to DistributedDataParallel
. Some minor changes on the code should also be made. However, there is no need to use multiple GPUs for this model. If you want to increase the batch size, you can simply change the values of batch_size
in the config files.
I'm closing this issue. Please feel free to re-open it if you have any further questions.
Thanks for you work! I got some questions:
torch.cat(o_blob,h_blob)
to concatenate?