search

KaihuaTang / Scene-Graph-Benchmark.pytorch

A new codebase for popular Scene Graph Generation methods (2020). Visualization & Scene Graph Extraction on custom images/datasets are provided. It's also a PyTorch implementation of paper “Unbiased Scene Graph Generation from Biased Training CVPR 2020”
MIT License
1.03k stars 228 forks source link

Code for scene graph visualization #2

Open iamsusiep opened 4 years ago

iamsusiep commented 4 years ago

❓ Questions and Help

Is there code for scene graph visualization after running inference/evaluation?

KaihuaTang commented 4 years ago

My visualization codes are written in several independent jupyter notes, and not included in the main project by now. I will consider releasing them in the future after I organize all the codes.

Iceland-Leo commented 4 years ago

@KaihuaTang Have you organized the code for the visualization section? This is what I need urgently at present. Even if it is still in jupyter format, can it be made public? Thank you！

KaihuaTang commented 4 years ago

@KaihuaTang Have you organized the code for the visualization section? This is what I need urgently at present. Even if it is still in jupyter format, can it be made public? Thank you！

Check https://github.com/KaihuaTang/Scene-Graph-Benchmark.pytorch/blob/master/visualization/1.visualize_PredCls_and_SGCls.ipynb

bbox with label & index are drawn on the picture. ground-truth relations and predicted relations are printed as a list. Since the predicted relations are pretty dense, you need to set a threshold to show the top-K. Then I drew the graphs through ppt for the best visual effects.

Iceland-Leo commented 4 years ago

@KaihuaTang Thank you very much! One more question, when I start training, the glove will be download. But an error ocurred. File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/roi_relation_predictors.py", line 459, in __init__ self.context_layer = VCTreeLSTMContext(config, obj_classes, rel_classes, statistics, in_channels) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/model_vctree.py", line 93, in __init__ obj_embed_vecs = obj_edge_vectors(self.obj_classes, wv_dir=self.cfg.GLOVE_DIR, wv_dim=self.embed_dim) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/utils_motifs.py", line 152, in obj_edge_vectors wv_dict, wv_arr, wv_size = load_word_vectors(wv_dir, wv_type, wv_dim) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/utils_motifs.py", line 205, in load_word_vectors with zipfile.ZipFile(fname, "r") as zf: File "/home/cilab-cv/miniconda3/envs/sg/lib/python3.6/zipfile.py", line 1131, in __init__ self._RealGetContents() File "/home/cilab-cv/miniconda3/envs/sg/lib/python3.6/zipfile.py", line 1198, in _RealGetContents raise BadZipFile("File is not a zip file") zipfile.BadZipFile: File is not a zip file

KaihuaTang commented 4 years ago

I don't know. I‘ve never seen this bug before.

JincanDeng commented 4 years ago

@KaihuaTang Can you release the visualization code for sgdet results?

KaihuaTang commented 4 years ago

@KaihuaTang Can you release the visualization code for sgdet results?

It seems like I may lost this part of codes. The main difference between sgdet visualization and others is a bbox selection function (with threshold), otherwise there would be too many bboxes. I'm currently working on another project, maybe you can write your own.

navidre commented 4 years ago

@KaihuaTang Thanks for the great and tidy code.

@JincanDeng A PR is opened to add the SGDet visualization. Not many changes from the SGCls notebook.

runzeer commented 4 years ago

@navidre Thanks for your visualization for the SGDets code. Could you tell me how to get to your visualization results? Especially for the pred scores? The Numerical values seem very small. Why are your results are so big?

runzeer commented 4 years ago

@navidre if connenient, could you share the config file for test?

folkaholic commented 4 years ago

@KaihuaTang Thank you very much! One more question, when I start training, the glove will be download. But an error ocurred. File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/roi_relation_predictors.py", line 459, in __init__ self.context_layer = VCTreeLSTMContext(config, obj_classes, rel_classes, statistics, in_channels) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/model_vctree.py", line 93, in __init__ obj_embed_vecs = obj_edge_vectors(self.obj_classes, wv_dir=self.cfg.GLOVE_DIR, wv_dim=self.embed_dim) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/utils_motifs.py", line 152, in obj_edge_vectors wv_dict, wv_arr, wv_size = load_word_vectors(wv_dir, wv_type, wv_dim) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/utils_motifs.py", line 205, in load_word_vectors with zipfile.ZipFile(fname, "r") as zf: File "/home/cilab-cv/miniconda3/envs/sg/lib/python3.6/zipfile.py", line 1131, in __init__ self._RealGetContents() File "/home/cilab-cv/miniconda3/envs/sg/lib/python3.6/zipfile.py", line 1198, in _RealGetContents raise BadZipFile("File is not a zip file") zipfile.BadZipFile: File is not a zip file

Did you solve this problem？ I encountered the same problem. It is very strange that I downloaded glove.6B.zip in the glove folder, but every time I train, it will re-download a file called glove.6B.200d. Please help me, thank you!

KaihuaTang commented 4 years ago

@KaihuaTang Thank you very much! One more question, when I start training, the glove will be download. But an error ocurred. File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/roi_relation_predictors.py", line 459, in __init__ self.context_layer = VCTreeLSTMContext(config, obj_classes, rel_classes, statistics, in_channels) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/model_vctree.py", line 93, in __init__ obj_embed_vecs = obj_edge_vectors(self.obj_classes, wv_dir=self.cfg.GLOVE_DIR, wv_dim=self.embed_dim) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/utils_motifs.py", line 152, in obj_edge_vectors wv_dict, wv_arr, wv_size = load_word_vectors(wv_dir, wv_type, wv_dim) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/utils_motifs.py", line 205, in load_word_vectors with zipfile.ZipFile(fname, "r") as zf: File "/home/cilab-cv/miniconda3/envs/sg/lib/python3.6/zipfile.py", line 1131, in __init__ self._RealGetContents() File "/home/cilab-cv/miniconda3/envs/sg/lib/python3.6/zipfile.py", line 1198, in _RealGetContents raise BadZipFile("File is not a zip file") zipfile.BadZipFile: File is not a zip file

Did you solve this problem？ I encountered the same problem. It is very strange that I downloaded glove.6B.zip in the glove folder, but every time I train, it will re-download a file called glove.6B.200d. Please help me, thank you!

I'm not sure. Maybe changing the path of the customized GLOVE_DIR in the command?

Iceland-Leo commented 4 years ago

You can unzip the glove.6b.zip, then train the model.

发自我的iPhone

------------------ Original ------------------ From: Kaihua Tang <notifications@github.com> Date: Mon,Apr 13,2020 10:20 PM To: KaihuaTang/Scene-Graph-Benchmark.pytorch <Scene-Graph-Benchmark.pytorch@noreply.github.com> Cc: Liucheng <785091715@qq.com>, Comment <comment@noreply.github.com> Subject: Re: [KaihuaTang/Scene-Graph-Benchmark.pytorch] Code for scene graph visualization (#2)

@KaihuaTang Thank you very much! One more question, when I start training, the glove will be download. But an error ocurred. File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/roi_relation_predictors.py", line 459, in init self.context_layer = VCTreeLSTMContext(config, obj_classes, rel_classes, statistics, in_channels) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/model_vctree.py", line 93, in init obj_embed_vecs = obj_edge_vectors(self.obj_classes, wv_dir=self.cfg.GLOVE_DIR, wv_dim=self.embed_dim) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/utils_motifs.py", line 152, in obj_edge_vectors wv_dict, wv_arr, wv_size = load_word_vectors(wv_dir, wv_type, wv_dim) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/utils_motifs.py", line 205, in load_word_vectors with zipfile.ZipFile(fname, "r") as zf: File "/home/cilab-cv/miniconda3/envs/sg/lib/python3.6/zipfile.py", line 1131, in init self._RealGetContents() File "/home/cilab-cv/miniconda3/envs/sg/lib/python3.6/zipfile.py", line 1198, in _RealGetContents raise BadZipFile("File is not a zip file") zipfile.BadZipFile: File is not a zip file

Did you solve this problem？ I encountered the same problem. It is very strange that I downloaded glove.6B.zip in the glove folder, but every time I train, it will re-download a file called glove.6B.200d. Please help me, thank you!

I'm not sure. Maybe changing the path of the customized GLOVE_DIR in the command?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

folkaholic commented 4 years ago

@KaihuaTang Thank you very much! One more question, when I start training, the glove will be download. But an error ocurred. File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/roi_relation_predictors.py", line 459, in __init__ self.context_layer = VCTreeLSTMContext(config, obj_classes, rel_classes, statistics, in_channels) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/model_vctree.py", line 93, in __init__ obj_embed_vecs = obj_edge_vectors(self.obj_classes, wv_dir=self.cfg.GLOVE_DIR, wv_dim=self.embed_dim) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/utils_motifs.py", line 152, in obj_edge_vectors wv_dict, wv_arr, wv_size = load_word_vectors(wv_dir, wv_type, wv_dim) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/utils_motifs.py", line 205, in load_word_vectors with zipfile.ZipFile(fname, "r") as zf: File "/home/cilab-cv/miniconda3/envs/sg/lib/python3.6/zipfile.py", line 1131, in __init__ self._RealGetContents() File "/home/cilab-cv/miniconda3/envs/sg/lib/python3.6/zipfile.py", line 1198, in _RealGetContents raise BadZipFile("File is not a zip file") zipfile.BadZipFile: File is not a zip file

Did you solve this problem？ I encountered the same problem. It is very strange that I downloaded glove.6B.zip in the glove folder, but every time I train, it will re-download a file called glove.6B.200d. Please help me, thank you!

I'm not sure. Maybe changing the path of the customized GLOVE_DIR in the command?

Thank you for your prompt reply！My problem may solved by @Iceland-Leo .Because the zip file I downloaded have problem in unzipping, the system will re-download the unzip file of glove.6B.zip everytime I train the model.

folkaholic commented 4 years ago

You can unzip the glove.6b.zip, then train the model. 发自我的iPhone … ------------------ Original ------------------ From: Kaihua Tang <notifications@github.com> Date: Mon,Apr 13,2020 10:20 PM To: KaihuaTang/Scene-Graph-Benchmark.pytorch <Scene-Graph-Benchmark.pytorch@noreply.github.com> Cc: Liucheng <785091715@qq.com>, Comment <comment@noreply.github.com> Subject: Re: [KaihuaTang/Scene-Graph-Benchmark.pytorch] Code for scene graph visualization (#2) @KaihuaTang Thank you very much! One more question, when I start training, the glove will be download. But an error ocurred. File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/roi_relation_predictors.py", line 459, in init self.context_layer = VCTreeLSTMContext(config, obj_classes, rel_classes, statistics, in_channels) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/model_vctree.py", line 93, in init obj_embed_vecs = obj_edge_vectors(self.obj_classes, wv_dir=self.cfg.GLOVE_DIR, wv_dim=self.embed_dim) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/utils_motifs.py", line 152, in obj_edge_vectors wv_dict, wv_arr, wv_size = load_word_vectors(wv_dir, wv_type, wv_dim) File "/mnt/disk/cv/Liucheng/pytorchCode/Scene-Graph-Benchmark.pytorch-master/maskrcnn_benchmark/modeling/roi_heads/relation_head/utils_motifs.py", line 205, in load_word_vectors with zipfile.ZipFile(fname, "r") as zf: File "/home/cilab-cv/miniconda3/envs/sg/lib/python3.6/zipfile.py", line 1131, in init self._RealGetContents() File "/home/cilab-cv/miniconda3/envs/sg/lib/python3.6/zipfile.py", line 1198, in _RealGetContents raise BadZipFile("File is not a zip file") zipfile.BadZipFile: File is not a zip file Did you solve this problem？ I encountered the same problem. It is very strange that I downloaded glove.6B.zip in the glove folder, but every time I train, it will re-download a file called glove.6B.200d. Please help me, thank you! I'm not sure. Maybe changing the path of the customized GLOVE_DIR in the command? — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

Thank you for your prompt reply！Problem solved!

dami23 commented 3 years ago

thank you for the share. A question about the visualize code, what is the definition of 'dispaly' in draw_iamge? Thank you!