search

rajatkoner08 / Graphhopper

This is a code repository of Graphhopper: Multi-Hop Scene GraphReasoning for Visual Question Answering
Apache License 2.0
18 stars 1 forks source link

About code of another paper RTN #5

Open zmx1712 opened 2 years ago

zmx1712 commented 2 years ago

Dear scholar, is this project have the code about "scene graph generation for GQA images" or anything else related to another paper "Relation Transformer Network"? If not, How long may I need wait to get the "RTN" code for GQA in "rtn" this github project? Thank you.

rajatkoner08 commented 2 years ago

i have uploaded the draft code for GQA, its is a draft version...in case of issue contact me, will reply asap

zmx1712 commented 2 years ago

Thanks for your reply, but there are still some questions,

  1. the _batchlist.txt,_batch_listfull.txt and _batch_listorig.txt in the rtn-gqa_1.4/models are messy code under UTF-8, are they not coded by utf-8, or they are not important?
  2. can the code in rtn-gqa be trained under pytorch==1.7, as the swin transformer requires pytorch1.7.1.
  3. I can't find the dict files of object classes and relations in the project . your reply is appreciated
rajatkoner08 commented 2 years ago
  1. batch list .txt files are used only for debugging, so u can ignore them
  2. yes it sud work, also u can get rid of ROI align and NMS like function in 1.7
  3. Do u need the refined (800 cls and 170 relations?) data? or can u specify which file is missing..i can provide those ...please let me know or drop me an email
zmx1712 commented 2 years ago

Yes, actually I want to see the detailed scene graph output of GQA dataset in your model that may help me learn the SGDET better.

But when I try to run the model code under linux, I meet some troubles：

  1. The first step in README let me use the pip install -r requirement.txt to download dependences, however, I can't find this file except for another requirements.txt under the mmdetection folder. Is this requirements.txt that I should run at first step? If it is, that seems to contradict with 3rd step under mmdetection folder.
  2. If I want to see the last result of scene graph directly, Which python file should I run, the eval_rels.py or eval_inference.py ?
  3. About the data file, as I want to see the last results, so I guess that the dict_file about the obj cls and rels of GQA images must be needed, but the project in the github does not have the data folder. And as your paper rtn writes, the number of cls and relations in your dict is more than motifi this project, I'm interested in that. So could you provide the data files or other necessary files which can help run the code under GQA images or tell me where can I get them?

Besides, I also have several questions about the details of rtn this paper:

  1. The Object Detection you last use under GQA is Faster R-CNN or Mask R-CNN?
  2. Did you train the model from scratch with the GQA data set or just use the pre-trained model and fine-tuning？The paper does not give the test results of GQA data under the SGDET task, so can this code complete the SGDET task under GQA data?
  3. How much time was your model trained(I see your paper says 20 epoch costs about 2 days) and how much training data was used under GQA?
  4. Did you just train the GQA model on a RTX 3090 GPU with 24GB memory and 64GB RAM? Thx anyway.