search

rowanz / neural-motifs

Code for Neural Motifs: Scene Graph Parsing with Global Context (CVPR 2018)
https://rowanzellers.com/neuralmotifs
MIT License
525 stars 114 forks source link

Generating Scene Graphs for random Images #26

Open ritwickchaudhry opened 6 years ago

ritwickchaudhry commented 6 years ago

@rowanz , I wanted to generate scene graphs for images not belonging to the Visual Genome dataset, using your model. Can I do this using the existing codebase? If yes, can you please guide me through it?

rowanz commented 6 years ago

You should be able to! Just change the dataloader to load from your own dataset. You’ll need to provide bounding boxes if using SGCls mode, boxes and labels if Predcls, or just use SGDet (nothing needed)

On Mon, Oct 1, 2018 at 2:35 AM Ritwick Chaudhry notifications@github.com wrote:

@rowanz https://github.com/rowanz , I wanted to generate scene graphs for images not belonging to the Visual Genome dataset, using your model. Can I do this using the existing codebase? If yes, can you please guide me through it?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/rowanz/neural-motifs/issues/26, or mute the thread https://github.com/notifications/unsubscribe-auth/ABWJx5HqWfvddvDAKRBDv_YoDPJKFgjtks5ugeHIgaJpZM4XBsuk .

ritwickchaudhry commented 6 years ago

Thanks a lot for the help @rowanz . I'll try to get it to work, and get back. Thanks!

ritwickchaudhry commented 6 years ago

@rowanz , I'm still facing some issues. For example, in SGDet mode, there should be no Ground Truth right? (I just want to produce scene graphs from images, and I don't have any ground truth available).

The data loaders by default load the gt_boxes and gt_relations. What exactly am I supposed to do about those in the data loader. The blobs also process these for the collate. I'm currently stuck there. In SGDet mode, functions like get_counts also access the gt elements.

Can you please help me through this?

Also, the rel_model_stanford seems to not have gt elements in sgdet mode, but the checkpoint isn't available. Can you please provide that?

Thanks!

rowanz commented 6 years ago

I'm still facing some issues. For example, in SGDet mode, there should be no Ground Truth right? (I just want to produce scene graphs from images, and I don't have any ground truth available).

The data loaders by default load the gt_boxes and gt_relations. What exactly am I supposed to do about those in the data loader. The blobs also process these for the collate. I'm currently stuck there.

Yep, you can replace the gt boxes with zeros and that should be okay. You don't need get_counts for running MotifNet.

Also, the rel_model_stanford seems to not have gt elements in sgdet mode, but the checkpoint isn't available. Can you please provide that?

Not sure what you mean by this (none of the models should). I don't support the checkpoints for this model, but you can reproduce my results by following the training script for that :)

hengqiali commented 5 years ago

@ritwickchaudhry I also want to generate scene graphs for COCO datasets without any groundtruth.Did you succeeded??

Thoudancer commented 5 years ago

@ritwickchaudhry hello, I met the same question as you. I want to use the trained sgdet model to generate sg, but i own only image, without files like image_data.json, VG-SGG.h5, VG-SGG-dicts.json and proposals.h5. If I set None to these, suffered error about get_counts(). Would you like to share the modification about the code?

ankitsamal commented 4 years ago

@rowanz , I'm still facing some issues. For example, in SGDet mode, there should be no Ground Truth right? (I just want to produce scene graphs from images, and I don't have any ground truth available).

The data loaders by default load the gt_boxes and gt_relations. What exactly am I supposed to do about those in the data loader. The blobs also process these for the collate. I'm currently stuck there. In SGDet mode, functions like get_counts also access the gt elements.

Can you please help me through this?

Also, the rel_model_stanford seems to not have gt elements in sgdet mode, but the checkpoint isn't available. Can you please provide that?

Thanks!

Hey Ritwik,

Were you able to figure this out. I am stuck in the same place trying to figure out how to input custom images

forbiddenname commented 4 years ago

@rowanz , I'm still facing some issues. For example, in SGDet mode, there should be no Ground Truth right? (I just want to produce scene graphs from images, and I don't have any ground truth available). The data loaders by default load the gt_boxes and gt_relations. What exactly am I supposed to do about those in the data loader. The blobs also process these for the collate. I'm currently stuck there. In SGDet mode, functions like get_counts also access the gt elements. Can you please help me through this? Also, the rel_model_stanford seems to not have gt elements in sgdet mode, but the checkpoint isn't available. Can you please provide that? Thanks!

Hey Ritwik,

Were you able to figure this out. I am stuck in the same place trying to figure out how to input custom images

hi ！Have you solved this problem? tks

forbiddenname commented 4 years ago

Thanks a lot for the help @rowanz . I'll try to get it to work, and get back. Thanks!

hi ！Have you solved this problem? tks

forbiddenname commented 4 years ago

@ritwickchaudhry I also want to generate scene graphs for COCO datasets without any groundtruth.Did you succeeded??

hi ！Have you solved this problem? tks

alice-cool commented 3 years ago

https://github.com/rowanz/neural-motifs/issues/26#issuecomment-427087900 but at now the project only use 150 object classes. My detector detect more object class, but the project will fail to more object classes.

Yudezhi commented 2 years ago

Same issue here. comment for subscribe.