search

bknyaz / sgg

Train Scene Graph Generation for Visual Genome and GQA in PyTorch >= 1.2 with improved zero and few-shot generalization.
https://arxiv.org/abs/2007.05756
Other
130 stars 20 forks source link

I am getting assertion error like this what it means and how solve this? #4

Closed Monesh-97 closed 3 years ago

Monesh-97 commented 3 years ago

Screenshot (1)

donggu-kang commented 3 years ago

Oh my god, I had the same problem.

This issue occurs with the files and versions below.

@moneshj

File Location: https://github.com/bknyaz/sgg/blob/master/Visualize_SG_predictions.ipynb

Commit No: https://github.com/bknyaz/sgg/commit/a329844d3311e38ad187db339ede05deedee116b#diff-0555baa09fe0deb86a957b7537cceb082cfbdf906779aed72c4429aad0d9f0e9

donggu-kang commented 3 years ago

In [3] :

train, val_splits = VG.splits(data_dir=data_dir,
                              num_val_im=5000,
                              mrcnn=detector_model == 'mrcnn',
                              min_graph_size=-1,
                              max_graph_size=-1,
                              filter_non_overlap=mode == 'sgdet')

train_loader, val_loaders = VGDataLoader.splits(train, val_splits,
                                               mode='rel',
                                               batch_size=1,
                                               num_workers=0,
                                               num_gpus=1)

val_loader, val_loader_zs, test_loader, test_loader_zs = val_loaders

detector = RelModelStanford(train_data=train,
                            num_gpus=1,
                            mode=mode,
                            use_bias=False,
                            detector_model=detector_model,
                            RELS_PER_IMG=1024)

print("Loading EVERYTHING from %s" % checkpoint_path)
ckpt = torch.load(checkpoint_path, map_location='cpu')
success = optimistic_restore(detector, ckpt['state_dict'])

assert success, 'something wrong when loading the detector'
detector.to(device)
detector.eval()
detector.detector.eval()
detector.detector.mode = 'refinerels' if mode == 'sgdet' else 'gtbox'
print('done')

ERROR LOG :

TRAIN DATASET
Loading GQA-train questions...
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-3-a4a33e4a7f77> in <module>
      4                               min_graph_size=-1,
      5                               max_graph_size=-1,
----> 6                               filter_non_overlap=mode == 'sgdet')
      7 
      8 train_loader, val_loaders = VGDataLoader.splits(train, val_splits,

~/donggu/sgg/dataloaders/visual_genome.py in splits(cls, *args, **kwargs)
    315         """ Helper method to generate splits of the dataset"""
    316         print('\nTRAIN DATASET')
--> 317         train = cls('train', *args, **kwargs)
    318 
    319         print('\nVAL DATASET (ALL)')

~/donggu/sgg/dataloaders/visual_genome.py in __init__(self, mode, data_dir, filter_empty_rels, num_im, num_val_im, filter_duplicate_rels, filter_non_overlap, max_graph_size, min_graph_size, mrcnn, device, training_triplets, exclude_left_right)
    116                 # Use only images having question-answer pairs in the balanced split
    117                 print('Loading GQA-%s questions...' % mode)
--> 118                 with open(os.path.join(data_dir, data_name, '%s_balanced_questions.json' % f_mode), 'rb') as f:
    119                     Q_dict = json.load(f)
    120                 self.image_ids = set()

FileNotFoundError: [Errno 2] No such file or directory: '/scratch/ssd/data/GQA/train_balanced_questions.json'
bknyaz commented 3 years ago

It looks like there were some issues downloading GQA data. I suggest to download it manually using the link from here: https://github.com/bknyaz/sgg/blob/d2a78168cfcec95f27a1e6e8d66e636ed62d9736/lib/download.py#L12 And unpack according to the folder structure I describe in README.md

donggu-kang commented 3 years ago

My problem seems to be almost solved, but it still gives the same error.

  1. Directly access the URL mentioned on your laptop or desktop. (Cannot download with wget, etc.) And upload this file to the target server. 
    https://yadi.sk/d/FGOzRP649rZ2kQ

image

  1. The downloaded file is downloaded as .gz.

  2. Uncompress using the command gzip -d GQA_scenegraphs.gz. Caution! : The downloaded file is "GQA_scenegraphs.gz", but you need to change the file name to "GQA_scenegraphs.tar.gz" using the mv command and unzip it using the tar -zxvf command!

  3. When uncompressed, the following files are shown in the list.

    ./
./readme.txt
./readme_q.txt
./test_balanced_questions.json
./testdev_balanced_questions.json
./train_balanced_questions.json
./val_balanced_questions.json
./sceneGraphs/
./sceneGraphs/train_sceneGraphs.json
./sceneGraphs/val_sceneGraphs.json
./sceneGraphs/readme_sg.txt
./.submission_all_questions.json.swp
./train_images.json
./val_images.json

  4. Move the file to the correct location using the command below.

    mv ./"Downloaded Files dir"/. data_path/GQA/.

  5. data_path/GQA/.

    ubuntu@nipa2019-0494:~/donggu/sgg/data_path/GQA$ ll -h
total 1.1G
drwxr-xr-x 3 ubuntu ubuntu  327 Feb 20 22:50 ./
drwxrwxr-x 4 ubuntu ubuntu   27 Feb 16 11:54 ../
-rw-rw-r-- 1 ubuntu ubuntu 165M Feb 20 22:37 GQA_scenegraphs.tar
-rw-r--r-- 1 ubuntu ubuntu 2.2K Mar 26  2019 readme_q.txt
-rw-r--r-- 1 ubuntu ubuntu  920 Feb  1  2019 readme.txt
drwxr-xr-x 2 ubuntu ubuntu   85 Oct 26  2019 sceneGraphs/
-rw-r--r-- 1 ubuntu ubuntu 1.0K Nov  5  2019 .submission_all_questions.json.swp
-rw-r--r-- 1 ubuntu ubuntu  11M Feb  3  2019 test_balanced_questions.json
-rw-r--r-- 1 ubuntu ubuntu  11M Mar 26  2019 testdev_balanced_questions.json
-rw-r--r-- 1 ubuntu ubuntu 776M Feb  3  2019 train_balanced_questions.json
-rw-r--r-- 1 ubuntu ubuntu 764K Mar 30  2020 train_images.json
-rw-r--r-- 1 ubuntu ubuntu 109M Feb  3  2019 val_balanced_questions.json
-rw-r--r-- 1 ubuntu ubuntu 109K Mar 30  2020 val_images.json

  6. I followed your instructions, but I have the same problem. Please tell me if I am doing it wrong.

    
TRAIN DATASET
Loading GQA-train questions...
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-3-a4a33e4a7f77> in <module>
  4                               min_graph_size=-1,
  5                               max_graph_size=-1,
----> 6                               filter_non_overlap=mode == 'sgdet')
  7 
  8 train_loader, val_loaders = VGDataLoader.splits(train, val_splits,

~/donggu/sgg/dataloaders/visual_genome.py in splits(cls, *args, *kwargs) 315 """ Helper method to generate splits of the dataset""" 316 print('\nTRAIN DATASET') --> 317 train = cls('train', args, **kwargs) 318 319 print('\nVAL DATASET (ALL)')

~/donggu/sgg/dataloaders/visual_genome.py in init(self, mode, data_dir, filter_empty_rels, num_im, num_val_im, filter_duplicate_rels, filter_non_overlap, max_graph_size, min_graph_size, mrcnn, device, training_triplets, exclude_left_right) 116 # Use only images having question-answer pairs in the balanced split 117 print('Loading GQA-%s questions...' % mode) --> 118 with open(os.path.join(data_dir, data_name, '%s_balanced_questions.json' % f_mode), 'rb') as f: 119 Q_dict = json.load(f) 120 self.image_ids = set()

FileNotFoundError: [Errno 2] No such file or directory: '/scratch/ssd/data/GQA/train_balanced_questions.json'


8. When I run the source code below, I suddenly get this error. can I ignore this notification when running your source code?

```python
import torch
import numpy as np
import pandas as pd
import time
import pickle
import cv2
import networkx as nx
import matplotlib.pyplot as plt
from collections import defaultdict
from config import BOX_SCALE, IM_SCALE
from dataloaders.visual_genome import VGDataLoader, VG
donggu-kang commented 3 years ago

Looks like you found the problem!

I have confirmed that there are no files in these paths.

/scratch/ssd/data/

Therefore, I modified the source code to an absolute path to fit my path. The source code below is the part. Make sure to correct your path correctly too.

# data_dir = '/scratch/ssd/data/'
# I have confirmed that there are no files in these paths.
# "/scratch/ssd/data/"

# Therefore, I modified the source code to an absolute path to fit my path.
data_dir = '/home/ubuntu/donggu/sgg/data_path/'

VG.split = 'gqa'  # set VG, GQA or VTE split here to use as a global variable
checkpoint_path = '/mnt/data/bknyazev/checkpoints/VG/62566/vgrel.pth'  # 'gqa_sggen_62566.pth'
mode = 'sgdet'
device = 'cpu'
num_gpus = 1
detector_model = 'mrcnn'

# should be loaded after settings some params
from lib.pytorch_misc import *
from lib.evaluation.sg_eval import BasicSceneGraphEvaluator, calculate_mR_from_evaluator_list, eval_entry
from lib.rel_model_stanford import RelModelStanford

If you modify the source code as above and then continue running...

image

bknyaz commented 3 years ago

Yes, in the notebook you need to change the path appropriately.

Monesh-97 commented 3 years ago

Thank you @bknyaz

carloscaetano commented 2 months ago

Sorry for reopening this, but where can I download the ckpt mentioned in the notebook, @bknyaz and @donggu-kang ?