logic problem in flow_path ?

[HenrySomeCode]

I can't run the inference code using this:

python evaluation.py --model=flowisam --dataset=dvs16 --flow_gaps=-1 \
                      --max_obj=5 --num_gridside=10 \
                      --ckpt_path=frame_level_flowisam_vitb_train_on_oclrsyn_dvs16.pth \
                      --save_path=outputs

I suggest that the flow_path is always going to be 1 image less than the rgb_path or anno_path. This is because 2 rgb images will be converted into 1 flow image. For example, if there are 50 images in DAVIS2016/JPEGImages/480p/blackswan, there are going to be 49 images in DAVIS2016/FlowImages_gap1

I changed the code below of dvs_loader.py:

for idx, seq in enumerate(sorted(self.seqs)):
    if idx % 100 == 0:  print("Loading validation sequence: {}".format(idx))
    seq_dir = os.path.join(self.anno_dir, seq)
    for filename in sorted(os.listdir(seq_dir)):              
        flow_path = os.path.join(seq_dir.replace(self.anno_dir, self.flow_dir), filename.replace(".jpg", ".png"))
        rgb_path = os.path.join(seq_dir.replace(self.anno_dir, self.rgb_dir), filename.replace(".png", ".jpg"))
        self.anno_paths.append(os.path.join(seq_dir, filename.replace(".jpg", ".png")))
        self.flow_paths.append(flow_path)
        self.rgb_paths.append(rgb_path)

to this:

for idx, seq in enumerate(sorted(self.seqs)):
        if idx % 100 == 0:  print("Loading validation sequence: {}".format(idx))
        seq_dir = os.path.join(self.anno_dir, seq)

        last_item = sorted(os.listdir(seq_dir))[-1]
        print("fr dvs_loader.py: ",last_item)

        for filename in sorted(os.listdir(seq_dir)):  
            if filename != last_item:
                flow_path = os.path.join(seq_dir.replace(self.anno_dir, self.flow_dir), filename.replace(".jpg", ".png"))
                self.flow_paths.append(flow_path)

            rgb_path = os.path.join(seq_dir.replace(self.anno_dir, self.rgb_dir), filename.replace(".png", ".jpg"))
            self.anno_paths.append(os.path.join(seq_dir, filename.replace(".jpg", ".png")))
            self.rgb_paths.append(rgb_path)

And the inference code run completely but some images inside outputs/hung/blackswan are all black. Am I right?

[Jyxarthur]

Thanks for pointing this out. In the __init__ function in the data loader, all flow paths are initialized with a placeholder directory "FlowImages_gap1/xxxxx.png", no matter the file's existence.

This will be then tackled in train_loading / val_loading. For example,

https://github.com/Jyxarthur/flowsam/blob/1b61de686bff2cc1a72b15a9f66b3bfe4b64ccbc/data/dataloaders/dvs_loader.py#L257-L271

During which

• The "FlowImages_gap1" will be replaced by different flow gaps, e.g., "FlowImages_gap-1", "FlowImages_gap2", etc.
• The file's existence will be checked. For example, as "FlowImages_gap-1/00000.png" does not exist, the file "FlowImages_gap1/00000.png" (-1 * -1 = 1) will then be automatically loaded to compensate for the missing file.