visualize_outputs does not find rafs

[richardvogg]

I trained a model with my own data (using "configs/quick_schedules/Quick-FCSGG-HRNet-W32.yaml") and saved the weights.

When I try to run visualize_outputs on my own minitest set, I get the error that the pred_rafs were not found. Indeed, when I print the output

with torch.no_grad(): for d in dataloader: output = model(d) print(output) visualize_detections(d, output, out_dir, object_class_names, predicate_class_names)

it contains a lot of information, but pred_rafs = None. This breaks the visualization. Any idea what could be the reason, why I do not get any relationship affinity fields, or where to look in the code to see where the pred_rafs are being created?

Thank you!

[richardvogg]

I took a more detailed look and can specify more exactly why the problem is ocurring.

In fcsgg/modeling/meta_arch/onestage_detector.py we have the CenterNet class. This has an inference() function which takes images and processes them.

In the postprocessing of the output, we call decoder functions from fcsgg/utils/centernet_utils.py -> CenterNetDecoder class.

There is the decode() function, which contains these lines: if isinstance(results, list):

this will modify detections

        self.decode_multiscale_relations(results, detections, image_sizes)

else: if self.raf_type == "vector": self.decode_relations(results, detections, output_stride=output_stride)

This always goes into the else part. However, the decode_relations function does not produce any "rafs", while the decode_multiscale_relations function does. This explains why the error from above is happening. But how do I convert the results into a list of a suitable dimension?

Thanks!

[liuhengyue]

It does not always go to the 'else' part. The key I believe should be 'rafs'. I probably did not include the predicted rafs as output nor implement the visualization for single-scale output.

[richardvogg]

Thanks! I added the predicted rafs for single-scale output and it is working now. I am wondering why it produces single-scale output and not multi-scale output...is there a difference between the Quick schedules and the normal configs?

[liuhengyue]

You need to set INPUT.GT_SCALE_AWARE to True, and HEADS.OUTPUT_STRIDES a list of 4 values. Please check the difference between configs/quick_schedules/Quick-FCSGG-Res50-BiFPN-P2P5-MultiScaleHead-512x512-MS.yaml