search

CodeMonsterPHD / GaTector-A-Unified-Framework-for-Gaze-Object-Prediction

This repository is the official implementation of GaTector, which studies the newly proposed task, gaze object prediction. In this work, we build a novel framework named GaTector to tackle the gaze object prediction problem in a unified way. Particularly, a specific-general-specific (SGS) feature extractor is firstly proposed to utilize a shared backbone to extract general features for both scene and head images. To better consider the specificity of inputs and tasks, SGS introduces two input-specific blocks before the shared backbone and three task-specific blocks after the shared backbone. Specifically, a novel defocus layer is designed to generate object-specific features for object detection task without losing information or requiring extra computations. Moreover, the energy aggregation loss is introduced to guide the gaze heatmap to concentrate on the stared box. In the end, we propose a novel mDAP metric that can reveal the difference between boxes even when they share no overlapping area. Extensive experiments on the GOO dataset verify the superiority of our method in all three tracks, i.e., object detection, gaze estimation, and gaze object prediction.
https://arxiv.org/abs/2112.03549
64 stars 11 forks source link

Memory #1

Open cjxjbd opened 1 year ago

cjxjbd commented 1 year ago

When I try to reproduce your code, starting from the second epoch, the memory will be full. Is there any solution? Thanks!

您好，我在尝试复现您的代码的时候，从第二个epoch开始，运存会占满，请问有什么解决办法吗？谢谢！

cjxjbd commented 1 year ago

When using the goosynth dataset.

huTao1030 commented 1 year ago

When I try to reproduce your code, starting from the second epoch, the memory will be full. Is there any solution? Thanks!

您好，我在尝试复现您的代码的时候，从第二个epoch开始，运存会占满，请问有什么解决办法吗？谢谢！

非常感谢您对我们的工作的关注。 Goosynth数据集所有数据标签被打包成一个单独的pickle文件，里面包含了非常丰富的标注类型，其中场景中每件物品的语义分割标注占了绝大部分存储空间并且在我们的工作中是没有被使用的（我们使用目标检测相关标签）。 您可以尝试将pickle文件进行拆分并修改DataLoader的代码，或者删除标注文件中语义分割部分的相关内容以减少在训练过程中的运行内存的占用问题。 如果显卡显存不够可以尝试缩小batchsize以保证代码可以正常运行。如果有什么其他问题，欢迎提问。