High Memory Usage During Initialization

[kaankarakose]

Hi, thank you so much for sharing this great work with us.

I've been working with your repository and have encountered a situation where the software requires a significant amount of memory for initialization (demo_automatic.py). Specifically, during initialization, it allocates around 35GiB of memory for a moment, and subsequently, for processing each frame, it uses about 22GiB. Because of this, it cannot run on 24GiB GPU.

Are there any recommended strategies or modifications to reduce the memory usage for initialization?

Here my configs: Configuration: {'model': './saves/DEVA-propagation.pth', 'output': '/deva_output', 'save_all': True, 'amp': True, 'key_dim': 64, 'value_dim': 512, 'pix_feat_dim': 512, 'disable_long_term': False, 'max_mid_term_frames': 10, 'min_mid_term_frames': 5, 'max_long_term_elements': 10000, 'num_prototypes': 128, 'top_k': 30, 'mem_every': 5, 'chunk_size': 2, 'size': 360, 'GROUNDING_DINO_CONFIG_PATH': './saves/GroundingDINO_SwinT_OGC.py', 'GROUNDING_DINO_CHECKPOINT_PATH': './saves/groundingdino_swint_ogc.pth', 'DINO_THRESHOLD': 0.35, 'DINO_NMS_THRESHOLD': 0.8, 'SAM_ENCODER_VERSION': 'vit_h', 'SAM_CHECKPOINT_PATH': './saves/sam_vit_h_4b8939.pth', 'MOBILE_SAM_CHECKPOINT_PATH': './saves/mobile_sam.pt', 'SAM_NUM_POINTS_PER_SIDE': 64, 'SAM_NUM_POINTS_PER_BATCH': 64, 'SAM_PRED_IOU_THRESHOLD': 0.88, 'SAM_OVERLAP_THRESHOLD': 0.8, 'img_path': 'test.mp4', 'detection_every': 5, 'num_voting_frames': 3, 'temporal_setting': 'online', 'max_missed_detection_count': 5, 'max_num_objects': 50, 'sam_variant': 'mobile', 'suppress_small_objects': True, 'enable_long_term': True, 'enable_long_term_count_usage': True}

Thank you so much.

[hkchengrex]

Thank you for letting me know. I will look into this.

[QiruiH]

I'm having the same issue, have you solved this problem?