How can we use SoM for video segmentation

[QinghongLin]

Hi, thanks for the awesome works! wondering have you provided any codes or instructions about how to use SoM for video segmentation or tracking?

[jwyang]

thanks for your interest! We are going to release all evaluation scripts for the tasks in our paper very soon!

[QinghongLin]

Thanks for response, btw, I meet such error regarding MPI, have you meet it in previously :D

*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[cr1-p4de24xlarge-2:367084] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed

[jwyang]

I think you can skip this MPI installation, we intended to remove the dependencies on MPI but please let me know if it does not work without MPI.

[QinghongLin]

It happen when I run demo_som.py, I think it caused by seem. If want to skip this, can you point out which rows I should comment.

[jwyang]

got it, let me try to figure it out.

[jwyang]

Hi, @QinghongLin, had just remove the dependencies on MPI for SEEM, can you install seem again and see whether it works?

[QinghongLin]

Hi @jwyang , Thanks for the update! I removed the: opt_seem = init_distributed_seem(opt_seem) in line 49 of demo_som.py and replace the state_dict = torch.load(load_dir, map_location=self.opt['device']) as state_dict = torch.load(load_dir, map_location='cpu')

it can work. Thanks a lot!

[jwyang]

awesome!

[abrichr]

See https://github.com/microsoft/SoM/issues/12#issuecomment-1930432657 for an explanation and fix for the MPI issue.