search

z-x-yang / Segment-and-Track-Anything

An open-source project dedicated to tracking and segmenting any objects in videos, either automatically or interactively. The primary algorithms utilized include the Segment Anything Model (SAM) for key-frame segmentation and Associating Objects with Transformers (AOT) for efficient tracking and propagation purposes.
GNU Affero General Public License v3.0
2.75k stars 332 forks source link

UserWarning: Failed to load custom C++ ops. Running on CPU mode Only! #152

Open ouyangbetter opened 4 months ago

ouyangbetter commented 4 months ago

/home/user/repos/Segment-and-Track-Anything/src/groundingdino/groundingdino/models/GroundingDINO/ms_deform_attn.py:31: UserWarning: Failed to load custom C++ ops. Running on CPU mode Only!

TangZhengxu commented 3 months ago

1. Create and activate a new environment

conda create -n sta_env python=3.10 conda activate sta_env

2. Clone the repository

git clone https://github.com/z-x-yang/Segment-and-Track-Anything.git cd Segment-and-Track-Anything

3. Install PyTorch with CUDA 11.8

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

4. Install the sam package

cd sam pip install -e . cd ..

5. Install GroundingDINO

pip install -e git+https://github.com/IDEA-Research/GroundingDINO.git@main#egg=GroundingDINO

6. Install required packages

pip install numpy opencv-python pycocotools matplotlib scikit-image

7. Install compatible version of Pillow and supervision

pip install Pillow==9.5.0 pip install supervision

8. Install additional packages

pip install gradio==3.39.0 gdown ffmpeg==1.4

9. Clone and set up Pytorch-Correlation-extension

git clone https://github.com/ClementPinard/Pytorch-Correlation-extension.git cd Pytorch-Correlation-extension pip install -e . cd ..

10. Create ckpt directory and download model files

mkdir ckpt cd ckpt

Download sam_vit_b_01ec64.pth, R50_DeAOTL_PRE_YTB_DAV.pth, groundingdino_swint_ogc.pth

cd ..

11. Run the application

python app.py

Here's how I solve, you also need to check your gcc version and cuda (use "which nvcc"), if there's nothing, you need to download it

(wget https://developer.download.nvidia.com/compute/cuda/11.8.0/local_installers/cuda_11.8.0_520.61.05_linux.run ) (sh cuda_11.8.0_520.61.05_linux.run --silent --toolkit --override --installpath=$HOME/cuda ) (export CUDA_HOME=$HOME/cuda export PATH=$CUDA_HOME/bin:$PATH export LD_LIBRARY_PATH=$CUDA_HOME/lib64:$LD_LIBRARY_PATH ) (echo 'export CUDA_HOME=$HOME/cuda' >> ~/.bashrc echo 'export PATH=$CUDA_HOME/bin:$PATH' >> ~/.bashrc echo 'export LD_LIBRARY_PATH=$CUDA_HOME/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc source ~/.bashrc )

here's how to update gcc

  1. wget https://ftp.gnu.org/gnu/gcc/gcc-9.3.0/gcc-9.3.0.tar.gz
  2. tar -xzf gcc-9.3.0.tar.gz
  3. cd gcc-9.3.0
  4. ./contrib/download_prerequisites
  5. mkdir ../gcc-build cd ../gcc-build
  6. ../gcc-9.3.0/configure --prefix=$HOME/gcc-9.3.0 --disable-multilib
  7. make -j$(nproc)
  8. make install
  9. export PATH=$HOME/gcc-9.3.0/bin:$PATH export LD_LIBRARY_PATH=$HOME/gcc-9.3.0/lib64:$LD_LIBRARY_PATH
  10. echo 'export PATH=$HOME/gcc-9.3.0/bin:$PATH' >> ~/.bashrc echo 'export LD_LIBRARY_PATH=$HOME/gcc-9.3.0/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc source ~/.bashrc