ShirAmir / dino-vit-features

Official implementation for the paper "Deep ViT Features as Dense Visual Descriptors".
https://dino-vit-features.github.io
MIT License
383 stars 44 forks source link

Extractor feature OOM #9

Open StarsTesla opened 2 years ago

StarsTesla commented 2 years ago

Hi there, I tried the code, at the beginning , everything sames fine, but when I try to use the extractor on my own images, more specificly, extract feature of high resolution pciture and visualize the pca picture. I tried the code on 100 800 x 800 images, set the load_size=224 and stride=2, it seems fine, so the code maybe run the image separately?

How should I calculate the memory of GPU and Could the Extractor could modify to using on multiple GPUs?

ShirAmir commented 2 years ago

Hi @StarsTesla,

The current PCA computation code extracts features from each image using the GPU. AFAIK, There are two possible options to the OOM error you receive: (1) Your GPU doesn't contain enough RAM to extract features for a single high resolution image. (You receive the OOM error before even extracting a single image). In this case you can either (i) run on CPU (by setting device='cpu' here). (ii) reduce load_size increase stride (e.g. load_size=224, stride=8).

(2) Your GPU manages to extract a single image in this manner and doesn't handle properly many extracted images (You receive the OOM error after extracting features for several images). In this case I would recommend: (i) Running on CPU in the same manner as (1)(i). (ii) Create a script that extracts features for a single image and saves the features do disk, run it with GPU on all the images (the GPU RAM will be freed between each run of the script) and alter the PCA script to load descriptors from disc. (iii) Purge the GPU RAM after each features extraction in the original script.

Hope this helps, and good luck!

StarsTesla commented 1 year ago

@ShirAmir Well,Is there a way to make a high resolution extraction? Upsampling seems lose alot of details.