real world scalability?

leeesangwon / PyTorch-Image-Retrieval

A PyTorch framework for an image retrieval task including implementation of N-pair Loss (NIPS 2016) and Angular Loss (ICCV 2017).

MIT License

275 stars 55 forks source link

real world scalability? #13

Closed batrlatom closed 5 years ago

batrlatom commented 5 years ago

Hi, in db_augmentation and average_query_expansion you are calculating similarity matrix. But when I want to search for example in cub200_2011 dataset, you have 20k+ reference points and do not have enough memory for this. In fact, my computer with 16GB ram and gtx1080ti runs out of memory even if I search among 50 images.

Do you have any sugestions how to scale this? Thanks, T

batrlatom commented 5 years ago

My bad, looks like it was just about the batch size