search

davidsandberg / facenet

Face recognition using Tensorflow
MIT License
13.72k stars 4.8k forks source link

Batch Size for Online Triplet Mining #1191

Open Neihtq opened 3 years ago

Neihtq commented 3 years ago

Hi,

I read through the official paper of FaceNet and there it is stated, that a batch size of 1800 is used for online triplet mining. This number seems to be quite high. I have acces to an IBM Power Instance with a 32GB Nvidia Tesla V100 GPU but having a batch size that large with images from the LFW is infeasible.

Is the triplet mining performed on CPU? I tried to create an embedding of one batch (with size 1800) on aformentioned IBM instance. However, my jupyternotebook crashes - I assume that the batch size is still too large.

The triplet mining on my side performs Batch Hard Mining. How should I determine a good batch size?

Jakub-Svoboda commented 3 years ago

Hi, I don't have the answer for you, but I got puzzled by the same thing when reading this paper. I can fit up to 128 images into a single batch and I don't think there is a GPU which could fit all the 1800 images per batch into memory as they say in the paper. Therefore, I suspect that what they mean by "mini-batch" in their paper is not an actual training batch that they send to GPU at one time, but rather a subset of images which they work with to select triplets. I suspect that they go through this mini-batch in smaller batches and only then find the semi hard examples from the whole 1800 images.