Open kritiagg opened 4 years ago
Hi, good questions, sorry for the late answer.
Thanks Lucas. I have another question regarding the sampling for the data. Since, the data distribution for BiT-L is heavily tailed. Was the distribution converted to uniform or other to pretrain the model on such a large dataset?
As mentioned in issue [https://github.com/google-research/big_transfer/issues/26] the loss is sigmoid binary cross entropy for each label. I have few more questions about the loss: 1) How is the objects present in the image but label being zero accounted or handled?. For example: there is a picture containing : dog, cat but the label is only dog, and not {cat, animal}. 2) Is there negative sampling done from all label classes for negatives or all the other classes are taken as negative? @lucasb-eyer @akolesnikoff @ebursztein @jessicayung @kolesman