Open CQUTWangHong opened 2 years ago
Another question
names = ["f30k_caption_karpathy_test"] when split == "val" or "test" at loading f30k dataset
but, it is necessary to run get_false_Image or get_false_text function during the test phase?
def get_suite(self, index):
result = None
while result is None:
try:
ret = dict()
ret.update(self.get_image(index))
if not self.image_only:
txt = self.get_text(index)
ret.update({"replica": True if txt["cap_index"] > 0 else False})
ret.update(txt)
for i in range(self.draw_false_image):
——> ret.update(self.get_false_image(i))
for i in range(self.draw_false_text):
——> ret.update(self.get_false_text(i))
result = True
except Exception as e:
print(f"Error while read file idx {index} in {self.names[0]} -> {e}")
index = random.randint(0, len(self.index_mapper) - 1)
return ret
thx!
hi, @dandelin
Run the rank loop before each epoch. It takes me a lot of time , so what the program rank loop is doing during fine-tune?
rank loop: 100%|████████████████████████████████████████████████████████████████████████| 250/250 [3:42:38<00:00, 53.43s/it]
thank you!
hello, @dandelin
I have tracked the code at vilt_module.py
from the training_step function -> set_task function
ITM task will enabled only when v>=1
However, no matter what pre training task, all ITM parameters are set to 0.5 in config.py
Is there a problem with my understanding?
thank you!