I'm trying to reproduce evaluate_sbert.py on the hotpotqa dataset on an A100 GPU (AWS ml.p4d.24xlarge instance), using msmarco-distilbert-base-tas-b model.
According to the progress, it seems to be taking about 8 minutes for ~ 10,000 corpus passages, implying it will take about 69 hours for the entire 5,233,329 passages. Is this normal, or am I doing something really wrong? If the latter, could anybody share some expected times, or any tips?
Thanks a lot!
I'm trying to reproduce evaluate_sbert.py on the
hotpotqadataset on an A100 GPU (AWS ml.p4d.24xlarge instance), usingmsmarco-distilbert-base-tas-bmodel. According to the progress, it seems to be taking about 8 minutes for ~ 10,000 corpus passages, implying it will take about 69 hours for the entire 5,233,329 passages. Is this normal, or am I doing something really wrong? If the latter, could anybody share some expected times, or any tips? Thanks a lot!