non-deterministic behavior with batch size > 1

yuwenmichael / Grounding-DINO-Batch-Inference

Support batch inference of Grounding DINO. "Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection"

2 stars 0 forks source link

non-deterministic behavior with batch size > 1 #4

Open FiorenzoParascandolo1 opened 4 months ago

FiorenzoParascandolo1 commented 4 months ago

I noticed this strange behavior: the logits outputs of the model vary slightly with respect to different runs, resulting in non-deterministic behavior. This effect does not happen in case of batch size = 1. Do you know what the reason could be? The difference between the logits (comparing two separate script executions) increases as the batch size increases