If the embedding value is different from the GPU or server, a different value comes out from the 8th digit of the prime number

FlagOpen / FlagEmbedding

Retrieval and Retrieval-augmented LLMs

MIT License

6.59k stars 469 forks source link

If the embedding value is different from the GPU or server, a different value comes out from the 8th digit of the prime number #813

Open jhyeom1545 opened 3 months ago

jhyeom1545 commented 3 months ago

I am using flagbedding well. Among them, I found something I don't understand. If the same sentence is inferred from different GPUs on different servers, the value of the embedding vector is different.

example

- Server1: H100 (80GB) : [0.02650870382785797....., ...]
- Server2: A100 (80GB): [0.026508718729019165, ..., ...]

The value changes from the 8th digit of the prime number, so if you know why, I would appreciate it if you could let me know.

staoxiao commented 3 months ago

We also don't know the specific reason. But the difference is very small and it does not influence performance.

jhyeom1545 commented 2 months ago

@staoxiao Thanks for your comment. I don't know exactly why, but the result of calculating the embeddings on different servers using the same GPU model seems to be the same.