search

openai / grok

MIT License
4.06k stars 506 forks source link

questions regarding modulus division #2

Open garyhlai opened 2 years ago

garyhlai commented 2 years ago

Thanks for the very interesting paper! I have two questions regarding modular division:

  1. Does figure 1 of the paper correspond to this equation x◦y=x/y (mod p) for 0≤x<p, 0<y<p where p == 97?
  2. Wouldn't x/y (mod p) produce fractional results? How would you get the cross-entropy loss (against these fractional targets) then?

I tried staring at the code but couldn't really connect the dots :(

Demonstrandum commented 3 months ago

97 is a prime number, and the integers modulo a prime power form a field. Specifically, that means multiplication forms an Abelian group, so division is well defined for all elements (except zero) and closed over the field.

i.e. division of integers modulo 97 will produce another integer (module 97), since every element has a multiplicative inverse.