Basically, we break up the song into several subsections. In each subsection, we count the number of occurrences of each note, to create a 12-dimensional vector. For example, let's convert Mary Had a Little Lamb to such a vector. The notes are EDCDEEEDDDEGGEDCDEEEEDDEDC. In this case, the vector would be <3, 0, 10, 0, 11, 0, 0, 2, 0, 0, 0, 0>. Then, we dot the vector by a mask corresponding to each key. For C Major the mask could be <0.5, 0.1, 0.5, 0.1, 0.5, 0.5, 0.1, 0.5, 0.1, 0.5, 0.1, 0.5>, while for C# major the mask would be <0.5, 0.5, 0.1, 0.5, 0.1, 0.5, 0.5, 0.1, 0.5, 0.1, 0.5, 0.1>. The dot product is 13, while <C# major> is 3.8. Thus, our song is much more likely to be in the key of C. If we continue for the other keys, we'll also get 13 for F major and G major.
Basically, we break up the song into several subsections. In each subsection, we count the number of occurrences of each note, to create a 12-dimensional vector. For example, let's convert Mary Had a Little Lamb to such a vector. The notes are EDCDEEEDDDEGGEDCDEEEEDDEDC. In this case, the vector would be <3, 0, 10, 0, 11, 0, 0, 2, 0, 0, 0, 0>. Then, we dot the vector by a mask corresponding to each key. For C Major the mask could be <0.5, 0.1, 0.5, 0.1, 0.5, 0.5, 0.1, 0.5, 0.1, 0.5, 0.1, 0.5>, while for C# major the mask would be <0.5, 0.5, 0.1, 0.5, 0.1, 0.5, 0.5, 0.1, 0.5, 0.1, 0.5, 0.1>. The dot product is 13, while <C# major> is 3.8. Thus, our song is much more likely to be in the key of C. If we continue for the other keys, we'll also get 13 for F major and G major.