hyperdimensional-computing / torchhd

Torchhd is a Python library for Hyperdimensional Computing and Vector Symbolic Architectures
https://torchhd.readthedocs.io
MIT License
229 stars 24 forks source link

Fix the nonlinear bug and add mnist example #110

Closed Orienfish closed 1 year ago

Orienfish commented 1 year ago

Add one more step in the nonlinear encoding: normalize the weight, which is similar to the random projection case. To test it, I create a mnist_nonlinear.py in the examples. Without this normalization, the accuracy of running python mnist_nonlinear.py is 10.75%. After adding this normalization, the accuracy becomes 83.93%.

Another example that justify this change is in the random_projection.py example:

    def encode(self, x):
        enc = self.project(x)
        sample_hv = torch.cos(enc + self.bias) * torch.sin(enc)
        return torchhd.hard_quantize(sample_hv)

This encoding function is doing nothing but first a random projection and then passing through the nonlinear encoding. Since in the random projection, the weight has been normalized; thus in the nonlinear encoding, the weight should also be normalized.

Please let me know if you have any questions or concerns.

mikeheddes commented 1 year ago

Congrats on your first contribution to the library! Thank you for your help :)