search

RobinDeSmet / BazelBot

This is a discord bot that generates non-sensical gibberish for entertainment purposes.
MIT License
0 stars 0 forks source link

experiment with quantization #22

Open RobinDeSmet opened 4 months ago

RobinDeSmet commented 4 months ago

Execution time is a big issue with the bazelbot, we should look for ways to reduce the inference step. Quantization might help because it reduces the amount of parameters without throwing away too much power.