Closed rd4com closed 10 months ago
I implemented the vectorize, as for the benchmarks i have no idea how to do it, i have mojo 0.4.0 right now, any idea ? (keep in mind i don't understand ai at all, just trying to contribute with mojo code and learn)
@rd4com , there are examples of how to benchmark in the matmul notebook in the Mojo documentation. For this change doing a small benchmark that only tests the changed loop code at vocab_size of 32000 iterations would be interesting. Also just running mojo llama2.mojo stories15M.bin
5-10 times from this PR branch and again from master branch and reporting the results is useful information.
@rd4com would you mind to run some benchmarks ? Or better let's focus on Mojo 0.4.0 migration, for this let's not include vectorize
change for now. Feel free to send another PR.
Ok, i updated the repo, do you want me to close this pr and create another one ? please verify the code is running fine before
hope it helps, i also added parallel to:
Apply the temperature to the logits
https://docs.modular.com/mojo/changelog.html#v0.4.0-2023-10-05