search

google / gemma_pytorch

The official PyTorch implementation of Google's Gemma models
https://ai.google.dev/gemma
Apache License 2.0
5.26k stars 503 forks source link

fix(temperature): allow passing 0 or None as the temperature parameter #43

Closed joselpart closed 6 months ago

joselpart commented 7 months ago

Add additional check to allow temperature value to be set to 0 or None. In such case, sampling defaults to greedy search. This PR addresses issue https://github.com/google/gemma_pytorch/issues/42.

pengchongjin commented 6 months ago

Thanks the fix, @joselpart looking good.

Could you please also apply the same fix to model_xla.py?

joselpart commented 6 months ago

@pengchongjin, I made the equivalent changes for the XLA implementation; however, the main changes were made to the run_xla.py script, where the generate method is implemented. Since I assume that script is mostly used for tests, and the temperature parameter is hard-coded there, I guess the fix is not that critical, but it's there now. One thing to bear in mind is that I was not able to test these changes.

pengchongjin commented 6 months ago

Thanks, @joselpart. This looks good.