search

lucidrains / vector-quantize-pytorch

Vector (and Scalar) Quantization, in Pytorch
MIT License
2.12k stars 179 forks source link

Add note to maybe experiment with temperature #98

Closed matwilso closed 5 months ago

matwilso commented 6 months ago

Feel free to reject if you think 100.0 is a better default, but for my own problem I found much better results for inv temperature of 10 or even 1.0

lucidrains commented 6 months ago

ah interesting

i'm guessing you are using LFQ for some reason for tesla autopilot?

lucidrains commented 6 months ago

so the author of the technique actually set it to 100, so i'm wary of changing it

your experience is worth something though, and maybe it can be put in the readme somewhere

matwilso commented 6 months ago

ah interesting

i'm guessing you are using LFQ for some reason for tesla autopilot?

Haha potentially. Not at the moment, just kind of messing with it for now, studying some recent video generative models that look interesting (magvit-v2 / videopoet)

matwilso commented 6 months ago

so the author of the technique actually set it to 100, so i'm wary of changing it

your experience is worth something though, and maybe it can be put in the readme somewhere

Yeah fair enough, I can see about just adding a note so that people know to maybe mess with that parameter.

lucidrains commented 5 months ago

👍