search

uber-research / PPLM

Plug and Play Language Model implementation. Allows to steer topic and attributes of GPT-2 models.
Apache License 2.0
1.13k stars 202 forks source link

Inference Time #32

Open yijunzhouzoey opened 3 years ago

yijunzhouzoey commented 3 years ago

Thanks for your brilliant work!

Currently, I am trying the PPLM with a discriminator on GPU but it still needs around 5 mins to generate 512 tokens. I wonder if there is any way to speed up the inference time?

Many thanks and best regards, Yijun

andreamad8 commented 3 years ago

Hi @EstelleZhou,

yes, you could try to speed this up by decreasing the number of iteration per token. However, this may lead to a worst result, in term of positivity/negativity, compared to the one reported in the paper.

Andrea