search

rpryzant / delete_retrieve_generate

PyTorch implementation of the Delete, Retrieve Generate style transfer algorithm
MIT License
132 stars 26 forks source link

Meaning of WPS #14

Closed lzzhaha closed 4 years ago

lzzhaha commented 4 years ago

Hi,

In your code, a variable called 'WPS' is recorded in every logging step.

By looking at the source code, it seems to me that this variable represents number of examples being processed by the model in every logging step.

But I don't understand why you want to log this variable and why its value would decrease as the training proceeds.

Could you give some elaboration about this?

Thanks.

rpryzant commented 4 years ago

Thanks for reaching out! WPS stands for "words per second" but in practice corresponds to examples per second (sorry for the confusion! Cleaned things up in this commit: https://github.com/rpryzant/delete_retrieve_generate/commit/6f9069e6f8f2ee20e620e02cb4e5736535a5a4fd).

It's a measure of system speed (how many example are being processed per second).

lzzhaha commented 4 years ago

Thanks for your explanation.

Do you have any idea why this value would decrease and finally become stable as training proceeds?

Thanks.

rpryzant commented 4 years ago

I don't, sorry. It depends on your operating system and hardware.

My guess would be that the initial variance is due to cache warmup and process scheduling/priority.

But the good news changes in training speed will not affect the system's behavior :)

lzzhaha commented 4 years ago

Got it. Thank you very much.