In this linecode
I saw that the code sets MaskLogitsProcessorCasualLM Init process uses deepcopy(self._policy_model).eval() and during the generate process, GenerationMixinWithRawScores.sampler executes pre-process distribution and Hook calls to the custom LogitsProcessor. I compared next_token_logits_raw from policy_model is indeed different from next_token_logits from mask_model in the same generate pipeline, what is the meaning of doing here? I really want to know?
In this linecode I saw that the code sets
MaskLogitsProcessorCasualLMInit process usesdeepcopy(self._policy_model).eval()and during the generate process,GenerationMixinWithRawScores.samplerexecutes pre-process distribution and Hook calls to the customLogitsProcessor. I comparednext_token_logits_rawfrom policy_model is indeed different fromnext_token_logitsfrom mask_model in the same generate pipeline, what is the meaning of doing here? I really want to know?