some functions and variables missing in contrastive_generate.py

[zzxzzt333]

Thanks for excellent code, but when I tried to run the demo, I found that some functions and variables in the 'contrastive_generate.py' file were not defined, and the parameters in 'self._get_logits_processor' did not exist. Could you please improve the file?

[lwpyh]

Can you provide a screenshot to explain what specific error you are encountering?

[zzxzzt333]

1、 CONTRASTIVE_SEARCH = "contrastive_search" GREEDY_SEARCH = "greedy_search" SAMPLE = "sample" ASSISTED_GENERATION = "assisted_generation"

Beam methods

BEAM_SEARCH = "beam_search"
BEAM_SAMPLE = "beam_sample"
CONSTRAINED_BEAM_SEARCH = "constrained_beam_search"
GROUP_BEAM_SEARCH = "group_beam_search" （These methods seem to have only greedy_search）

2、self.generation_config._original_object_hash、self._validate_model_kwargsn and self._validate_generated_length does not exist 3、generation_mode = self._get_generation_mode(generation_config, assistant_model) does not exist 4、 logits_processor = self._get_logits_processor( generation_config=generation_config, input_ids_seq_length=input_ids_length, encoder_input_ids=inputs_tensor, prefix_allowed_tokens_fn=prefix_allowed_tokens_fn, logits_processor=logits_processor,

model_kwargs=model_kwargs,

    # negative_prompt_ids=negative_prompt_ids,
    # negative_prompt_attention_mask=negative_prompt_attention_mask,
)

The last three variables do not exist,I have commented out 5、 GreedySearchDecoderOnlyOutput( sequences=input_ids, scores=scores, attentions=decoder_attentions, hidden_states=decoder_hidden_states,

past_key_values=model_kwargs.get("past_key_values"),

        )

The last one variables do not exist,I have commented out

This is all the problem I sorted out. After modifying it, I can run it, but I am not sure if there is any difference with the original result.

[lwpyh]

Actually, it should use "sample" rather than "greedy_search". Thanks for the detailed reply, we will modify the demo soon.

[lwpyh]

I will let you know once it is finished.

[lwpyh]

Hi zzxzzt333,

We reran the demo on different machines, and it still did not produce any errors. The current demo files are ready for use. The contrastive_generate.py file is essentially a reimplementation of the utils.py functions from the transformers library (https://github.com/huggingface/transformers/blob/66bc4def9505fa7c7fe4aa7a248c34a026bb552b/src/transformers/generation/utils.py). We modified the generate function according to the methods described in our paper, turning it into the generate_post function. The functions you mentioned that seem not to exist are actually calls to the default functions from the transformers library. You can print the outputs of these functions to see the specific results. The generation_mode = self._get_generation_mode(generation_config, assistant_model) function will output the mode as "GenerationMode.SAMPLE," which in turn calls the corresponding sub-function to execute. I hope this clarifies your doubts. I suggest running the main file directly to check if the codes functions correctly. Please feel free to reach out to us for any further questions or concerns.