Open cocoshe opened 1 month ago
GPU
No response
Since 4.42.0, the _extract_past_from_model_output changed the return formats
_extract_past_from_model_output
https://github.com/huggingface/transformers/blob/v4.42.0/src/transformers/generation/utils.py#L633-L650
# transformers >= 4.42.0 def _extract_past_from_model_output(self, outputs: ModelOutput, standardize_cache_format: bool = False): past_key_values = None cache_name = "past_key_values" if "past_key_values" in outputs: past_key_values = outputs.past_key_values elif "mems" in outputs: past_key_values = outputs.mems elif "past_buckets_states" in outputs: past_key_values = outputs.past_buckets_states elif "cache_params" in outputs: past_key_values = outputs.cache_params cache_name = "cache_params" # Bloom fix: standardizes the cache format when requested if standardize_cache_format and hasattr(self, "_convert_to_standard_cache"): batch_size = outputs.logits.shape[0] past_key_values = self._convert_to_standard_cache(past_key_values, batch_size=batch_size) return cache_name, past_key_values
https://github.com/huggingface/transformers/blob/v4.41.2/src/transformers/generation/utils.py#L613-L626
# transformers < 4.42.0 def _extract_past_from_model_output(self, outputs: ModelOutput, standardize_cache_format: bool = False): past_key_values = None if "past_key_values" in outputs: past_key_values = outputs.past_key_values elif "mems" in outputs: past_key_values = outputs.mems elif "past_buckets_states" in outputs: past_key_values = outputs.past_buckets_states # Bloom fix: standardizes the cache format when requested if standardize_cache_format and hasattr(self, "_convert_to_standard_cache"): batch_size = outputs.logits.shape[0] past_key_values = self._convert_to_standard_cache(past_key_values, batch_size=batch_size) return past_key_values
Better set the specific version of transformers in requirements.txt and highlight in installation readme, or change the modeling code here https://huggingface.co/THUDM/cogagent-chat-hf/blob/d519da3b191401234f4bd86ce1c287c61bc276a3/modeling_cogagent.py#L878-L880:
requirements.txt
model_kwargs["past_key_values"] = self._extract_past_from_model_output( outputs, standardize_cache_format=standardize_cache_format )
to
_, model_kwargs["past_key_values"] = self._extract_past_from_model_output( outputs, standardize_cache_format=standardize_cache_format )
with the latest version of transformers lib.
Fix the version bug
also comment out standardize_cache_format=standardize_cache_format. Later versions remove this parameter and cog only ever references it here AFIAK
System Info / 系統信息
GPU
Who can help? / 谁可以帮助到您?
No response
Information / 问题信息
Reproduction / 复现过程
Since 4.42.0, the
_extract_past_from_model_output
changed the return formatshttps://github.com/huggingface/transformers/blob/v4.42.0/src/transformers/generation/utils.py#L633-L650
https://github.com/huggingface/transformers/blob/v4.41.2/src/transformers/generation/utils.py#L613-L626
Better set the specific version of transformers in
requirements.txt
and highlight in installation readme, or change the modeling code here https://huggingface.co/THUDM/cogagent-chat-hf/blob/d519da3b191401234f4bd86ce1c287c61bc276a3/modeling_cogagent.py#L878-L880:to
with the latest version of transformers lib.
Expected behavior / 期待表现
Fix the version bug