THUDM / CogVLM

a state-of-the-art-level open visual language model | 多模态预训练模型
Apache License 2.0
5.86k stars 400 forks source link

Better set a specific version of transformers lib #517

Open cocoshe opened 1 month ago

cocoshe commented 1 month ago

System Info / 系統信息

GPU

Who can help? / 谁可以帮助到您?

No response

Information / 问题信息

Reproduction / 复现过程

Since 4.42.0, the _extract_past_from_model_output changed the return formats

https://github.com/huggingface/transformers/blob/v4.42.0/src/transformers/generation/utils.py#L633-L650

# transformers >= 4.42.0
    def _extract_past_from_model_output(self, outputs: ModelOutput, standardize_cache_format: bool = False):
        past_key_values = None
        cache_name = "past_key_values"
        if "past_key_values" in outputs:
            past_key_values = outputs.past_key_values
        elif "mems" in outputs:
            past_key_values = outputs.mems
        elif "past_buckets_states" in outputs:
            past_key_values = outputs.past_buckets_states
        elif "cache_params" in outputs:
            past_key_values = outputs.cache_params
            cache_name = "cache_params"

        # Bloom fix: standardizes the cache format when requested
        if standardize_cache_format and hasattr(self, "_convert_to_standard_cache"):
            batch_size = outputs.logits.shape[0]
            past_key_values = self._convert_to_standard_cache(past_key_values, batch_size=batch_size)
        return cache_name, past_key_values

https://github.com/huggingface/transformers/blob/v4.41.2/src/transformers/generation/utils.py#L613-L626

# transformers < 4.42.0
    def _extract_past_from_model_output(self, outputs: ModelOutput, standardize_cache_format: bool = False):
        past_key_values = None
        if "past_key_values" in outputs:
            past_key_values = outputs.past_key_values
        elif "mems" in outputs:
            past_key_values = outputs.mems
        elif "past_buckets_states" in outputs:
            past_key_values = outputs.past_buckets_states

        # Bloom fix: standardizes the cache format when requested
        if standardize_cache_format and hasattr(self, "_convert_to_standard_cache"):
            batch_size = outputs.logits.shape[0]
            past_key_values = self._convert_to_standard_cache(past_key_values, batch_size=batch_size)
        return past_key_values

Better set the specific version of transformers in requirements.txt and highlight in installation readme, or change the modeling code here https://huggingface.co/THUDM/cogagent-chat-hf/blob/d519da3b191401234f4bd86ce1c287c61bc276a3/modeling_cogagent.py#L878-L880:

model_kwargs["past_key_values"] = self._extract_past_from_model_output(
    outputs, standardize_cache_format=standardize_cache_format
)

to

_, model_kwargs["past_key_values"] = self._extract_past_from_model_output(
    outputs, standardize_cache_format=standardize_cache_format
)

with the latest version of transformers lib.

Expected behavior / 期待表现

Fix the version bug

JBurtn commented 3 weeks ago

also comment out standardize_cache_format=standardize_cache_format. Later versions remove this parameter and cog only ever references it here AFIAK