求int8版本 🌹 int4实测，精度损失有点大，多模态效果不好，int8也许会好很多

[okwinds]

          求int8版本 🌹 int4实测，精度损失有点大，多模态效果不好，int8也许会好很多

Originally posted by @okwinds in https://github.com/THUDM/CogVLM2/issues/2#issuecomment-2159952541

[QwertyJack]

# install bitsandbytes: pip install bitsandbytes

...
# load model
model = AutoModelForCausalLM.from_pretrained(
        MODEL_PATH,
        trust_remote_code=True,
        torch_dtype=torch_type,
        device_map='auto',
        load_in_8bit=True,
)
...

[zRzRzRzRzRzRzR]

可以使用这个办法记载，我们暂时没有单独推出int8的模型

[okwinds]

# install bitsandbytes: pip install bitsandbytes

...
# load model
model = AutoModelForCausalLM.from_pretrained(
        MODEL_PATH,
        trust_remote_code=True,
        torch_dtype=torch_type,
        device_map='auto',
        load_in_8bit=True,
)
...

谢谢你的回复，bitsandbytes方式的量化，跑起来会比较慢，之前已经跑过了，所以还是求官方的int8.

[QwertyJack]

官方的 4bit 也是 bitsandbytes 方式量化的：https://huggingface.co/THUDM/cogvlm2-llama3-chinese-chat-19B-int4/blob/main/config.json#L37 合理推测，官方的 int8 跟 load_in_8bit 方式没有本质区别。

[okwinds]

官方的 4bit 也是 bitsandbytes 方式量化的：https://huggingface.co/THUDM/cogvlm2-llama3-chinese-chat-19B-int4/blob/main/config.json#L37

合理推测，官方的 int8 跟 load_in_8bit 方式没有本质区别。

4bit量化，对于多模态而言，损失太高了，顶多8bit可用，4bit相比fp16太拉跨了

[QwertyJack]

我晕。。。逻辑是这样的：

1. 常见的量化有 4bit 和 8bit
2. 官方目前提供了 4bit 量化，没有提供 8bit 量化
3. 官方没有提供 8bit 量化，但是你想用，怎么办？
4. 参考官方提供的 4bit 量化方式，即 bitsandbytes 量化，那么，你也可以使用 bitsandbytes 方法，自己制作一个 8bit 量化
5. 自己制作的 bitsandbytes-8bit 量化跑起来比较慢怎么办？没办法，官方也是这么做的

以上。

[okwinds]

我晕。。。逻辑是这样的：

1. 常见的量化有 4bit 和 8bit
2. 官方目前提供了 4bit 量化，没有提供 8bit 量化
3. 官方没有提供 8bit 量化，但是你想用，怎么办？
4. 参考官方提供的 4bit 量化方式，即 bitsandbytes 量化，那么，你也可以使用 bitsandbytes 方法，自己制作一个 8bit 量化
5. 自己制作的 bitsandbytes-8bit 量化跑起来比较慢怎么办？没办法，官方也是这么做的

以上。

谢谢，你的第一个回复，我已经看明白了，你的意思是，官方int4是这样的：

  "quantization_config": {
    "_load_in_4bit": true,
    "_load_in_8bit": false,
    "bnb_4bit_compute_dtype": "float32",
    "bnb_4bit_quant_storage": "uint8",
    "bnb_4bit_quant_type": "fp4",
    "bnb_4bit_use_double_quant": false,
    "llm_int8_enable_fp32_cpu_offload": false,
    "llm_int8_has_fp16_weight": false,
    "llm_int8_skip_modules": null,
    "llm_int8_threshold": 6.0,
    "load_in_4bit": true,
    "load_in_8bit": false,
    "quant_method": "bitsandbytes"
  },

官方int4离线量化后的权重，打出来是这样的：

参数: model.embed_tokens.weight, 数据类型: torch.float16
参数: model.layers.0.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.0.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.0.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.0.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.0.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.0.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.0.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.0.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.0.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.0.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.0.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.0.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.0.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.1.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.1.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.1.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.1.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.1.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.1.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.1.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.1.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.1.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.1.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.1.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.1.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.1.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.2.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.2.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.2.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.2.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.2.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.2.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.2.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.2.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.2.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.2.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.2.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.2.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.2.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.3.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.3.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.3.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.3.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.3.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.3.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.3.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.3.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.3.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.3.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.3.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.3.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.3.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.4.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.4.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.4.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.4.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.4.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.4.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.4.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.4.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.4.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.4.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.4.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.4.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.4.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.5.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.5.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.5.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.5.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.5.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.5.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.5.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.5.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.5.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.5.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.5.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.5.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.5.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.6.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.6.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.6.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.6.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.6.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.6.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.6.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.6.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.6.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.6.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.6.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.6.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.6.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.7.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.7.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.7.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.7.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.7.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.7.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.7.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.7.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.7.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.7.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.7.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.7.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.7.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.8.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.8.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.8.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.8.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.8.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.8.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.8.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.8.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.8.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.8.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.8.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.8.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.8.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.9.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.9.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.9.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.9.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.9.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.9.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.9.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.9.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.9.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.9.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.9.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.9.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.9.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.10.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.10.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.10.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.10.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.10.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.10.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.10.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.10.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.10.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.10.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.10.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.10.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.10.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.11.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.11.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.11.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.11.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.11.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.11.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.11.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.11.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.11.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.11.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.11.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.11.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.11.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.12.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.12.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.12.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.12.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.12.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.12.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.12.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.12.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.12.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.12.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.12.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.12.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.12.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.13.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.13.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.13.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.13.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.13.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.13.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.13.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.13.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.13.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.13.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.13.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.13.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.13.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.14.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.14.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.14.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.14.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.14.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.14.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.14.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.14.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.14.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.14.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.14.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.14.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.14.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.15.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.15.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.15.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.15.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.15.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.15.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.15.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.15.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.15.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.15.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.15.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.15.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.15.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.16.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.16.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.16.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.16.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.16.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.16.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.16.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.16.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.16.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.16.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.16.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.16.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.16.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.17.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.17.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.17.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.17.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.17.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.17.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.17.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.17.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.17.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.17.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.17.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.17.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.17.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.18.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.18.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.18.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.18.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.18.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.18.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.18.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.18.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.18.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.18.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.18.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.18.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.18.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.19.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.19.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.19.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.19.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.19.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.19.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.19.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.19.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.19.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.19.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.19.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.19.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.19.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.20.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.20.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.20.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.20.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.20.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.20.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.20.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.20.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.20.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.20.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.20.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.20.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.20.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.21.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.21.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.21.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.21.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.21.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.21.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.21.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.21.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.21.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.21.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.21.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.21.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.21.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.22.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.22.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.22.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.22.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.22.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.22.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.22.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.22.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.22.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.22.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.22.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.22.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.22.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.23.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.23.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.23.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.23.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.23.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.23.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.23.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.23.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.23.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.23.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.23.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.23.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.23.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.24.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.24.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.24.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.24.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.24.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.24.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.24.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.24.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.24.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.24.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.24.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.24.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.24.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.25.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.25.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.25.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.25.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.25.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.25.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.25.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.25.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.25.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.25.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.25.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.25.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.25.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.26.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.26.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.26.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.26.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.26.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.26.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.26.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.26.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.26.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.26.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.26.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.26.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.26.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.27.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.27.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.27.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.27.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.27.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.27.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.27.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.27.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.27.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.27.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.27.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.27.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.27.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.28.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.28.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.28.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.28.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.28.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.28.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.28.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.28.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.28.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.28.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.28.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.28.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.28.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.29.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.29.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.29.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.29.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.29.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.29.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.29.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.29.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.29.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.29.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.29.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.29.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.29.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.30.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.30.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.30.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.30.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.30.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.30.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.30.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.30.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.30.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.30.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.30.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.30.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.30.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.layers.31.self_attn.vision_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.31.self_attn.vision_expert_query_key_value.bias, 数据类型: torch.float16
参数: model.layers.31.self_attn.vision_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.31.self_attn.language_expert_query_key_value.weight, 数据类型: torch.uint8
参数: model.layers.31.self_attn.language_expert_dense.weight, 数据类型: torch.uint8
参数: model.layers.31.mlp.language_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.31.mlp.language_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.31.mlp.language_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.31.mlp.vision_mlp.gate_proj.weight, 数据类型: torch.uint8
参数: model.layers.31.mlp.vision_mlp.up_proj.weight, 数据类型: torch.uint8
参数: model.layers.31.mlp.vision_mlp.down_proj.weight, 数据类型: torch.uint8
参数: model.layers.31.input_layernorm.weight, 数据类型: torch.float16
参数: model.layers.31.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.norm.weight, 数据类型: torch.float16
参数: model.vision.boi, 数据类型: torch.float16
参数: model.vision.eoi, 数据类型: torch.float16
参数: model.vision.patch_embedding.cls_embedding, 数据类型: torch.float16
参数: model.vision.patch_embedding.proj.weight, 数据类型: torch.float16
参数: model.vision.patch_embedding.proj.bias, 数据类型: torch.float16
参数: model.vision.patch_embedding.position_embedding.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.0.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.0.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.0.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.0.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.0.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.0.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.0.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.0.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.0.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.0.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.0.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.0.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.1.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.1.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.1.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.1.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.1.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.1.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.1.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.1.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.1.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.1.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.1.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.1.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.2.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.2.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.2.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.2.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.2.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.2.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.2.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.2.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.2.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.2.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.2.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.2.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.3.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.3.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.3.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.3.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.3.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.3.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.3.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.3.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.3.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.3.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.3.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.3.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.4.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.4.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.4.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.4.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.4.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.4.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.4.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.4.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.4.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.4.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.4.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.4.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.5.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.5.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.5.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.5.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.5.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.5.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.5.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.5.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.5.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.5.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.5.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.5.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.6.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.6.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.6.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.6.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.6.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.6.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.6.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.6.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.6.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.6.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.6.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.6.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.7.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.7.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.7.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.7.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.7.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.7.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.7.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.7.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.7.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.7.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.7.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.7.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.8.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.8.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.8.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.8.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.8.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.8.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.8.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.8.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.8.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.8.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.8.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.8.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.9.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.9.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.9.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.9.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.9.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.9.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.9.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.9.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.9.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.9.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.9.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.9.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.10.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.10.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.10.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.10.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.10.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.10.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.10.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.10.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.10.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.10.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.10.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.10.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.11.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.11.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.11.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.11.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.11.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.11.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.11.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.11.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.11.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.11.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.11.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.11.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.12.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.12.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.12.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.12.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.12.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.12.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.12.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.12.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.12.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.12.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.12.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.12.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.13.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.13.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.13.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.13.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.13.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.13.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.13.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.13.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.13.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.13.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.13.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.13.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.14.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.14.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.14.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.14.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.14.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.14.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.14.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.14.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.14.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.14.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.14.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.14.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.15.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.15.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.15.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.15.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.15.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.15.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.15.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.15.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.15.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.15.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.15.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.15.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.16.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.16.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.16.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.16.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.16.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.16.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.16.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.16.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.16.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.16.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.16.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.16.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.17.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.17.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.17.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.17.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.17.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.17.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.17.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.17.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.17.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.17.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.17.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.17.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.18.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.18.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.18.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.18.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.18.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.18.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.18.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.18.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.18.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.18.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.18.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.18.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.19.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.19.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.19.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.19.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.19.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.19.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.19.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.19.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.19.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.19.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.19.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.19.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.20.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.20.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.20.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.20.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.20.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.20.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.20.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.20.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.20.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.20.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.20.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.20.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.21.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.21.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.21.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.21.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.21.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.21.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.21.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.21.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.21.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.21.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.21.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.21.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.22.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.22.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.22.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.22.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.22.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.22.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.22.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.22.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.22.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.22.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.22.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.22.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.23.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.23.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.23.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.23.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.23.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.23.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.23.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.23.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.23.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.23.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.23.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.23.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.24.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.24.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.24.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.24.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.24.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.24.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.24.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.24.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.24.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.24.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.24.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.24.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.25.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.25.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.25.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.25.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.25.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.25.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.25.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.25.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.25.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.25.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.25.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.25.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.26.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.26.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.26.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.26.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.26.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.26.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.26.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.26.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.26.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.26.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.26.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.26.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.27.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.27.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.27.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.27.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.27.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.27.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.27.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.27.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.27.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.27.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.27.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.27.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.28.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.28.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.28.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.28.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.28.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.28.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.28.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.28.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.28.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.28.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.28.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.28.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.29.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.29.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.29.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.29.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.29.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.29.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.29.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.29.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.29.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.29.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.29.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.29.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.30.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.30.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.30.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.30.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.30.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.30.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.30.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.30.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.30.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.30.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.30.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.30.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.31.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.31.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.31.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.31.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.31.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.31.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.31.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.31.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.31.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.31.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.31.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.31.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.32.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.32.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.32.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.32.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.32.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.32.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.32.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.32.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.32.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.32.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.32.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.32.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.33.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.33.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.33.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.33.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.33.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.33.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.33.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.33.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.33.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.33.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.33.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.33.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.34.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.34.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.34.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.34.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.34.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.34.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.34.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.34.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.34.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.34.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.34.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.34.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.35.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.35.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.35.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.35.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.35.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.35.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.35.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.35.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.35.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.35.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.35.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.35.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.36.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.36.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.36.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.36.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.36.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.36.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.36.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.36.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.36.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.36.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.36.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.36.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.37.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.37.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.37.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.37.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.37.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.37.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.37.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.37.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.37.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.37.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.37.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.37.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.38.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.38.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.38.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.38.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.38.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.38.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.38.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.38.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.38.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.38.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.38.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.38.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.39.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.39.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.39.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.39.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.39.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.39.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.39.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.39.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.39.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.39.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.39.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.39.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.40.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.40.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.40.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.40.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.40.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.40.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.40.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.40.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.40.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.40.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.40.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.40.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.41.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.41.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.41.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.41.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.41.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.41.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.41.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.41.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.41.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.41.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.41.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.41.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.42.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.42.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.42.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.42.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.42.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.42.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.42.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.42.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.42.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.42.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.42.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.42.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.43.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.43.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.43.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.43.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.43.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.43.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.43.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.43.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.43.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.43.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.43.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.43.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.44.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.44.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.44.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.44.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.44.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.44.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.44.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.44.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.44.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.44.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.44.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.44.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.45.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.45.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.45.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.45.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.45.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.45.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.45.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.45.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.45.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.45.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.45.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.45.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.46.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.46.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.46.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.46.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.46.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.46.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.46.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.46.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.46.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.46.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.46.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.46.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.47.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.47.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.47.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.47.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.47.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.47.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.47.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.47.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.47.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.47.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.47.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.47.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.48.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.48.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.48.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.48.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.48.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.48.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.48.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.48.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.48.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.48.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.48.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.48.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.49.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.49.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.49.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.49.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.49.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.49.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.49.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.49.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.49.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.49.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.49.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.49.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.50.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.50.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.50.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.50.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.50.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.50.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.50.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.50.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.50.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.50.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.50.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.50.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.51.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.51.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.51.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.51.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.51.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.51.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.51.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.51.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.51.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.51.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.51.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.51.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.52.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.52.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.52.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.52.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.52.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.52.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.52.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.52.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.52.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.52.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.52.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.52.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.53.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.53.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.53.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.53.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.53.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.53.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.53.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.53.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.53.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.53.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.53.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.53.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.54.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.54.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.54.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.54.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.54.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.54.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.54.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.54.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.54.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.54.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.54.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.54.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.55.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.55.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.55.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.55.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.55.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.55.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.55.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.55.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.55.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.55.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.55.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.55.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.56.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.56.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.56.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.56.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.56.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.56.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.56.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.56.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.56.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.56.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.56.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.56.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.57.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.57.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.57.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.57.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.57.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.57.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.57.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.57.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.57.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.57.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.57.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.57.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.58.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.58.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.58.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.58.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.58.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.58.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.58.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.58.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.58.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.58.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.58.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.58.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.59.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.59.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.59.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.59.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.59.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.59.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.59.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.59.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.59.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.59.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.59.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.59.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.60.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.60.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.60.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.60.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.60.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.60.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.60.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.60.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.60.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.60.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.60.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.60.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.61.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.61.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.61.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.61.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.61.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.61.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.61.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.61.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.61.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.61.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.61.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.61.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.62.input_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.62.input_layernorm.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.62.attention.query_key_value.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.62.attention.query_key_value.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.62.attention.dense.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.62.attention.dense.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.62.mlp.fc1.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.62.mlp.fc1.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.62.mlp.fc2.weight, 数据类型: torch.uint8
参数: model.vision.transformer.layers.62.mlp.fc2.bias, 数据类型: torch.float16
参数: model.vision.transformer.layers.62.post_attention_layernorm.weight, 数据类型: torch.float16
参数: model.vision.transformer.layers.62.post_attention_layernorm.bias, 数据类型: torch.float16
参数: model.vision.linear_proj.linear_proj.weight, 数据类型: torch.uint8
参数: model.vision.linear_proj.norm1.weight, 数据类型: torch.float16
参数: model.vision.linear_proj.norm1.bias, 数据类型: torch.float16
参数: model.vision.linear_proj.dense_h_to_4h.weight, 数据类型: torch.uint8
参数: model.vision.linear_proj.gate_proj.weight, 数据类型: torch.uint8
参数: model.vision.linear_proj.dense_4h_to_h.weight, 数据类型: torch.uint8
参数: model.vision.conv.weight, 数据类型: torch.float16
参数: model.vision.conv.bias, 数据类型: torch.float16
参数: lm_head.weight, 数据类型: torch.float16

以上官方int4实测精度够不上业务。

你写的那一段无法做到离线量化8bit哈，你可以测一下。 我做的离线量化，只是权重存储体积下来一些，但是推理效率不行。

所以，我遇到问题，想请官方帮忙，但是官方目前不打算支持int8，那么，我就找别的方式。

谢谢你~

[QwertyJack]

怎么理解“无法做到离线量化8bit”？

# load model
tokenizer = ...
model = AutoModelForCausalLM.from_pretrained(..., load_in_8bit=True)

# save model
model.save_pretrained(quant8_saved_dir)
tokenizer.save_pretrained(quant8_saved_dir)

[okwinds]

怎么理解“无法做到离线量化8bit”？

# load model
tokenizer = ...
model = AutoModelForCausalLM.from_pretrained(..., load_in_8bit=True)

# save model
model.save_pretrained(quant8_saved_dir)
tokenizer.save_pretrained(quant8_saved_dir)

你执行一下，对比一下存储空间，就知道

[QwertyJack]

你执行一下，对比一下存储空间，就知道

1. 社区没有义务帮你 debug，何况并环境、配置、版本、代码等必要的信息并没有给出；
2. 如果你遇到了问题，可以把问题贴出来，如果不知道怎么提问，建议先学习如何提问。

[okwinds]

你执行一下，对比一下存储空间，就知道

1. 社区没有义务帮你 debug，何况并环境、配置、版本、代码等必要的信息并没有给出；
2. 如果你遇到了问题，可以把问题贴出来，如果不知道怎么提问，建议先学习如何提问。

不好意思，你的这段回复让我有点莫名。

我曾在上面的回复中提到过，如下：

“我遇到问题，想请官方帮忙，但是官方目前不打算支持int8，那么，我就找别的方式。

谢谢你~”

这段话的意思很明确： 1、官方没有支持，我就去想其他办法 2、我对你的热心也表达了谢意。但是这个方法它不work，所以我继续寻找我的办法

以上，我好像没有对社区有什么其他非份的想法或要求吧？你这句话从何而来。

是什么让你心碎了。我没理解。😂

都挺忙的，就不回复了。再次感谢~

[QwertyJack]

你执行一下，对比一下存储空间，就知道

你试过了吗？有什么问题吗？