PaddlePaddle / PaddleSlim

PaddleSlim is an open-source library for deep model compression and architecture search.
https://paddleslim.readthedocs.io/zh_CN/latest/
Apache License 2.0
1.56k stars 345 forks source link

我使用自定义数据集进行静态离线量化的时候出现了问题 #1813

Open sranqiu opened 9 months ago

sranqiu commented 9 months ago

Bug: raise TypeError( TypeError: batch data con only contains: tensor, numpy.ndarray, dict, list, number, but got <class 'paddle.fluid.framework.Variable'

版本信息: paddlepaddle-gpu 2.5.2 paddleslim release/2.5(python stepup.py install 安装)

静态离线量化代码:

import argparse
import paddle
from paddle.io import DataLoader
from paddleslim.quant import quant_post_static
from model.dataset import VimeoDataset

paddle.enable_static()
paddle.set_device("gpu")

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument('--batch_size', default=16, type=int, help='minibatch size')
    parser.add_argument('--data_root', type=str, required=True, help='Path to the dataset')
    args = parser.parse_args()

    img0 = paddle.static.data(name="img0", shape=[None, 3, None, None], dtype="float32")
    img1 = paddle.static.data(name="img1", shape=[None, 3, None, None], dtype="float32")
    timestep = paddle.static.data(name="timestep", shape=[None, 1, None, None], dtype="float32")

    val_dataset = VimeoDataset(args.data_root, "val")
    val_dataloader = DataLoader(val_dataset, feed_list=[img0, img1, timestep], return_list=False, batch_size=args.batch_size, shuffle=False, drop_last=False)

    exe = paddle.paddle.static.Executor(paddle.CUDAPlace(0))
    quant_post_static(
        executor=exe,
        model_dir="./static_eval_model",
        quantize_model_path="./quant_post_static_model",
        data_loader=val_dataloader,
        model_filename="flownet.pdmodel",
        params_filename="flownet.pdiparams",
        batch_size=args.batch_size,
        algo="KL",
    )
ZhangHandi commented 7 months ago

batch data类型不支持paddle.fluid.framework.Variable,可以先检查下是不是dataset的问题