yeyupiaoling / PPASR

基于PaddlePaddle实现端到端中文语音识别,从入门到实战,超简单的入门案例,超实用的企业项目。支持当前最流行的DeepSpeech2、Conformer、Squeezeformer模型
Apache License 2.0
797 stars 131 forks source link

标准值问题 #161

Closed yourengod closed 12 months ago

yourengod commented 1 year ago

这个函数: def compute_mean_istd(self, preprocess_configs, manifest_path, num_workers=4, num_samples=5000): """从随机抽样的实例中计算均值和标准值,并写入到文件中

    :param preprocess_configs: 数据预处理配置参数
    :param manifest_path: 数据列表文件路径
    :param num_workers: 计算的线程数量
    :param num_samples: 用于计算均值和标准值的音频数量
    """
    manifest = read_manifest(manifest_path)
    if num_samples < 0 or num_samples > len(manifest):
        sampled_manifest = manifest
    else:
        sampled_manifest = random.sample(manifest, num_samples)
    logger.info('开始抽取{}条数据计算均值和标准值...'.format(len(sampled_manifest)))
    dataset = NormalizerDataset(sampled_manifest, preprocess_configs)
    test_loader = DataLoader(dataset=dataset, batch_size=64, collate_fn=collate_fn, num_workers=num_workers)
    with paddle.no_grad():
        # 求总和
        std, means = None, None
        number = 0
        for std1, means1, number1 in tqdm(test_loader()):
            number += number1
            if means is None:
                means = means1
            else:
                means += means1
            if std is None:
                std = std1
            else:
                std += std1
        # 求总和的均值和标准值
        for i in range(len(means)):
            means[i] /= number
            std[i] = std[i] / number - means[i] * means[i]
            if std[i] < 1.0e-20:
                std[i] = 1.0e-20
            std[i] = math.sqrt(std[i])
    istd = 1.0 / std
    # 写入到文件中
    data = {'mean': means.tolist(),
            'istd': istd.tolist(),
            'feature_method': preprocess_configs.feature_method}
    with open(self.mean_std_filepath, 'w', encoding='utf-8') as f:
        json.dump(data, f)

=============================================================================

求总和的均值和标准值

        for i in range(len(means)):
            means[i] /= number
            std[i] = std[i] / number - means[i] * means[i]
            if std[i] < 1.0e-20:
                std[i] = 1.0e-20
            std[i] = math.sqrt(std[i])
yourengod commented 1 year ago

求总和的均值和标准值

for i in range(len(means)): means[i] /= number std[i] = std[i] / number - means[i] * means[i] if std[i] < 1.0e-20: std[i] = 1.0e-20 std[i] = math.sqrt(std[i]) 这个是求“标准差”的吧,怎么不是标准的形式啊?

yeyupiaoling commented 1 year ago

这种求法是没有问题的。