时间轴有问题 - Githubissues

Lixi20 commented 2 months ago

版本

funasr                   1.1.4
modelscope               1.16.1

运行代码（你们官方示例代码）：


from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks

if __name__ == '__main__':
    audio_in = 'https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/asr_speaker_demo.wav'
    output_dir = "./results"
    inference_pipeline = pipeline(
        task=Tasks.auto_speech_recognition,
        model='iic/speech_paraformer-large-vad-punc-spk_asr_nat-zh-cn',
        model_revision='v2.0.4',
        vad_model='iic/speech_fsmn_vad_zh-cn-16k-common-pytorch', vad_model_revision="v2.0.4",
        punc_model='iic/punc_ct-transformer_cn-en-common-vocab471067-large', punc_model_revision="v2.0.4",
        output_dir=output_dir,
    )
    rec_result = inference_pipeline(audio_in, batch_size_s=300, batch_size_token_threshold_s=40)
    print(rec_result)

运行日志(取其中一个)： [{'key': 'asr_speaker_demo', 'text': '非常高兴哈能够和几位的话呢一起来讨论互联网企业如何决胜全球化新高地这个话题。然后第二块其实是游戏平台。所谓游戏平台，它主要是呃简单来说就是一个商店加社区的这样一个模式。而这么多年我们随着整个业务的拓张呢会发现跟阿里云有非常紧密的联系。因为刚开始伟光在介绍的时候也讲阿里云也是阿里巴巴的云。所以这个过程中一会儿也可以稍微展开。跟大家讲一下我们跟云是怎么一路走来的。其实的确的话呢，就对我们互联网公司来说，如果不能够问当地的人口的话，我想我们可能这个整个的就失去了后边所有的这个动力。不知道你们各位怎么看，就是我们最大的这个问题是不是效率优先？Yes, oh no.然后如果是讲一个最关键的，你们是怎么来克服这些挑战的啊？因因因为其我们最近一直在做海外业务，嗯，就是所以说这呃我们碰到了些问题，可以一起分享出来给大家，其实一起探讨一下。嗯嗯，其实海外外就就我我们是这个强观的说是呃，无论你准备工作做的有多充分，嗯，无论你有就是呃学习能力有多强。嗯，你一个中国企业负责人其实在出海的时候，呃，他整体还是一个强试错的过程。嗯，后来退到德国或者拓大，新加坡、印尼、越南等等这些地方，那每一个地方走过去。都面临的一个问题是建站的效率怎么样能够快速的把这个站站能建起来。一方面我们当初刚好从一四年刚好开始要出去的时候呢，去国内就是三个北上广深。那当在海外呢要同时开服北美、美东美西，对吧？欧洲日本。那我还记得那个时候，那我们在海外如何去建立这种IDC的勘探，建设基础设施建设、云服务的部署，那都是一个全新的挑战。', 'timestamp': [[50, 130], [130, 250], [250, 410], [410, 650], [650, 890], [1190, 1430], [1430, 1670], [1870, 2110], [2230, 2430], [2430, 2670], [2690, 2850], [2850, 3090], [3150, 3390], [3810, 3990], [3990, 4230], [4230, 4450], [4450, 4650], [4650, 4890], [5450, 5670], [5670, 5790], [5790, 6030], [6050, 6270], [6270, 6490], [6490, 6690], [6690, 6890], [6890, 7130], [7170, 7410], [7790, 8029], [8070, 8310], [8310, 8550], [8570, 8790], [8790, 8970], [8970, 9170], [9170, 9270], [9270, 9390], [9390, 9570], [9570, 9810], [10290, 10410], [10410, 10530], [10530, 10650], [10650, 10830], [10830, 11070], [11150, 11250], [11250, 11370], [11370, 11530], [11530, 11670], [11670, 11810], [11810, 11910], [11910, 12150], [12790, 13030], [13050, 13290], [13290, 13410], [13410, 13550], [13550, 13650], [13650, 13890], [14010, 14210], [14210, 14370], [14370, 14490], [14490, 14730], [15330, 15570], [15790, 15930], [15930, 16110], [16110, 16290], [16290, 16470], [16470, 16630], [16630, 16830], [16830, 16930], [16930, 17150], [17150, 17290], [17290, 17530], [17530, 17690], [17690, 17890], [17890, 18010], [18010, 18190], [18190, 18290], [18290, 18370], [18370, 18470], [18470, 18550], [18550, 18670], [18670, 18910], [19370, 19590], [19590, 19690], [19690, 19830], [19830, 19990], [19990, 20230], [20250, 20410], [20410, 20550], [20550, 20710], [20710, 20850], [20850, 21010], [21010, 21130], [21130, 21250], [21250, 21330], [21330, 21450], [21450, 21610], [21610, 21790], [21790, 21990], [21990, 22190], [22190, 22330], [22330, 22450], [22450, 22590], [22590, 22690], [22690, 22870], [22870, 23110], [23690, 23930], [24090, 24250], [24250, 24490], [24570, 24730], [24730, 24910], [24910, 25050], [25050, 25210], [25210, 25330], [25330, 25430], [25430, 25670], [25990, 26230], [26270, 26450], [26450, 26690], [26710, 26810], [26810, 26990], [26990, 27090], [27090, 27170], [27170, 27290], [27290, 27410], [27410, 27510], [27510, 27590], [27590, 27670], [27670, 27910], [28230, 28430], [28430, 28550], [28550, 28790], [28790, 28910], [28910, 29030], [29030, 29110], [29110, 29230], [29230, 29330], [29330, 29450], [29450, 29570], [29570, 29770], [29770, 29930], [29930, 30170], [30330, 30470], [30470, 30590], [30590, 30710], [30710, 30830], [30830, 30950], [30950, 31030], [31030, 31130], [31130, 31210], [31210, 31310], [31310, 31390], [31390, 31490], [31490, 31570], [31570, 31750], [31750, 31910], [31910, 32030], [32030, 32170], [32170, 32270], [32270, 32390], [32390, 32509], [32509, 32630], [32630, 32730], [32730, 32810], [32810, 32990], [32990, 33070], [33070, 33270], [33270, 33450], [33450, 33550], [33550, 33710], [33710, 33910], [33910, 34315], [35110, 35270], [35270, 35510], [35510, 35750], [36070, 36210], [36210, 36350], [36350, 36510], [36510, 36710], [36710, 36870], [36870, 37110], [37170, 37290], [37290, 37410], [37410, 37530], [37530, 37610], [37610, 37710], [37710, 37830], [37830, 37910], [37910, 38030], [38030, 38190], [38190, 38370], [38370, 38490], [38490, 38730], [38750, 38850], [38850, 38970], [38970, 39050], [39050, 39130], [39130, 39310], [39310, 39550], [39590, 39730], [39730, 39970], [39970, 40210], [40250, 40450], [40450, 40650], [40650, 40810], [40810, 41050], [41250, 41410], [41410, 41590], [41590, 41670], [41670, 41850], [41850, 42010], [42010, 42250], [42750, 42970], [42970, 43210], [43290, 43510], [43510, 43750], [43750, 43990], [43990, 44230], [44290, 44390], [44390, 44570], [44570, 44710], [44710, 44870], [44870, 45110], [45150, 45290], [45290, 45470], [45470, 45590], [45590, 45670], [45670, 45790], [45790, 45950], [45950, 46130], [46130, 46210], [46210, 46290], [46290, 46470], [46470, 46610], [46610, 46810], [46810, 46970], [46970, 47210], [47270, 47370], [47370, 47490], [47490, 47730], [48190, 48390], [48390, 48630], [48650, 48750], [48750, 48850], [48850, 49050], [49050, 49230], [49230, 49370], [49370, 49470], [49470, 49610], [49610, 49770], [49770, 50010], [50170, 50370], [50370, 50490], [50490, 50730], [50950, 51150], [51150, 51350], [51350, 51510], [51510, 51590], [51590, 51830], [52290, 52850], [52850, 53175], [54000, 54200], [54200, 54440], [54460, 54600], [54600, 54760], [54760, 55145], [56990, 57230], [57290, 57450], [57450, 57590], [57590, 57770], [57770, 57990], [57990, 58210], [58210, 58450], [58550, 58750], [58750, 58870], [58870, 59050], [59050, 59150], [59150, 59270], [59270, 59510], [59530, 59750], [59750, 59990], [60610, 60850], [60870, 61110], [61510, 61750], [61770, 62010], [62070, 62310], [62310, 62635], [64610, 64750], [64750, 64850], [64850, 65090], [65110, 65190], [65190, 65390], [65390, 65470], [65470, 65570], [65570, 65670], [65670, 65850], [65850, 65950], [65950, 66050], [66050, 66210], [66210, 66350], [66350, 66450], [66450, 66590], [66590, 66750], [66750, 66990], [67110, 67330], [67330, 67430], [67430, 67570], [67570, 67670], [67670, 67790], [67790, 68030], [68210, 68450], [68450, 68550], [68550, 68690], [68690, 68790], [68790, 68910], [68910, 68990], [68990, 69070], [69070, 69150], [69150, 69250], [69250, 69410], [69410, 69510], [69510, 69750], [69930, 70110], [70110, 70250], [70250, 70350], [70350, 70530], [70530, 70650], [70650, 70750], [70750, 70890], [70890, 71010], [71010, 71250], [71270, 71430], [71430, 71670], [71690, 71790], [71790, 71970], [71970, 72090], [72090, 72230], [72230, 72330], [72330, 72450], [72450, 72690], [73110, 73350], [73590, 73770], [73770, 73910], [73910, 74130], [74130, 74370], [74990, 75230], [75390, 75510], [75510, 75650], [75650, 75750], [75750, 75870], [75870, 75990], [75990, 76110], [76110, 76190], [76190, 76330], [76330, 76470], [76470, 76710], [76750, 76950], [76950, 77190], [77790, 78030], [78250, 78390], [78390, 78570], [78570, 78810], [79350, 79530], [79530, 79710], [79710, 79830], [79830, 79970], [79970, 80070], [80070, 80190], [80190, 80270], [80270, 80410], [80410, 80550], [80550, 80790], [80910, 81150], [81370, 81510], [81510, 81710], [81710, 81950], [82050, 82290], [82690, 82890], [82890, 83130], [83350, 83590], [83630, 83730], [83730, 83910], [83910, 83990], [83990, 84090], [84090, 84210], [84210, 84290], [84290, 84410], [84410, 84650], [84950, 85110], [85110, 85210], [85210, 85310], [85310, 85410], [85410, 85550], [85550, 85650], [85650, 85770], [85770, 85870], [85870, 85950], [85950, 86130], [86130, 86330], [86330, 86430], [86430, 86550], [86550, 86690], [86690, 86850], [86850, 87030], [87030, 87150], [87150, 87270], [87270, 87510], [88050, 88290], [88350, 88550], [88550, 88650], [88650, 88770], [88770, 88890], [88890, 89010], [89010, 89090], [89090, 89190], [89190, 89370], [89370, 89490], [89490, 89670], [89670, 89810], [89810, 89910], [89910, 90150], [90390, 90630], [90750, 90870], [90870, 91030], [91030, 91170], [91170, 91370], [91370, 91530], [91530, 91770], [91810, 91950], [91950, 92150], [92150, 92310], [92310, 92450], [92450, 92610], [92610, 92790], [92790, 93030], [93110, 93330], [93330, 93530], [93530, 93690], [93690, 93890], [93890, 93990], [93990, 94190], [94190, 94290], [94290, 94430], [94430, 94530], [94530, 94770], [95070, 95310], [95610, 95850], [95850, 95930], [95930, 96030], [96030, 96150], [96150, 96269], [96269, 96390], [96390, 96570], [96570, 96810], [96830, 97070], [97130, 97290], [97290, 97410], [97410, 97550], [97550, 97650], [97650, 97850], [97850, 97950], [97950, 98130], [98130, 98370], [98550, 98730], [98730, 98910], [98910, 99010], [99010, 99150], [99150, 99390], [99490, 99630], [99630, 99750], [99750, 99870], [99870, 99950], [99950, 100070], [100070, 100230], [100230, 100350], [100350, 100430], [100430, 100530], [100530, 100650], [100650, 100750], [100750, 100830], [100830, 101010], [101010, 101130], [101130, 101250], [101250, 101430], [101430, 101570], [101570, 101670], [101670, 101790], [101790, 101970], [101970, 102090], [102090, 102170], [102170, 102270], [102270, 102510], [102790, 102930], [102930, 103050], [103050, 103230], [103230, 103350], [103350, 103470], [103470, 103650], [103650, 103770], [103770, 103910], [103910, 104130], [104130, 104250], [104250, 104430], [104430, 104550], [104550, 104670], [104670, 104790], [104790, 104910], [104910, 105030], [105030, 105270], [105550, 105790], [105990, 106090], [106090, 106250], [106250, 106490], [106490, 106610], [106610, 106730], [106730, 106910], [106910, 107150], [107370, 107570], [107570, 107690], [107690, 108085], [108750, 108970], [108970, 109090], [109090, 109270], [109270, 109430], [109430, 109610], [109610, 109730], [109730, 109870], [109870, 109970], [109970, 110170], [110170, 110330], [110330, 110570], [110770, 110990], [110990, 111230], [111510, 111650], [111650, 111890], [111910, 112070], [112070, 112310], [112450, 112670], [112670, 112850], [112850, 113030], [113030, 113270], [113410, 113610], [113610, 113850], [114190, 114390], [114390, 114490], [114490, 114590], [114590, 114690], [114690, 114810], [114810, 114930], [114930, 115030], [115030, 115110], [115110, 115230], [115230, 115470], [115490, 115590], [115590, 115710], [115710, 115830], [115830, 115950], [115950, 116130], [116130, 116230], [116230, 116350], [116350, 116490], [116490, 116589], [116589, 116730], [116730, 116830], [116830, 116950], [116950, 117370], [117370, 117470], [117470, 117670], [117670, 117910], [118030, 118270], [118270, 118510], [118730, 118870], [118870, 119050], [119050, 119170], [119170, 119290], [119290, 119450], [119450, 119690], [119890, 120130], [120130, 120290], [120290, 120410], [120410, 120510], [120510, 120650], [120650, 120890], [121370, 121570], [121570, 121730], [121730, 121870], [121870, 121970], [121970, 122050], [122050, 122150], [122150, 122270], [122270, 122410], [122410, 122570], [122570, 122805]], 'sentence_info': [{'text': '非常高兴哈能够和几位的话呢一起来讨论互联网企业如何决胜全球化新高地这个话题。', 'start': 9570, 'end': 9810, 'timestamp': [[50, 130], [130, 250], [250, 410], [410, 650], [650, 890], [1190, 1430], [1430, 1670], [1870, 2110], [2230, 2430], [2430, 2670], [2690, 2850], [2850, 3090], [3150, 3390], [3810, 3990], [3990, 4230], [4230, 4450], [4450, 4650], [4650, 4890], [5450, 5670], [5670, 5790], [5790, 6030], [6050, 6270], [6270, 6490], [6490, 6690], [6690, 6890], [6890, 7130], [7170, 7410], [7790, 8029], [8070, 8310], [8310, 8550], [8570, 8790], [8790, 8970], [8970, 9170], [9170, 9270], [9270, 9390], [9390, 9570], [9570, 9810]], 'spk': 0},...]

BUG：

sentence_info 字段里面的 'start': 9570, 'end': 9810 为什么要取 最后一个字 的开始时间和结束时间？你在 sentence_info 字段里面不应该取的是这句话的开始时间和结束时间吗？难道不应该是'start': 50, 'end': 9810吗？ @wenmengzhou @tastelikefeet @wangxingjun778 @Jintao-Huang @Firmament-cyou

slin000111 commented 2 months ago

sentence_info, https://github.com/modelscope/FunASR/blob/main/funasr/auto/auto_model.py#L553

Lixi20 commented 2 months ago

sentence_info, https://github.com/modelscope/FunASR/blob/main/funasr/auto/auto_model.py#L553

Why is this?

slin000111 commented 2 months ago

sentence_info, https://github.com/modelscope/FunASR/blob/main/funasr/auto/auto_model.py#L553

Why is this?

debug时，https://github.com/modelscope/FunASR/blob/main/funasr/auto/auto_model.py#L553 出现了问题中描述的'start': 9570, 'end': 9810，可以在 https://github.com/modelscope/FunASR 开个issue讨论。

Lixi20 commented 2 months ago

sentence_info, https://github.com/modelscope/FunASR/blob/main/funasr/auto/auto_model.py#L553

Why is this?

debug时，https://github.com/modelscope/FunASR/blob/main/funasr/auto/auto_model.py#L553 出现了问题中描述的'start': 9570, 'end': 9810，可以在 https://github.com/modelscope/FunASR 开个issue讨论。

好的，已经开了

github-actions[bot] commented 1 month ago

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

github-actions[bot] commented 1 month ago

This issue was closed because it has been stalled for 5 days with no activity.

modelscope / modelscope

时间轴有问题 #933