search

zh-plus / openlrc

Transcribe and translate voice into LRC file using Whisper and LLMs (GPT, Claude, et,al). 使用whisper和LLM(GPT，Claude等)来转录、翻译你的音频为字幕文件。
https://zh-plus.github.io/openlrc/
MIT License
449 stars 32 forks source link

可以正常转译，但不能翻译，翻译功能无法正常运作 #24

Closed infinitebook closed 7 months ago

infinitebook commented 10 months ago

这个preprocessed.lrc打开还是英文的，open ai的api也没有使用记录

[2023-12-28 20:23:59] INFO     [Producer_0] Start Sentence Segmentation
 [2023-12-28 20:24:00] INFO     [Producer_0] Sentence Segmentation Elapsed: 0.35s
 [2023-12-28 20:24:00] INFO     [Producer_0] Detected language: en
 [2023-12-28 20:24:00] INFO     [Producer_0] Transcription process Elapsed: 11.74s
 [2023-12-28 20:24:00] INFO     [Producer_0] File saved to D:\CloudMusic\电台节目\preprocessed\阿坑是个坑 - 四六级长难句精听磨耳朵 19_preprocessed_transcribed.json
 [2023-12-28 20:24:00] INFO     [Producer_0] Transcription producer finished.
 [2023-12-28 20:24:00] INFO     [ThreadPoolExecutor-1_0] Got transcription: D:\CloudMusic\电台节目\preprocessed\阿坑是个坑 - 四六级长难句精听磨耳朵 19_preprocessed_transcribed.json
 [2023-12-28 20:24:00] INFO     [ThreadPoolExecutor-1_0] Optimized json file saved to D:\CloudMusic\电台节目\preprocessed\阿坑是个坑 - 四六级长难句精听磨耳朵 19_preprocessed_transcribed_optimized.json
 [2023-12-28 20:24:00] INFO     [ThreadPoolExecutor-1_0] Start Translation process
 [2023-12-28 20:24:00] INFO     [ThreadPoolExecutor-1_0] Translating 阿坑是个坑 - 四六级长难句精听磨耳朵 19_preprocessed: 1 chunks, 8 lines in total.
 [2023-12-28 20:24:00] INFO     [ThreadPoolExecutor-1_0] Translation process Elapsed: 0.57s
 [2023-12-28 20:24:00] INFO     [ThreadPoolExecutor-1_0] File saved to D:\CloudMusic\电台节目\preprocessed\阿坑是个坑 - 四六级长难句精听磨耳朵 19_preprocessed.lrc
 [2023-12-28 20:24:00] INFO     [ThreadPoolExecutor-1_0] Translation fee til now: 0.0000 USD
 [2023-12-28 20:24:00] INFO     [Consumer_0] Transcription consumer finished.
 [2023-12-28 20:24:00] INFO     [MainThread] Transcription (Producer) and Translation (Consumer) process Elapsed: 12.33s
Traceback (most recent call last):
  File "D:\pythonProject\open use1\1.py", line 9, in <module>
    lrcer.run("D:\CloudMusic\电台节目\阿坑是个坑 - 四六级长难句精听磨耳朵 19.mp3",
  File "D:\conda\envs\openlrc\Lib\site-packages\openlrc\openlrc.py", line 241, in run
    raise self.exception
  File "D:\conda\envs\openlrc\Lib\site-packages\openlrc\openlrc.py", line 134, in consumer_worker
    final_subtitle = self._translate(audio_name, prompter, target_lang, transcribed_opt_sub,
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\conda\envs\openlrc\Lib\site-packages\openlrc\openlrc.py", line 159, in _translate
    target_texts = translator.translate(
                   ^^^^^^^^^^^^^^^^^^^^^
  File "D:\conda\envs\openlrc\Lib\site-packages\openlrc\translate.py", line 133, in translate
    response = translate_bot.message(messages_list, output_checker=prompter.check_format)[0]
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\conda\envs\openlrc\Lib\site-packages\openlrc\chatbot.py", line 144, in message
    token_numbers = [get_messages_token_number(message) for message in messages_list]
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\conda\envs\openlrc\Lib\site-packages\openlrc\chatbot.py", line 144, in <listcomp>
    token_numbers = [get_messages_token_number(message) for message in messages_list]
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\conda\envs\openlrc\Lib\site-packages\openlrc\utils.py", line 108, in get_messages_token_number
    total = sum([get_text_token_number(element['content'], model=model) for element in messages])
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\conda\envs\openlrc\Lib\site-packages\openlrc\utils.py", line 108, in <listcomp>
    total = sum([get_text_token_number(element['content'], model=model) for element in messages])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\conda\envs\openlrc\Lib\site-packages\openlrc\utils.py", line 102, in get_text_token_number
    tokens = tiktoken.encoding_for_model(model).encode(text)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\conda\envs\openlrc\Lib\site-packages\tiktoken\model.py", line 75, in encoding_for_model
    return get_encoding(encoding_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\conda\envs\openlrc\Lib\site-packages\tiktoken\registry.py", line 63, in get_encoding
    enc = Encoding(**constructor())
                     ^^^^^^^^^^^^^
  File "D:\conda\envs\openlrc\Lib\site-packages\tiktoken_ext\openai_public.py", line 64, in cl100k_base
    mergeable_ranks = load_tiktoken_bpe(
                      ^^^^^^^^^^^^^^^^^^
  File "D:\conda\envs\openlrc\Lib\site-packages\tiktoken\load.py", line 115, in load_tiktoken_bpe
    return {
           ^
  File "D:\conda\envs\openlrc\Lib\site-packages\tiktoken\load.py", line 117, in <dictcomp>
    for token, rank in (line.split() for line in contents.splitlines() if line)
        ^^^^^^^^^^^
ValueError: too many values to unpack (expected 2)
zh-plus commented 10 months ago

可能是tiktoken的版本的原因，可以看一下你的pip环境吗？

把pip list的输出结果贴上来就行。

infinitebook commented 10 months ago

可能是tiktoken的版本的原因，可以看一下你的pip环境吗？

把pip list的输出结果贴上来就行。

新年快乐~ 我重装了一个conda环境，依旧报错 环境如下：

(open_lrc_test2) C:\Users\infin>conda list
packages in environment at D:\conda\envs\open_lrc_test2:

 Name                    Version                   Build  Channel
aiohttp                   3.9.1                    pypi_0    pypi
aiosignal                 1.3.1                    pypi_0    pypi
annotated-types           0.6.0                    pypi_0    pypi
antlr4-python3-runtime    4.9.3                    pypi_0    pypi
anyio                     4.2.0                    pypi_0    pypi
appdirs                   1.4.4                    pypi_0    pypi
attrs                     23.1.0                   pypi_0    pypi
audioread                 3.0.1                    pypi_0    pypi
av                        10.0.0                   pypi_0    pypi
blas                      1.0                         mkl
blis                      0.7.11                   pypi_0    pypi
brotli-python             1.0.9           py311hd77b12b_7
bzip2                     1.0.8                he774522_0
ca-certificates           2023.12.12           haa95532_0
catalogue                 2.0.10                   pypi_0    pypi
certifi                   2023.11.17      py311haa95532_0
cffi                      1.16.0          py311h2bbff1b_0
charset-normalizer        2.0.4              pyhd3eb1b0_0
click                     8.1.7                    pypi_0    pypi
cloudpathlib              0.16.0                   pypi_0    pypi
colorama                  0.4.6                    pypi_0    pypi
coloredlogs               15.0.1                   pypi_0    pypi
colorlog                  6.8.0                    pypi_0    pypi
confection                0.1.4                    pypi_0    pypi
cryptography              41.0.7          py311h89fc84f_0
ctranslate2               3.23.0                   pypi_0    pypi
cuda-cccl                 12.3.101                      0    nvidia
cuda-cudart               11.8.89                       0    nvidia
cuda-cudart-dev           11.8.89                       0    nvidia
cuda-cupti                11.8.87                       0    nvidia
cuda-libraries            11.8.0                        0    nvidia
cuda-libraries-dev        11.8.0                        0    nvidia
cuda-nvrtc                11.8.89                       0    nvidia
cuda-nvrtc-dev            11.8.89                       0    nvidia
cuda-nvtx                 11.8.86                       0    nvidia
cuda-profiler-api         12.3.101                      0    nvidia
cuda-runtime              11.8.0                        0    nvidia
cymem                     2.0.8                    pypi_0    pypi
deepfilterlib             0.5.6                    pypi_0    pypi
deepfilternet             0.5.6                    pypi_0    pypi
distro                    1.9.0                    pypi_0    pypi
en-core-web-sm            3.7.1                    pypi_0    pypi
faster-whisper            0.10.0                   pypi_0    pypi
ffmpeg-normalize          1.27.7                   pypi_0    pypi
ffmpeg-progress-yield     0.7.8                    pypi_0    pypi
ffmpeg-python             0.2.0                    pypi_0    pypi
filelock                  3.13.1          py311haa95532_0
filetype                  1.2.0                    pypi_0    pypi
flatbuffers               23.5.26                  pypi_0    pypi
freetype                  2.12.1               ha860e81_0
frozenlist                1.4.1                    pypi_0    pypi
fsspec                    2023.12.2                pypi_0    pypi
future                    0.18.3                   pypi_0    pypi
giflib                    5.2.1                h8cc25b3_3
gmpy2                     2.1.2           py311h7f96b67_0
h11                       0.14.0                   pypi_0    pypi
httpcore                  1.0.2                    pypi_0    pypi
httpx                     0.26.0                   pypi_0    pypi
huggingface-hub           0.20.1                   pypi_0    pypi
humanfriendly             10.0                     pypi_0    pypi
idna                      3.4             py311haa95532_0
intel-openmp              2023.1.0         h59b6b97_46320
jaconvv2                  0.4                      pypi_0    pypi
jinja2                    3.1.2           py311haa95532_0
jpeg                      9e                   h2bbff1b_1
langcodes                 3.3.0                    pypi_0    pypi
language-data             1.1                      pypi_0    pypi
lerc                      3.0                  hd77b12b_0
libcublas                 11.11.3.6                     0    nvidia
libcublas-dev             11.11.3.6                     0    nvidia
libcufft                  10.9.0.58                     0    nvidia
libcufft-dev              10.9.0.58                     0    nvidia
libcurand                 10.3.4.101                    0    nvidia
libcurand-dev             10.3.4.101                    0    nvidia
libcusolver               11.4.1.48                     0    nvidia
libcusolver-dev           11.4.1.48                     0    nvidia
libcusparse               11.7.5.86                     0    nvidia
libcusparse-dev           11.7.5.86                     0    nvidia
libdeflate                1.17                 h2bbff1b_1
libffi                    3.4.4                hd77b12b_0
libjpeg-turbo             2.0.0                h196d8e1_0
libnpp                    11.8.0.86                     0    nvidia
libnpp-dev                11.8.0.86                     0    nvidia
libnvjpeg                 11.9.0.86                     0    nvidia
libnvjpeg-dev             11.9.0.86                     0    nvidia
libpng                    1.6.39               h8cc25b3_0
libtiff                   4.5.1                hd77b12b_0
libuv                     1.44.2               h2bbff1b_0
libwebp                   1.3.2                hbc33d0d_0
libwebp-base              1.3.2                h2bbff1b_0
lingua-language-detector  1.3.4                    pypi_0    pypi
loguru                    0.7.2                    pypi_0    pypi
lz4-c                     1.9.4                h2bbff1b_0
marisa-trie               0.7.8                    pypi_0    pypi
markupsafe                2.1.1           py311h2bbff1b_0
mkl                       2023.1.0         h6b88ed4_46358
mkl-service               2.4.0           py311h2bbff1b_1
mkl_fft                   1.3.8           py311h2bbff1b_0
mkl_random                1.2.4           py311h59b6b97_0
mpc                       1.1.0                h7edee0f_1
mpfr                      4.0.2                h62dcd97_1
mpir                      3.0.0                hec2e145_1
mpmath                    1.3.0           py311haa95532_0
multidict                 6.0.4                    pypi_0    pypi
murmurhash                1.0.10                   pypi_0    pypi
networkx                  3.1             py311haa95532_0
numpy                     1.26.2          py311hdab7c0b_0
numpy-base                1.26.2          py311hd01c5d8_0
omegaconf                 2.3.0                    pypi_0    pypi
onnxruntime               1.16.3                   pypi_0    pypi
openai                    1.6.1                    pypi_0    pypi
openjpeg                  2.4.0                h4fc8c34_0
openlrc                   1.0.1                    pypi_0    pypi
openssl                   3.0.12               h2bbff1b_0
packaging                 23.2                     pypi_0    pypi
pandas                    2.1.4                    pypi_0    pypi
pillow                    10.0.1          py311h045eedc_0
pip                       23.3.1          py311haa95532_0
preshed                   3.0.9                    pypi_0    pypi
protobuf                  4.25.1                   pypi_0    pypi
punctuators               0.0.5                    pypi_0    pypi
pycparser                 2.21               pyhd3eb1b0_0
pydantic                  2.5.3                    pypi_0    pypi
pydantic-core             2.14.6                   pypi_0    pypi
pyopenssl                 23.2.0          py311haa95532_0
pyreadline3               3.4.1                    pypi_0    pypi
pysbd                     0.3.4                    pypi_0    pypi
pysocks                   1.7.1           py311haa95532_0
python                    3.11.5               he1021f5_0
python-dateutil           2.8.2                    pypi_0    pypi
pytorch                   2.1.2           py3.11_cuda11.8_cudnn8_0    pytorch
pytorch-cuda              11.8                 h24eeafa_5    pytorch
pytorch-mutex             1.0                        cuda    pytorch
pytz                      2023.3.post1             pypi_0    pypi
pyyaml                    6.0.1           py311h2bbff1b_0
regex                     2023.12.25               pypi_0    pypi
requests                  2.31.0          py311haa95532_0
sentencepiece             0.1.99                   pypi_0    pypi
setuptools                68.2.2          py311haa95532_0
six                       1.16.0                   pypi_0    pypi
smart-open                6.4.0                    pypi_0    pypi
sniffio                   1.3.0                    pypi_0    pypi
soundfile                 0.12.1                   pypi_0    pypi
spacy                     3.7.2                    pypi_0    pypi
spacy-legacy              3.0.12                   pypi_0    pypi
spacy-loggers             1.0.5                    pypi_0    pypi
sqlite                    3.41.2               h2bbff1b_0
srsly                     2.4.8                    pypi_0    pypi
sympy                     1.12            py311haa95532_0
tbb                       2021.8.0             h59b6b97_0
thinc                     8.2.2                    pypi_0    pypi
tiktoken                  0.3.3                    pypi_0    pypi
tk                        8.6.12               h2bbff1b_0
tokenizers                0.15.0                   pypi_0    pypi
torchaudio                2.1.2                    pypi_0    pypi
torchvision               0.16.2                   pypi_0    pypi
tqdm                      4.66.1                   pypi_0    pypi
typer                     0.9.0                    pypi_0    pypi
typing_extensions         4.7.1           py311haa95532_0
tzdata                    2023.4                   pypi_0    pypi
urllib3                   1.26.18         py311haa95532_0
vc                        14.2                 h21ff451_1
vs2015_runtime            14.27.29016          h5e58377_2
wasabi                    1.1.2                    pypi_0    pypi
weasel                    0.3.4                    pypi_0    pypi
wheel                     0.41.2          py311haa95532_0
win32-setctime            1.1.0                    pypi_0    pypi
win_inet_pton             1.1.0           py311haa95532_0
xz                        5.4.5                h8cc25b3_0
yaml                      0.2.5                he774522_0
yarl                      1.9.4                    pypi_0    pypi
zhconv                    1.4.3                    pypi_0    pypi
zlib                      1.2.13               h8cc25b3_0
zstd                      1.5.5                hd43e919_0

运行test chatbot 报错如下

Error
Traceback (most recent call last):
  File "C:\Users\infin\Documents\GitHub\openlrc\tests\test_chatbot.py", line 32, in test_estimate_fee
    fee = bot.estimate_fee(messages)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\infin\Documents\GitHub\openlrc\openlrc\chatbot.py", line 61, in estimate_fee
    token_map[message['role']] += get_text_token_number(message['content'])
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\infin\Documents\GitHub\openlrc\openlrc\utils.py", line 103, in get_text_token_number
    tokens = tiktoken.encoding_for_model(model).encode(text)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\conda\envs\open_lrc_test2\Lib\site-packages\tiktoken\model.py", line 75, in encoding_for_model
    return get_encoding(encoding_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\conda\envs\open_lrc_test2\Lib\site-packages\tiktoken\registry.py", line 63, in get_encoding
    enc = Encoding(**constructor())
                     ^^^^^^^^^^^^^
  File "D:\conda\envs\open_lrc_test2\Lib\site-packages\tiktoken_ext\openai_public.py", line 64, in cl100k_base
    mergeable_ranks = load_tiktoken_bpe(
                      ^^^^^^^^^^^^^^^^^^
  File "D:\conda\envs\open_lrc_test2\Lib\site-packages\tiktoken\load.py", line 115, in load_tiktoken_bpe
    return {
           ^
  File "D:\conda\envs\open_lrc_test2\Lib\site-packages\tiktoken\load.py", line 117, in <dictcomp>
    for token, rank in (line.split() for line in contents.splitlines() if line)
        ^^^^^^^^^^^
ValueError: too many values to unpack (expected 2)
infinitebook commented 10 months ago

可能是tiktoken版本的原因，可以看看你的pip环境吗？

把pip list输出结果贴上来就行。

又重装了一个conda环境，之前是用conda命令装的包，现在全用pip安装，pip check 之后发现numba和numpy的版本好像默认就有点冲突，更新了numba版本之后不冲突了，但是跟tiktoken没啥关系，tiktoken依旧报错，真是没办法了😭😭 大佬的代码写的真好❤️很想用上😭

环境如下

(openlrc) C:\Users\infin>pip list
Package                    Version
-------------------------- ------------
aiohttp                    3.9.1
aiosignal                  1.3.1
annotated-types            0.6.0
antlr4-python3-runtime     4.9.3
anyio                      4.2.0
appdirs                    1.4.4
attrs                      23.2.0
audioread                  3.0.1
av                         10.0.0
blis                       0.7.11
catalogue                  2.0.10
certifi                    2022.12.7
cffi                       1.16.0
charset-normalizer         2.1.1
click                      8.1.7
cloudpathlib               0.16.0
colorama                   0.4.6
coloredlogs                15.0.1
colorlog                   6.8.0
colorthief                 0.2.1
confection                 0.1.4
ctranslate2                3.23.0
customtkinter              5.2.0
cymem                      2.0.8
darkdetect                 0.8.0
DeepFilterLib              0.5.6
DeepFilterNet              0.5.6
distro                     1.9.0
en-core-web-sm             3.7.1
faster-whisper             0.10.0
ffmpeg-normalize           1.27.7
ffmpeg-progress-yield      0.7.8
ffmpeg-python              0.2.0
filelock                   3.9.0
filetype                   1.2.0
flatbuffers                23.5.26
frozenlist                 1.4.1
fsspec                     2023.12.2
future                     0.18.3
h11                        0.14.0
httpcore                   1.0.2
httpx                      0.26.0
huggingface-hub            0.20.2
humanfriendly              10.0
idna                       3.4
jaconvV2                   0.4
Jinja2                     3.1.2
langcodes                  3.3.0
language-data              1.1
lingua-language-detector   1.3.4
llvmlite                   0.41.0
loguru                     0.7.2
marisa-trie                0.7.8
MarkupSafe                 2.1.2
more-itertools             10.1.0
mpmath                     1.3.0
multidict                  6.0.4
murmurhash                 1.0.10
networkx                   3.0
numba                      0.58.1
numpy                      1.26.3
omegaconf                  2.3.0
onnxruntime                1.16.3
openai                     1.6.1
openai-whisper             20230918
openlrc                    1.0.1
packaging                  23.2
pandas                     2.1.4
Pillow                     9.3.0
pip                        23.3.2
preshed                    3.0.9
protobuf                   4.25.1
punctuators                0.0.5
pycparser                  2.21
pydantic                   2.5.3
pydantic_core              2.14.6
pyreadline3                3.4.1
pysbd                      0.3.4
PySide6                    6.4.2
PySide6-Addons             6.4.2
PySide6-Essentials         6.4.2
PySide6-Fluent-Widgets     1.4.5
PySideSix-Frameless-Window 0.3.6
python-dateutil            2.8.2
pytz                       2023.3.post1
pywin32                    306
PyYAML                     6.0.1
regex                      2023.10.3
requests                   2.28.1
scipy                      1.11.4
sentencepiece              0.1.99
setuptools                 65.5.0
shiboken6                  6.4.2
six                        1.16.0
smart-open                 6.4.0
sniffio                    1.3.0
soundfile                  0.12.1
spacy                      3.7.2
spacy-legacy               3.0.12
spacy-loggers              1.0.5
srsly                      2.4.8
sympy                      1.12
thinc                      8.2.2
tiktoken                   0.3.3
tokenizers                 0.15.0
torch                      2.1.0+cu118
torchaudio                 2.1.0+cu118
torchvision                0.16.0+cu118
tqdm                       4.66.1
typer                      0.9.0
typing_extensions          4.9.0
tzdata                     2023.4
urllib3                    1.26.13
wasabi                     1.1.2
weasel                     0.3.4
win32-setctime             1.1.0
yarl                       1.9.4
zhconv                     1.4.3
zh-plus commented 9 months ago

看了相关库的版本（tiktoken=0.3.3），跟我本地是一致的，应该不是库版本的原因。

我搜了下tiktoken库的issue，可能跟用了以前版本的cache有关系，你可以参考下这个回答：https://github.com/openai/tiktoken/issues/136#issuecomment-1563998742 试试能不能解决问题。