评测 M3E-base 模型报错

zhangyisheng0629 commented 1 year ago

🐛 bug 说明

你好，我在运行以下指令时出现问题，但我不知道哪里错了 python run_mteb_zh.py --model-type sentence_transformer --model-id moka-ai/m3e-base

(embed2) zhangyicheng@xd-deva:/data/zhangyicheng/uniem/mteb-zh$ python run_mteb_zh.py --model-type sentence_transformer --model-id moka-ai/m3e-base
╭────────────────────────── Traceback (most recent call last) ──────────────────────────╮
│ /data/zhangyicheng/uniem/mteb-zh/run_mteb_zh.py:65 in <module>                        │
│                                                                                       │
│   62                                                                                  │
│   63                                                                                  │
│   64 if __name__ == '__main__':                                                       │
│ ❱ 65 │   typer.run(main)                                                              │
│   66                                                                                  │
│                                                                                       │
│ ╭───────────────────────────────────── locals ──────────────────────────────────────╮ │
│ │           AbsTask = <class 'mteb.abstasks.AbsTask.AbsTask'>                       │ │
│ │         Annotated = <class 'typing.Annotated'>                                    │ │
│ │     default_tasks = [                                                             │ │
│ │                     │   <mteb_zh.tasks.TYQSentiment object at 0x7fe767c12590>,    │ │
│ │                     │   <mteb_zh.tasks.TNews object at 0x7fe4c4139690>,           │ │
│ │                     │   <mteb_zh.tasks.JDIphone object at 0x7fe4c2bd7b50>,        │ │
│ │                     │   <mteb_zh.tasks.StockComSentiment object at                │ │
│ │                     0x7fe767d2be10>,                                              │ │
│ │                     │   <mteb_zh.tasks.GubaEastmony object at 0x7fe767d41110>,    │ │
│ │                     │   <mteb_zh.tasks.IFlyTek object at 0x7fe4c457abd0>,         │ │
│ │                     │   <mteb_zh.tasks.T2RReranking object at 0x7fe7639a0350>,    │ │
│ │                     │   <mteb_zh.tasks.T2RRetrieval object at 0x7fe4c2bf5150>,    │ │
│ │                     │   <mteb_zh.tasks.MedQQPairs object at 0x7fe4c2bf5390>       │ │
│ │                     ]                                                             │ │
│ │    filter_by_name = <function filter_by_name at 0x7fe768f05f80>                   │ │
│ │    filter_by_type = <function filter_by_type at 0x7fe4c2be2480>                   │ │
│ │      GubaEastmony = <class 'mteb_zh.tasks.GubaEastmony'>                          │ │
│ │           IFlyTek = <class 'mteb_zh.tasks.IFlyTek'>                               │ │
│ │          JDIphone = <class 'mteb_zh.tasks.JDIphone'>                              │ │
│ │        load_model = <function load_model at 0x7fe4c2b81da0>                       │ │
│ │              main = <function main at 0x7fe4c2be3560>                             │ │
│ │        MedQQPairs = <class 'mteb_zh.tasks.MedQQPairs'>                            │ │
│ │         ModelType = <enum 'ModelType'>                                            │ │
│ │              MTEB = <class 'mteb.evaluation.MTEB.MTEB'>                           │ │
│ │              Path = <class 'pathlib.Path'>                                        │ │
│ │ StockComSentiment = <class 'mteb_zh.tasks.StockComSentiment'>                     │ │
│ │      T2RReranking = <class 'mteb_zh.tasks.T2RReranking'>                          │ │
│ │      T2RRetrieval = <class 'mteb_zh.tasks.T2RRetrieval'>                          │ │
│ │          TaskType = <enum 'TaskType'>                                             │ │
│ │             TNews = <class 'mteb_zh.tasks.TNews'>                                 │ │
│ │             typer = <module 'typer' from                                          │ │
│ │                     '/data/zhangyicheng/anaconda3/envs/embed2/lib/python3.11/sit… │ │
│ │      TYQSentiment = <class 'mteb_zh.tasks.TYQSentiment'>                          │ │
│ ╰───────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                       │
│ /data/zhangyicheng/anaconda3/envs/embed2/lib/python3.11/site-packages/typer/main.py:1 │
│ 056 in run                                                                            │
│                                                                                       │
│ /data/zhangyicheng/anaconda3/envs/embed2/lib/python3.11/site-packages/typer/main.py:3 │
│ 28 in __call__                                                                        │
│                                                                                       │
│ /data/zhangyicheng/anaconda3/envs/embed2/lib/python3.11/site-packages/typer/main.py:3 │
│ 11 in __call__                                                                        │
│                                                                                       │
│ /data/zhangyicheng/anaconda3/envs/embed2/lib/python3.11/site-packages/typer/main.py:3 │
│ 64 in get_command                                                                     │
│                                                                                       │
│ /data/zhangyicheng/anaconda3/envs/embed2/lib/python3.11/site-packages/typer/main.py:5 │
│ 77 in get_command_from_info                                                           │
│                                                                                       │
│ /data/zhangyicheng/anaconda3/envs/embed2/lib/python3.11/site-packages/typer/main.py:5 │
│ 53 in get_params_convertors_ctx_param_name_from_function                              │
│                                                                                       │
│ /data/zhangyicheng/anaconda3/envs/embed2/lib/python3.11/site-packages/typer/main.py:8 │
│ 50 in get_click_param                                                                 │
│                                                                                       │
│ /data/zhangyicheng/anaconda3/envs/embed2/lib/python3.11/site-packages/typer/main.py:7 │
│ 79 in get_click_type                                                                  │
╰───────────────────────────────────────────────────────────────────────────────────────╯
RuntimeError: Type not yet supported: str | None

Package Version

aiohttp 3.8.5 aiosignal 1.3.1 aniso8601 9.0.1 ansi2html 1.8.0 arrow 1.2.3 async-timeout 4.0.3 attrs 23.1.0 beautifulsoup4 4.12.2 beir 2.0.0 blinker 1.6.2 boltons 23.0.0 brotlipy 0.7.0 captum 0.6.0 certifi 2023.7.22 cffi 1.15.1 chardet 4.0.0 charset-normalizer 2.0.4 click 8.1.6 cmake 3.27.2 coloredlogs 15.0.1 conda 23.7.2 conda-build 3.26.0 conda_index 0.2.3 conda-package-handling 2.2.0 conda_package_streaming 0.9.0 contourpy 1.1.0 cryptography 41.0.2 cycler 0.11.0 Cython 0.29.34 datasets 2.14.4 dill 0.3.7 elasticsearch 7.9.1 enum-compat 0.0.3 faiss-cpu 1.7.4 filelock 3.9.0 Flask 2.3.2 Flask-RESTful 0.3.10 fonttools 4.42.0 frozenlist 1.4.0 fsspec 2023.6.0 gensim 4.3.1 glob2 0.7 huggingface-hub 0.16.4 humanfriendly 10.0 idna 3.4 itsdangerous 2.1.2 jieba 0.42.1 Jinja2 3.1.2 joblib 1.3.2 jsonlines 3.1.0 jsonpatch 1.32 jsonpointer 2.1 kiwisolver 1.4.4 libarchive-c 2.9 lit 16.0.6 loguru 0.7.0 markdown-it-py 3.0.0 MarkupSafe 2.1.1 matplotlib 3.7.2 mdurl 0.1.2 more-itertools 8.12.0 mpmath 1.3.0 mteb 1.1.0 multidict 6.0.4 multiprocess 0.70.15 networkx 3.1 nltk 3.8.1 numpy 1.24.3 nvgpu 0.10.0 openai 0.27.8 optimum 1.11.1 packaging 23.1 pandas 2.0.3 Pillow 9.3.0 pip 23.2.1 pkginfo 1.9.6 pluggy 1.0.0 protobuf 4.24.0 psutil 5.9.5 pyarrow 12.0.1 pycosat 0.6.4 pycparser 2.21 Pygments 2.16.1 pynvml 11.4.1 pyOpenSSL 23.2.0 pyparsing 3.0.9 PySocks 1.7.1 python-dateutil 2.8.2 pytrec-eval 0.5 pytz 2022.7 PyYAML 6.0 regex 2023.8.8 requests 2.31.0 rich 13.5.2 ruamel.yaml 0.17.21 safetensors 0.3.2 scikit-learn 1.3.0 scipy 1.11.1 sentence-transformers 2.2.2 sentencepiece 0.1.99 setuptools 68.0.0 six 1.16.0 smart-open 6.3.0 soupsieve 2.4 sympy 1.12 tabulate 0.9.0 termcolor 2.3.0 text2vec 1.2.2 threadpoolctl 3.2.0 tokenizers 0.13.3 toolz 0.12.0 torch 2.0.1+cu118 torch-model-archiver 0.8.1 torch-workflow-archiver 0.2.9 torchaudio 2.0.2+cu118 torchdata 0.6.1 torchpippy 0.1.1 torchserve 0.8.1 torchtext 0.15.2+cpu torchvision 0.15.2+cu118 tqdm 4.65.0 transformers 4.31.0 triton 2.0.0 typer 0.9.0 typing_extensions 4.7.1 tzdata 2023.3 urllib3 1.26.16 Werkzeug 2.3.6 wheel 0.40.0 xxhash 3.3.0 yarl 1.9.2 zstandard 0.19.0

上面是环境的python包，非常感谢

Python Version

3.11

wangyuxinwhy commented 1 year ago

RuntimeError: Type not yet supported: str | None

报这个错代表 python 的版本太低了，要 3.10 及以上

zhangyisheng0629 commented 1 year ago

你好我用的是3.11，您能给我一些建议吗，非常感谢~！

wangyuxinwhy commented 1 year ago

还真是，我测了一下发现是 python 3.10 和 3.11，在 typing.get_type_hints(str | None) 的返回结果不同导致的。3.10 返回 typing.Optional[str] 3.11 返回 str | None ，typer 没有办法处理 str | None

如何解决？第一种是使用 Python3.10 第二种是把 str | None 改成 Optional[str]

我这边会更新一下代码，把 str | None 改成 Optional[str]

wangyuxinwhy / uniem