alibaba / EasyNLP

EasyNLP: A Comprehensive and Easy-to-use NLP Toolkit
Apache License 2.0
2.02k stars 249 forks source link

AppZoo实践时checkpoint_dir找不到 #313

Open playeriu opened 1 year ago

playeriu commented 1 year ago

根据https://www.yuque.com/easyx/easynlp/ts4czl 文档上的代码尝试使用文本向量化功能,但是报错找不到checkpoint_dir 请问是什么地方可能有问题呢 File "/home/iu/anaconda3/envs/easyNLP/lib/python3.8/site-packages/pai_easynlp-0.1.2-py3.8.egg/easynlp/modelzoo/configuration_utils.py", line 490, in get_config_dict raise Exception(f'{pretrained_model_name_or_path} is not a filer or folder.') Exception: bert-small-uncased is not a filer or folder.

playeriu commented 1 year ago

以下是我使用的脚本 if [ ! -f ./dev.tsv ]; then wget http://atp-modelzoo-sh.oss-cn-shanghai.aliyuncs.com/release/tutorials/classification/dev.tsv fi

mode=$1

if [ "$mode" = "predict" ]; then

easynlp \
--mode=$mode \
--worker_gpu=1 \
--tables=dev.tsv \
--outputs=dev.pred.tsv \
--input_schema=label:str:1,sid1:str:1,sid2:str:1,sent1:str:1,sent2:str:1 \
--output_schema=pooler_output,first_token_output,all_hidden_outputs \
--first_sequence=sent1 \
--append_cols=label \
--checkpoint_dir=bert-small-uncased \
--micro_batch_size=32 \
--sequence_length=128 \
--app_name=vectorization

fi