Try to load tokenizer from Huggingface transformers... 链接到hugging失败

[LZzzin1]

[2023-09-19 17:08:32,244] [INFO] [RANK 0] Try to load tokenizer from Huggingface transformers... [2023-09-19 17:08:52,308] [INFO] [RANK 0] Cannot find THUDM/chatglm-6b from Huggingface or sat. Creating a fake tokenizer... Traceback (most recent call last): File "/mnt/workspace/VisualGLM-6B/finetune_visualglm.py", line 199, in training_main(args, model_cls=model, forward_step_function=forward_step, create_dataset_function=create_dataset_function, collate_fn=data_collator) File "/home/pai/lib/python3.9/site-packages/sat/training/deepspeed_training.py", line 67, in training_main train_data, val_data, test_data = make_loaders(args, hooks['create_dataset_function'], collate_fn=collate_fn) File "/home/pai/lib/python3.9/site-packages/sat/data_utils/configure_data.py", line 199, in make_loaders train = make_dataset(**data_set_args, args=args, dataset_weights=args.train_data_weights, is_train_data=True) File "/home/pai/lib/python3.9/site-packages/sat/data_utils/configure_data.py", line 125, in make_dataset_full d = create_dataset_function(p, args) File "/mnt/workspace/VisualGLM-6B/finetune_visualglm.py", line 160, in create_dataset_function dataset = FewShotDataset(path, image_processor, tokenizer, args) File "/mnt/workspace/VisualGLM-6B/finetune_visualglm.py", line 118, in init input0 = tokenizer.encode("", add_special_tokens=False) AttributeError: 'FakeTokenizer' object has no attribute 'encode' [2023-09-19 17:08:53,689] [INFO] [launch.py:315:sigkill_handler] Killing subprocess 681 [2023-09-19 17:08:53,690] [ERROR] [launch.py:321:sigkill_handler] ['/home/pai/bin/python', '-u', 'finetune_visualglm.py', '--local_rank=0', '--experiment-name', 'finetune-/_config', '--model-parallel-size', '1', '--mode', 'finetune', '--train-iters', '300', '--resume-dataloader', '--max_source_length', '64', '--max_target_length', '256', '--lora_rank', '10', '--layer_range', '0', '14', '--pre_seq_len', '4', '--train-data', './fewshot-data/dataset.json', '--valid-data', './fewshot-data/dataset.json', '--distributed-backend', 'nccl', '--lr-decay-style', 'cosine', '--warmup', '.02', '--checkpoint-activations', '--save-interval', '300', '--eval-interval', '10000', '--save', './checkpoints', '--split', '1', '--eval-iters', '10', '--eval-batch-size', '8', '--zero-stage', '1', '--lr', '0.0001', '--batch-size', '1', '--gradient-accumulation-steps', '4', '--skip-init', '--fp16', '--use_qlora'] exits with return code = 1

[LZzzin1]

How do I load this file from my local and what files do I need to download？

该如何从本地载入此文件呢，都需要下载哪些文件呢

[Liu1217]

你好请问这个问题后来是怎么解决的呢

[NOPAINE]

请问怎么解决的，万分感谢

[LZzzin1]

我使用了vpn，多尝试几次就可以下载完成，从Huggingface上下载模型还没尝试过，但应该有类似的教程