search

microsoft / unilm

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
https://aka.ms/GeneralAI
MIT License
19.25k stars 2.45k forks source link

DeltaLm: the model contains only the weights, where is model's config? #1261

Open Khaled-Elsaka opened 11 months ago

Khaled-Elsaka commented 11 months ago

I am trying to use the model for inference using fairseq like: import torch from deltalm.models.deltalm import DeltaLMModel model = DeltaLMModel.from_pretrained( model_dir, checkpoint_file=model_name, bpe='sentencepiece', sentencepiece_model=spm)

and I get this error : RuntimeError: Neither args nor cfg exist in state keys = dict_keys(['weights']) the deltalm-base.pt contains only the model's weights. how can I use the model properly for inference ?

shumingma commented 11 months ago

DeltaLM is a pretrained language model which is not meant to directly generate text. You can follow the example here to finetune the model and then use this script for inference,

Khaled-Elsaka commented 11 months ago

thanks for the clarification. When I try to fine-tune the model using the example you provided I get this error: 2023-08-27 03:24:24 | INFO | valid | epoch 001 | valid on 'valid' subset | loss -1 | num_updates 3991 Traceback (most recent call last): File "/content/drive/.shortcut-targets-by-id/1IveGZV83gTAvsbVdV7IHmIV7DhiTjtu0/Translation Fine-Tuning/Exploring new models/unilm/deltalm/train.py", line 7, in cli_main() File "/content/drive/.shortcut-targets-by-id/1IveGZV83gTAvsbVdV7IHmIV7DhiTjtu0/Translation Fine-Tuning/Exploring new models/unilm/deltalm/fairseq/fairseq_cli/train.py", line 557, in cli_main distributed_utils.call_main(cfg, main) File "/content/drive/.shortcut-targets-by-id/1IveGZV83gTAvsbVdV7IHmIV7DhiTjtu0/Translation Fine-Tuning/Exploring new models/unilm/deltalm/fairseq/fairseq/distributed/utils.py", line 369, in call_main main(cfg, *kwargs) File "/content/drive/.shortcut-targets-by-id/1IveGZV83gTAvsbVdV7IHmIV7DhiTjtu0/Translation Fine-Tuning/Exploring new models/unilm/deltalm/fairseq/fairseq_cli/train.py", line 190, in main valid_losses, should_stop = train(cfg, trainer, task, epoch_itr) File "/usr/lib/python3.10/contextlib.py", line 79, in inner return func(args, **kwds) File "/content/drive/.shortcut-targets-by-id/1IveGZV83gTAvsbVdV7IHmIV7DhiTjtu0/Translation Fine-Tuning/Exploring new models/unilm/deltalm/fairseq/fairseq_cli/train.py", line 330, in train valid_losses, should_stop = validate_and_save( File "/content/drive/.shortcut-targets-by-id/1IveGZV83gTAvsbVdV7IHmIV7DhiTjtu0/Translation Fine-Tuning/Exploring new models/unilm/deltalm/fairseq/fairseq_cli/train.py", line 421, in validate_and_save valid_losses = validate(cfg, trainer, task, epoch_itr, valid_subsets) File "/content/drive/.shortcut-targets-by-id/1IveGZV83gTAvsbVdV7IHmIV7DhiTjtu0/Translation Fine-Tuning/Exploring new models/unilm/deltalm/fairseq/fairseq_cli/train.py", line 517, in validate valid_losses.append(stats[cfg.checkpoint.best_checkpoint_metric]) KeyError: 'bleu'