search

dmmiller612 / bert-extractive-summarizer

Easy to use extractive text summarization with BERT
MIT License
1.38k stars 305 forks source link

OSError: Can't load config for 'bert-large-uncased'. Make sure that: #79

Closed SyedAgha closed 2 years ago

SyedAgha commented 3 years ago

Hi,

I am getting following error after running model = Summarizer() in the code:

OSError Traceback (most recent call last) /home/cdsw/.local/lib/python3.6/site-packages/transformers/configuration_utils.py in get_config_dict(cls, pretrained_model_name_or_path, **kwargs) 242 if resolved_config_file is None: --> 243 raise EnvironmentError 244 config_dict = cls._dict_from_json_file(resolved_config_file)

OSError:

During handling of the above exception, another exception occurred:

OSError Traceback (most recent call last)

in () 1 body = 'Text body that you want to summarize with BERT' ----> 2 model = Summarizer() /home/cdsw/.local/lib/python3.6/site-packages/summarizer/model_processors.py in __init__(self, model, custom_model, custom_tokenizer, hidden, reduce_option, sentence_handler, random_state) 174 """ 175 super(Summarizer, self).__init__( --> 176 model, custom_model, custom_tokenizer, hidden, reduce_option, sentence_handler, random_state 177 ) 178 /home/cdsw/.local/lib/python3.6/site-packages/summarizer/model_processors.py in __init__(self, model, custom_model, custom_tokenizer, hidden, reduce_option, sentence_handler, random_state) 135 model=model, custom_model=custom_model, custom_tokenizer=custom_tokenizer, 136 hidden=hidden, reduce_option=reduce_option, --> 137 sentence_handler=sentence_handler, random_state=random_state 138 ) 139 /home/cdsw/.local/lib/python3.6/site-packages/summarizer/model_processors.py in __init__(self, model, custom_model, custom_tokenizer, hidden, reduce_option, sentence_handler, random_state) 33 34 np.random.seed(random_state) ---> 35 self.model = BertParent(model, custom_model, custom_tokenizer) 36 self.hidden = hidden 37 self.reduce_option = reduce_option /home/cdsw/.local/lib/python3.6/site-packages/summarizer/bert_parent.py in __init__(self, model, custom_model, custom_tokenizer) 44 self.model = custom_model.to(self.device) 45 else: ---> 46 self.model = base_model.from_pretrained(model, output_hidden_states=True).to(self.device) 47 48 if custom_tokenizer: /home/cdsw/.local/lib/python3.6/site-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs) 585 proxies=proxies, 586 local_files_only=local_files_only, --> 587 **kwargs, 588 ) 589 else: /home/cdsw/.local/lib/python3.6/site-packages/transformers/configuration_utils.py in from_pretrained(cls, pretrained_model_name_or_path, **kwargs) 199 200 """ --> 201 config_dict, kwargs = cls.get_config_dict(pretrained_model_name_or_path, **kwargs) 202 return cls.from_dict(config_dict, **kwargs) 203 /home/cdsw/.local/lib/python3.6/site-packages/transformers/configuration_utils.py in get_config_dict(cls, pretrained_model_name_or_path, **kwargs) 250 f"- or '{pretrained_model_name_or_path}' is the correct path to a directory containing a {CONFIG_NAME} file\n\n" 251 ) --> 252 raise EnvironmentError(msg) 253 254 except json.JSONDecodeError: OSError: Can't load config for 'bert-large-uncased'. Make sure that: - 'bert-large-uncased' is a correct model identifier listed on 'https://huggingface.co/models' - or 'bert-large-uncased' is the correct path to a directory containing a config.json file How can I solve this error? Thanks, Agha
zdhernandez commented 3 years ago 
custom_config = AutoConfig.from_pretrained('./bert-large-uncased-config.json')
custom_config.output_hidden_states=True
custom_tokenizer = AutoTokenizer.from_pretrained('./bert-base-uncased-vocab.txt')
custom_model = AutoModel.from_pretrained('./bert-large-uncased-pytorch_model.bin', config=custom_config)

model = Summarizer(custom_model=custom_model, custom_tokenizer=custom_tokenizer, sentence_handler=handler)
print(model(body, num_sentences=3))