search

dmmiller612 / bert-extractive-summarizer

Easy to use extractive text summarization with BERT
MIT License
1.38k stars 305 forks source link

How can i apply your code for French? #69

Closed ghost closed 4 years ago

ghost commented 4 years ago

Hi @dmmiller612 and guys,

Hope you are all well !

I was wondering how do you apply models like camembert or others available at https://huggingface.co/models?filter=fr with bert-extractive-summarizer.

That would be awesome ! Ca serait super !

Thanks in advance for any insights or inputs on that.

Cheers, X

insop commented 4 years ago

I think this would work. Note that coref doesn't work for non English.

LANG_MODEL='camembert-base''

custom_config = AutoConfig.from_pretrained(LANG_MODEL)
custom_config.output_hidden_states=True
custom_tokenizer = AutoTokenizer.from_pretrained(LANG_MODEL)
custom_model = AutoModel.from_pretrained(LANG_MODEL, config=custom_config)

model = Summarizer(custom_model=custom_model, custom_tokenizer=custom_tokenizer)
resp = model(input_text)
dmmiller612 commented 4 years ago

Yep! @insop solution should do the trick. Thanks!

ghost commented 4 years ago

Hi guys,

Thanks very much for your replies :-) Merci beaucoup

How do I apply these changes to the server instance ? Sorry, I am much more a gopher than a pythonista so I still struggle to make such ports.

Which part summarizer object in the file https://github.com/dmmiller612/bert-extractive-summarizer/blob/master/server.py#L89-L107 ?

Cheers, X

ghost commented 4 years ago

@dmmiller612 any ideas ? just want to test it on a couple of pages.