search

oliverguhr / fullstop-deep-punctuation-prediction

A model that predicts the punctuation of English, Italian, French and German texts.
https://huggingface.co/oliverguhr/fullstop-punctuation-multilang-large
MIT License
72 stars 13 forks source link

How to use? #1

Closed gnmarten closed 2 years ago

gnmarten commented 2 years ago

How to use beyond info on card? https://huggingface.co/oliverguhr/fullstop-punctuation-multilang-large/tree/main

from transformers import AutoTokenizer, AutoModelForTokenClassification tokenizer = AutoTokenizer.from_pretrained("oliverguhr/fullstop-punctuation-multilang-large") model = AutoModelForTokenClassification.from_pretrained("oliverguhr/fullstop-punctuation-multilang-large")

thanks in advance

EDIT: nevermind, I found a related issue on Github

huggingface/hub-docs#11

oliverguhr commented 2 years ago

Hi @gnmarten, thanks for the feedback. I added the sample code to the model card. Just that you know: The pipeline code will only process the first 512 token of your input text and truncate the rest.