search

NirantK / Hinglish

Hinglish Text Classification
MIT License
30 stars 10 forks source link

Should we do this as a batch instead of one sentence at a time? #43

Open NirantK opened 4 years ago

NirantK commented 4 years ago

Reference Code: https://github.com/NirantK/Hinglish/blob/2798fda87b9b28fa1d7921203ed466c9fd23a28d/hinglishutils.py#L401-#L408

Recommended approach from Huggingface from here: 

batch_sentences = ["Hello I'm a single sentence",
                   "And another sentence",
                   "And the very very last one"]
encoded_inputs = tokenizer(batch_sentences)
print(encoded_inputs)
NirantK commented 3 years ago

Following up -- did you get a chance to look at this?