-
## Background
- here in mindspore, there is text part in dataset for doing data_augmentation in some nlp task
- in this process, we need some tokenizer to deal with origin text data, firstly tokeniz…
-
Hello,
We have been developing a FastAPI application where we use some external libraries to perform some NLP tasks, such as tokenization. On top of this, we are launching the service with Gunicorn …
-
CODE:
```
import transformers
import torch
tokenizer = transformers.LlamaTokenizer.from_pretrained('chaoyi-wu/PMC_LLAMA_7B')
model = transformers.LlamaForCausalLM.from_pretrained('chaoyi-wu/PMC_L…
-
can't run this example on jax or pytorch backend
it just works on tensorflow backend
https://keras.io/examples/nlp/neural_machine_translation_with_keras_nlp/
also inferencing is significantly s…
-
I am running your code and found there is one issue :
`sentence_embedding = torch.mean(hidden_states[-1], dim=1).squeeze()`
should be
`sentence_embedding = torch.mean(hidden_states[-1], dim=…
-
I love TextBlob, thank you so much for making this awesome Python tool :+1:
I am wondering if there is a solution to a tokenization issue I'm seeing. Here's some example code with an excerpt from G…
-
I expect text.parsedSentences should contain all sentences. let me explain the problem with code :)
place it in parse/tokenizer_test.go file
```
func TestTokenizeText(t *testing.T) {
rule := Ne…
-
Hi. Kudos for this nice work. I am trying to reproduce the results on DailyDialog dataset. It will be very helpful if you can clarify the following details.
In [Issue #13](https://github.com/ictnlp/D…
-
**Please describe the module you would like to add to the content library**
I have one large paragraph which contains multiple sentences, which I want to detect
**Do you already have an implementa…
-
Hi,
I am getting
ValueError: Stop argument for islice() must be None or an integer: 0 209 clear_cache=clear_cache,
210 no_cache=no_cache,
211 )
/usr/…