-
## 🐛 Bug
**Describe the bug**
In short, an empty generator is created when calling `__getattr__` with an unknown attribute on `torchtext.data.dataset`. [Here is code](https://github.com/pytorch/text…
-
/usr/bin/python3.5 /home/scrooge/chatbot/seqGanChatbot/execute.py
/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/dtypes.py:493: FutureWarning: Passing (type, 1) or '1type' as a sy…
-
I exported clip-ViT-B-32-multilingual-v1 to onnx with some modifications(no effect on the output embedding).
hf optimum onnx export can export this model with (0) Transformer and (1) Pooling. But …
yaman updated
4 months ago
-
From the discussion of #3243:
It would be nice to have a CharFilter? to mark sentence boundaries.
Such functionality would be useful for:
- prevent phrase queries with 0 slop from matching across sen…
-
Hi,
I'm using EasyNMT for translating customer reviews. During translation, I got this error
HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/api/models/Helsinki-NLP/opus-mt-…
-
## 🐞Describing the bug
The output of converted Coreml model and original Pytorch model is different. Obvious mismatch is observed. I also notice that there are some similar issues that have been prop…
-
While trying to retrain a sentence tokenizer model with `PunktTokenizer`, the NLTK code took up >200GB of RAM and a lot of swap and doesn't seem to end after 2 days of training.
```python
import …
-
### Question Validation
- [X] I have searched both the documentation and discord for an answer.
### Question
Hi, I want to know how to count tokens (Embedding Tokens, LLM Prompt Tokens, LLM Complet…
-
Hi, my collegues and I have released [UD-Kanbun](https://github.com/KoichiYasuoka/UD-Kanbun), a python-based tokenizer, POS-tagger, and dependency-parser for classical Chinese texts. And now we are in…
-
This is the future warning we are currently reciving:
transformers\tokenization_utils_base.py:1601: FutureWarning: `clean_up_tokenization_spaces` was not set. It will be set to `True` by default. T…