-
SentenceVAE/
│
├── encoder.py
│ ```python
│ import torch
│ from torch import nn
│
│ class SentenceEncoder(nn.Module):
│ '''Sentence Encoder with byte-level BPE tokenization, lear…
-
Why do I encounter a situation where the sentence list does not match the encoding list when I use start_rultisprocess_pool() to start the process pool and then start Python multithreading
eg:
batch…
-
```python
In [1]: from tatsu import compile
...:
...: parser = compile(r"""
...: start = text ;
...: text = { sentence }* ;
...: sentence = { word }+ …
gjask updated
2 weeks ago
-
Hi,
Just wondering is this possible to add this simple
task on top of Sen-Trans
Input: one sentence
Output: Multi-label classification
Could not find pre-built task on single sen…
-
## 🐛 Bug
Hi, I tried to test nllb for translating some English sentences to Chinese, and all my sentences are less than 60 tokens. However, most of sentences which more than 30 tokens cannot be gen…
-
`
import sys
sys.path.append('../')
import os
import torch
import math
import numpy as np
from infersent_model import InferSent
EPS = 1e-4
def cosine(u, v):
return np.dot(u, v) / (np…
-
Hello everyone,
I am seeking solutions to a problem I am facing:
I would like to encode a text that is longer than the model's input limit without truncating my text. One solution could be to sp…
-
Currently I need to load a tokenizer from HuggingFace, and use it for simply encoding and decoding sentences. While doing that from Transformers.jl interface is awkward already (I had to go `tok = Tra…
-
I'd like to use GPT to encode my dataset and use the representations further for the task of question generation. I have problems with understanding the code and the name of the arguments in the train…
-
### System Info
- `transformers` version: 4.45.2
- Platform: Linux-5.4.0-193-generic-x86_64-with-glibc2.31
- Python version: 3.12.7
- Huggingface_hub version: 0.25.2
- Safetensors version: 0.4.5
…