issues
search
ybracke
/
transnormer
A lexical normalizer for historical spelling variants using a transformer architecture.
GNU General Public License v3.0
6
stars
1
forks
source link
Efficient generation
#76
Closed
ybracke
closed
7 months ago
ybracke
commented
8 months ago
Huggingface platform on optimizing inference
link
Issue
concerning slow ByT5 tokenizer (is this outdated?)
Pad
with my notes on runtimes, etc.
[x] Sort data by length, so that batches are of similar length;
sorting datasets
[x] Write original index to output data when reordering it
[ ] TODO: Restore original order after sorting by length
Related to: #65
Related to: #65