OctoberChang / X-Transformer

X-Transformer: Taming Pretrained Transformers for eXtreme Multi-label Text Classification
BSD 3-Clause "New" or "Revised" License
134 stars 28 forks source link

Update preprocess.py #4

Closed simonlevine closed 4 years ago

simonlevine commented 4 years ago

Replacing feature pull-in function with one that should work more broadly. That is, lines with 'newline' characters previously would cause errors as a result of feature preprocessing. This is common in data with newline characters in each individual 'line' or example. Thanks!