How to generate my own wordpiece.json

RL4M / MRM-pytorch

An official implementation of Advancing Radiograph Representation Learning with Masked Record Modeling (ICLR'23)

MIT License

73 stars 5 forks source link

How to generate my own wordpiece.json #7

Open ToniChopp opened 1 year ago

ToniChopp commented 1 year ago

Hello, I have found that there are many semantic unrelated tokens in mimic_wordpiece.json. So I want to generate my own wordpiece file. How could I achieve it?