search

salesforce / jaxformer

Minimal library to train LLMs on TPU in JAX with pjit().
BSD 3-Clause "New" or "Revised" License
277 stars 37 forks source link

issue: why you comment this line? Do you still need the bos? #4

Closed PoodleWang closed 2 years ago

PoodleWang commented 2 years ago

https://github.com/salesforce/jaxformer/blob/b8b389493b7c641a1d38bbec1b590d9eed3ff3be/preprocess/5_shuffe_concat_eos.py#L70

Hi, I see your comment this line. Can you elaborate on why you do this?

PoodleWang commented 2 years ago

nevermind, it make sense,