search

lucidrains / routing-transformer

Fully featured implementation of Routing Transformer
MIT License
282 stars 29 forks source link

Enquiry about BPC Calculation #30

Open ShiweiLiuFdu opened 2 years ago

ShiweiLiuFdu commented 2 years ago

Hi, I have two questions:

  1. How to calucate the BPC score for the RoutingTransformerLM?
  2. What is the difference between the simple enwik-8 and the deepspeed enwik-8 in the folder 'examples'?