search

karpathy / minGPT

A minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training
MIT License
20.3k stars 2.53k forks source link

Information leak in training procedure？ #107

Closed ljch2018 closed 1 year ago

ljch2018 commented 1 year ago

In the demo，x=unsorted array + sorted array as input. When in training, the model may use sorted array as reference which may lead to information leak?