This architecture combines reinforcement learning with transformer architecture .
The above code was made to run into kaggle notebook so using kaggle is advised . For best results copy-paste the code into Kaggle and use it . Use your own parameters and dataset . Make sure there are the same parameters in Inference and Training codes. You can use Normal_gpu_whole_folder for training and Inference_Normal_gpu_whole_folder for inference There are other options as well , you can use just 1 selected .txt file for training or you can choose to train the model using TPU . For these options I don't offer Inference code at the momoent .