Running too slow when using gen_candidate.py to generate candidates

[aidejieceng]

I use the summary model trained based on pegasus-base. On the RTX 8000 GPU card, it takes several seconds for each row of training data to generate candidates, which is too slow. When I need to generate candidate summaries for 5,000,000 data, it takes a very long time. How to speed up the generation of candidates？

[thaokimctu]

In my case, I reduced the number of candidate summaries, split my data into smaller files and used google colab to run multiple notebook at the same time.