Fix preprocess taking too long

[ShiromiyaG]

Description

In the previous code, preprocess was taking too long because it had to read all the files twice

Motivation and Context

How has this been tested?

Doing the preprocess part

Screenshots (if appropriate):

Types of changes

• [X] Bug fix (non-breaking change which fixes an issue)
• [ ] New feature (non-breaking change which adds functionality)
• [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

• [X] My code follows the code style of this project.
• [ ] My change requires a change to the documentation.
• [ ] I have updated the documentation accordingly.
• [ ] I have added tests to cover my changes.
• [X] All new and existing tests passed.

[blaisewf]

is ready to merge?

[ShiromiyaG]

Yes, I think so