We need to have a folder in our project where all of our pre-processing functions live, so that we can standardize how pre-processing is done for all of the transformer based models we are going to use. Similarly to utils, it would used like this:
from preprocessing import process_dataset
The actual functions and how they work will still need to be decided, but at least this gives us a framework to build around.
We need to have a folder in our project where all of our pre-processing functions live, so that we can standardize how pre-processing is done for all of the transformer based models we are going to use. Similarly to
utils, it would used like this:The actual functions and how they work will still need to be decided, but at least this gives us a framework to build around.