Open liwenju0 opened 1 year ago
when i read the source code of preprocess_data.py file。 i am confused with below code: in function perfect_align :
when call apply_transformation, the cdoe ' '.join(T[j:k]) will insert 3 spaces between tokens.
but, look at the source code of apply_transformation:
it will call check_equal, check_casetype, check_verb, check_plural sequencely, but i think inserted space will affect these check funciton.
when i read the source code of preprocess_data.py file。 i am confused with below code: in function perfect_align :
when call apply_transformation, the cdoe ' '.join(T[j:k]) will insert 3 spaces between tokens.
but, look at the source code of apply_transformation:
it will call check_equal, check_casetype, check_verb, check_plural sequencely, but i think inserted space will affect these check funciton.