Overall the multiprocessing validator and singleprocess validator could take a List[Callable] and abstract out the verification process.
Any validation requires a file to be opened. This is strictly a pre-processing step before the files can be used for modeling.
Because of that it is possible to apply threshold values for fields within the open file to classify it as a validated_file or skip_file
yet it is not yet clear to me how to structure this code.
Overall the multiprocessing validator and singleprocess validator could take a
List[Callable]and abstract out the verification process.Any validation requires a file to be opened. This is strictly a pre-processing step before the files can be used for modeling. Because of that it is possible to apply threshold values for fields within the open file to classify it as a
validated_fileorskip_fileyet it is not yet clear to me how to structure this code.