Bootstrapper creates the dataset mappings when it initializes the Dataset Manager.
Optimizer uses Dataset Manager (passed from Bootstrapper -> Communication Manager -> Optimizer) in order to get dataset mappings whenever it gets initialization payload
Optimizer receives a uuid on initialization payload instead of the actual filepath.
It uses the mappings to map this uuid to a specific dataset filepath and it sets the filepath to the job.
Error handling for uuid
Tests and testing utils have been refactored all over to make these changes (might make the PR look long, but there was honestly a lot of stuff to update).
I might need to update documentation and clean up some code (my fault, I took a break on this bc finals and need to review everything a few more times), but the core logic, tests should stay the same and tests pass so it's pretty much ready for review..
Couple major changes:
I might need to update documentation and clean up some code (my fault, I took a break on this bc finals and need to review everything a few more times), but the core logic, tests should stay the same and tests pass so it's pretty much ready for review..