Open Dadle opened 4 years ago
This may get more responses in the discussion thread vs the bug reporting system: https://groups.google.com/forum/#!forum/deap-users
Can you run a tests to confirm that an evaluation is determinate between the two cases? Map to the cluster the same individual for eval and see what happens essentially, are the results all identical? I would guess off the top of my head that the global scope or something is not being setup in the workers the same way that the 'main' process had its scope constructed.
Not sure if this is a feature or a bug, but would love some input either way. Whenever I replace map to run DEAP parallellized as in code at the bottom I consistently get worse fitness development than with a single threaded version of the same code (just commenting out the dask_map line). These results have been replicated with both threading using multiprocessing.dummy and Dask over 5 runs each on 2 optimization problems.
What causes this consistent difference of 400 fitness between a single threaded and parallellized version of the algorithm?
Notice below fitness is minimized
Example fitness single threaded:![image](https://user-images.githubusercontent.com/16399255/69589117-33dbde80-0feb-11ea-954c-8f80fa6647dd.png)
Example fitness Dask multiprocessing with 2 CPUs:![image](https://user-images.githubusercontent.com/16399255/69589367-26732400-0fec-11ea-929b-5045cc89b820.png)
Setup of evolution:
Dask map function used: