DEAP / deap

Distributed Evolutionary Algorithms in Python
http://deap.readthedocs.org/
GNU Lesser General Public License v3.0
5.66k stars 1.11k forks source link

Fix and merge MetaCreator #711

Closed fmder closed 11 months ago

fmder commented 11 months ago

76 Merged into current master.

fixes #76

fmder commented 11 months ago

@bje- I think I got everything working from #76. Could you test if Ray can work with this? I added automated tests, but it is not 100% coverage unfortunately.

bje- commented 11 months ago

76 Merged into current master.

I don't see a merge commit on master?

bje- commented 11 months ago

I don't see a merge commit on master?

Ah, it's still on a branch.

bje- commented 11 months ago

No improvement that I can see:

(PoolActor pid=3240907) AttributeError: Can't get attribute 'Individual' on <module 'deap.creator' from '/home/bje/source/nemo/myenv/lib/python3.10/site-packages/deap/creator.py'> 
fmder commented 11 months ago

Was #76 fixing the issue before?

bje- commented 11 months ago

The changes don't merge so I wasn't easily able to test it.

fmder commented 11 months ago

You'd need the changes to be on master?

bje- commented 11 months ago

Still from the fmder-metacreator branch:

ray.exceptions.RaySystemError: System error: Can't get attribute 'Individual' on <module 'deap.creator' from '/home/bje/source/nemo/myenv/lib/python3.10/site-packages/deap/creator.py'>
fmder commented 11 months ago

Ok, i'll check later today if I can find something I missed while merging

fmder commented 11 months ago

Thanks for the feedback

fmder commented 11 months ago

Ok, I've been testing two small scripts recreating a messaging environment, one pickles to file, the other unpickles.

import pickle
from deap import creator

creator.create("TempInd", list)
i = creator.TempInd([1, 2, 3])

with open("ind.pkl", "wb") as f:
    pickle.dump(i, f)
import pickle

from deap import creator

with open("ind.pkl", "rb") as f:
    i = pickle.load(f)

With current version 1.3.3, I get the same error:

AttributeError: Can't get attribute 'TempInd' on <module 'deap.creator' from '[...]/lib/python3.9/site-packages/deap/creator.py'>

while, with the current version, there is no error. The MetaCreator seems to work as intended.

bje- commented 11 months ago

"current version" being master?

bje- commented 11 months ago

Follow-up: I made the change below to onemax_mp.py and it works. I think the problem must be in my code.

diff --git a/examples/ga/onemax_mp.py b/examples/ga/onemax_mp.py
index 32d34e0..3377b5f 100755
--- a/examples/ga/onemax_mp.py
+++ b/examples/ga/onemax_mp.py
@@ -18,6 +18,7 @@ import array
 import multiprocessing
 import random
 import sys
+import ray.util.multiprocessing

 if sys.version_info < (2, 7):
     print("mpga_onemax example requires Python >= 2.7.")
@@ -54,7 +55,7 @@ def main(seed):
     toolbox.register("select", tools.selTournament, tournsize=3)

     # Process Pool of 4 workers
-    pool = multiprocessing.Pool(processes=4)
+    pool = ray.util.multiprocessing.Pool(processes=4)
     toolbox.register("map", pool.map)

     pop = toolbox.population(n=300)