search

DEAP / deap

Distributed Evolutionary Algorithms in Python
http://deap.readthedocs.org/
GNU Lesser General Public License v3.0
5.75k stars 1.12k forks source link

How can I use GPU instead of multiprocessing or scoop in DEAP? #684

Open ppseverin opened 1 year ago

ppseverin commented 1 year ago

Hi! I'm new to using GPU to speed up computations, so this might be a silly question. I have a script that uses DEAP and actually it uses multiprocessing. I would like to use my GPU (it's a RTX 3080 TI, in case it's important to know) to speed up calculations and the whole process. I googled for some examples of how to do it, but couldn't find any :(. Is this possible? How can I achieve this?

fmder commented 1 year ago

Hi, no silly questions! You'll need to code your evaluation function in a GPU compatible langage such as Numba or CuPy or Pytorch core. However, moving from cpu to gpu has a lot of overhead, you may need to group the evaluation of your whole population together to observe speedup.

Mujtaba1986 commented 7 months ago

Last I checked about this a year ago, I found DEAP does not support this.

fmder commented 7 months ago

Your evaluation function must enable it.

Le jeu. 1 févr. 2024, 10 h 03, Mujtaba1986 @.***> a écrit :

Last I checked about this a year ago, I found DEAP does not support this.

— Reply to this email directly, view it on GitHub https://github.com/DEAP/deap/issues/684#issuecomment-1921536796, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHKXQXF2HUKITFHOU5WIF3YROVDDAVCNFSM6AAAAAAUAKJ4AKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRRGUZTMNZZGY . You are receiving this because you commented.Message ID: @.***>

Leon-Qiao commented 5 months ago

I also encountered a similar situation. The main task of my evaluation function was to make predictions through a pre-trained deep-learning model. I gave up using the map function, directly fed the list of individuals into the evaluate function, and converted it into a NumPy array. The return value of the evaluate function was also a batch of fitnesses. By implementing batch calculations instead of using the map to iterate over a list with multiple single-sample calls, the efficiency of my genetic algorithm dramatically improves.