search

schrum2 / MM-NEAT

Modular Multiobjective (Hyper) Neuro-Evolution of Augmenting Topologies + MAP-Elites: Java code for evolving intelligent agents in Ms. Pac-Man, Tetris, and more, as well as code for Procedural Content Generation in Mario, Zelda, Minecraft, and more!
http://people.southwestern.edu/~schrum2/re/mm-neat.php
Other
50 stars 20 forks source link

Benchmark Minecraft Evolution Speed #922

Closed schrum2 closed 1 year ago

schrum2 commented 1 year ago

I have concerns the the speed of evolution has greatly decreased, and I want to benchmark runs with different settings. The trick to doing this is to have a subdir in the batch directory called MinecraftBenchmark (don't even include year or "Experiments") that explores a lot of options. Importantly, all of these should have a greatly reduced mu and maxGens. Use the settings mu:10 maxGens:100 in all of these files. Otherwise, the first file you make should be a copy of the GECCO batch file MinecraftAccumulate-ME-Observer-VectorPistonOrientation.bat, but rename it to Baseline-NoGlass.bat. Even though clearing with glass is now set false by default, explicitly set minecraftClearWithGlass:false in this file. Then make another file with minecraftClearWithGlass:true called Baseline-Glass.bat.

Run these on separate machines and note the start times. I think the simulation will report the total run time at the end, but try to track it yourself as well. I'll propose more benchmarking in conjunction with other issues, but let's first get a sense for how much glass slows things down.

TjRaffert commented 1 year ago

With those settings Glass got done: 1.4107833333333335 minutes and no glass got 1.2863833333333334

TjRaffert commented 1 year ago

With gens increased to 1000 Glass got done: 11.4648 minutes and no glass got 10.2395

schrum2 commented 1 year ago

Running without glass seems way faster