Closed SepShr closed 2 years ago
So, I found three different issues, addressed in the following commits:
This is to fix a bug in calculating the safety_req_value
value of each complete solution. Previously, the value was zero for all complete solutions since the map()
function you used did not properly save the results in the corresponding field.
This is to reduce the "radius" (i.e., max_dist
) of a cs_region
; since 0.5 was too large, the search does not make fine-grained movements and is easily stuck in some regions.
This is to make the MTQ problem "solvable". Previously, use used h2= 150
but s_2 = 1.0/32.0
, meaning that it is very challenging to find the peak of h2 on a very small surface (1/32). So, I simply swapped h1
and h2
to make the problem easier to solve.
This is the full log: 20220609_160328_CCEA_MTQ.log
As you can see, we have reached the fitness score of 0.9735897570338863
.
Using the final solution x, y = [0.5853844653731718], [0.894080760273642]
, the function compute_safety_req_value()
returns -21.786309582260785, which is quite close to 0
.
In fact, on different runs, I found x, y = [0.6182038195955862], [0.8721837981696042]
, resulting the return value of compute_safety_req_value()
only 1.5513294434849598
.
Since the positive return value of compute_safety_req_value()
indicates "safe" and the negative one indicates "unsafe", the return value of zero indicates the "boundary". Therefore, we can say that the algorithm successfully found a complete solution (x, y) that lies near the boundary.
To conclude, the sanity check is passed, but it turned out that max_dist
should be carefully tuned for a given problem
To make the algorithm work with Pylot, aside from mirroring the changes in MTQ to Pylot's problem files, the returned values from the simulation had to be modified in the following manner: previously the values varied between 0
and 1
, where 1
corresponded to no deviation, whereas 0
meant violation of the safety limit. To ensure the sign changes in case of violation, -1
will be returned instead of 1
, but the rest remains the same. For instance, it does not matter how close to violation the complete solution is, as long as it has not violated the safety limit, it would have a positive value. In case of violation, we do not measure how much violation occurs and the value of -1
is returned. The reason for this decision is due to nature of the safety violation. In rough terms, if a safety violation occurs, it is game over and measuring how much violation has happened is not reasonable. However, violations can be differentiated based on their severity, for instance, crashing into a person is much more severe than crashing into a lamp post. However, for a single safety requirement, all violations can be seen as equal. Nevertheless, to improve our search algorithm, we might want to use a threshold to count a value as a violation rather than only focusing on 0
.
Here is a log of a test run: 20220613_171322_CCEA.log
The attached log is for the test run No. 5. In this test run, interestingly, a bug with the mutate_mlco()
was discovered. After breeding, the following mlco was created:
mlco=[[[0, 0, 0, 0.0, 50.0, 0.0, 50.0, 0.0, 50.0, 0.0, 50.0], [1, 0, 0, 0.0, 50.0, 0.0, 50.0, 0.0, 50.0, 0.101, 50.101]]
. Such an mlco would lead to a ZeroDivisionError
, since the start and end times are 0. Hopefully, commit e8b3ef43f54d829cfabc7f8eae76712ca6c879f9 addresses this issue at ln.201 of problem_utils.py
. Related to this issue, I realized that the test_mutate_mlco()
unit test is flakey, for which issue #40 is opened.
We can conclude the setup is reasonably working. However, due to the importance of the effort, your review of the updated code is much appreciated. To review the code, pull the latest commit and run python3 run_iccea.py
. Kindly note that you can modify the search parameters in ./search_config.py
.
If your review is satisfactory, then the last step before focusing on evaluations would be to parallelize the simulations, please see issue #39 .
In case of violation, we do not measure how much violation occurs and the value of -1 is returned. The reason for this decision is due to nature of the safety violation. In rough terms, if a safety violation occurs, it is game over and measuring how much violation has happened is not reasonable.
Okay, sounds reasonable. Also, since our current fitness evaluation only counts for the number of safe and unsafe complete solutions, it is enough to distinguish between safe and unsafe complete solutions without considering the degree of safety violations.
We can conclude the setup is reasonably working. However, due to the importance of the effort, your review of the updated code is much appreciated.
Okay, I will review the commits now. Thanks.
I made some improvements in mutate_time()
and test_mutate_time()` in this commit: 66fa7881c12eb2212f4e514e4a41be3eebf82fed.
I couldn't look at the other code, but I can do it tomorrow morning if you want. Otherwise, feel free to go ahead if you already feel it's good to try to run. :-)
As you know, I conducted a test run of around 31 hours. The results are, well, weird. The populations had a size of 5
each with the min_distance being 0.5
. The number of generations was 15
and the total number of evaluations was 427
simulations.
The detailed log of the run can be accessed here: 20220614_124948_CCEA.log.
The terminal output of the run is also available which provides additional information on duration of the run (I just realized that this can be simply added to the log :smile:. terminal_output.log
Finally, the statistical information on the fitness value for throughout the generations is available here: 20220614_124948_logbook.log. I have printed the logbook in the following.
fitness size
------------------------------------------------------------ -------------------
gen type min avg max std min avg max
0 scen [0.6234189] [0.68703697] [0.7341396] [0.03716327] 7 7 7
0 mlco [0.67981879] [0.70762463] [0.7341396] [0.01760144] 2 2 2
0 cs [0.44034454] [0.63897178] [0.7341396] [0.0728814] 2 2 2
1 scen [0.72929305] [0.74862936] [0.7995379] [0.02635198] 7 7 7
1 mlco [0.69340341] [0.73918904] [0.7995379] [0.03502136] 2 2 2
1 cs [0.56928779] [0.68733723] [0.7995379] [0.0486562] 2 2 2
2 scen [0.66852081] [0.71785378] [0.82932162] [0.05820451] 7 7 7
2 mlco [0.62530197] [0.76377781] [0.82932162] [0.08146579] 2 2 2
2 cs [0.55732867] [0.66228839] [0.82932162] [0.07133759] 2 2 2
3 scen [0.69144882] [0.73377064] [0.83057685] [0.04992332] 7 7 7
3 mlco [0.65040406] [0.7869763] [0.83057685] [0.06846054] 2 2 2
3 cs [0.58519065] [0.69083291] [0.83057685] [0.07632528] 2 2 2
4 scen [0.53940695] [0.60849553] [0.72236995] [0.0690791] 7 7 7
4 mlco [0.5987755] [0.64800697] [0.72236995] [0.04098705] 2 2 2
4 cs [0.50767005] [0.56693144] [0.72236995] [0.05131364] 2 2 2
5 scen [0.47575206] [0.55725396] [0.69194258] [0.07840432] 7 7 7
5 mlco [0.53404053] [0.60616453] [0.69194258] [0.06510572] 2 2 2
5 cs [0.44280526] [0.5130585] [0.69194258] [0.06298238] 2 2 2
6 scen [0.44901181] [0.56741654] [0.64454244] [0.066772] 7 7 7
6 mlco [0.57303726] [0.60352767] [0.64454244] [0.02468984] 2 2 2
6 cs [0.44901181] [0.55564525] [0.64454244] [0.05145507] 2 2 2
7 scen [0.42800984] [0.46625675] [0.50842683] [0.03011656] 7 7 7
7 mlco [0.42494099] [0.47881817] [0.50842683] [0.02940178] 2 2 2
7 cs [0.38592126] [0.44488609] [0.50842683] [0.03412655] 2 2 2
8 scen [0.35161408] [0.42322096] [0.52575649] [0.06955965] 7 7 7
8 mlco [0.43690625] [0.4806653] [0.52575649] [0.02818666] 2 2 2
8 cs [0.34561999] [0.42621187] [0.52575649] [0.06030999] 2 2 2
9 scen [0.37986032] [0.47973068] [0.53569025] [0.05711809] 7 7 7
9 mlco [0.47005378] [0.50323696] [0.53569025] [0.02646158] 2 2 2
9 cs [0.37986032] [0.48600296] [0.53569025] [0.04091374] 2 2 2
10 scen [0.53322187] [0.5766821] [0.60302637] [0.02513307] 7 7 7
10 mlco [0.53813287] [0.57034537] [0.60302637] [0.02241255] 2 2 2
10 cs [0.48848674] [0.54284181] [0.60302637] [0.03220183] 2 2 2
11 scen [0.54075197] [0.56935929] [0.58995124] [0.01665187] 7 7 7
11 mlco [0.54919638] [0.5694217] [0.58995124] [0.01402076] 2 2 2
11 cs [0.50059416] [0.55574353] [0.58995124] [0.02271427] 2 2 2
12 scen [0.51541146] [0.52568993] [0.54021618] [0.00847303] 7 7 7
12 mlco [0.52316174] [0.52916383] [0.54021618] [0.00581389] 2 2 2
12 cs [0.48564721] [0.5145295] [0.54021618] [0.01349907] 2 2 2
13 scen [0.49855108] [0.55390467] [0.57645644] [0.02810277] 7 7 7
13 mlco [0.55178689] [0.56853928] [0.57645644] [0.00905047] 2 2 2
13 cs [0.46909499] [0.54431] [0.57645644] [0.03087536] 2 2 2
14 scen [0.5244632] [0.53967216] [0.55956432] [0.01236508] 7 7 7
14 mlco [0.54255052] [0.55127563] [0.55956432] [0.00687808] 2 2 2
14 cs [0.43050504] [0.5219403] [0.55956432] [0.03245033] 2 2 2
The curious behavior in the fitness values does not match our expectation (it does not increase). However, in the logs, the cs with highest fitness is recorded as having the values of 0.83
. There might be issues with the logbook that I will investigate and report the results on. However, the algorithm has gotten stuck on a local optima from the 4rd generation and did not improve.
I will add more to the discussion in later comments.
Hi, I ran an MTQ for 100 generations. The log can be found here: 20220616_160610_CCEA_MTQ.log
The fitness stats per generation are as follows:
fitness size
------------------------------------------------------------ -------------------
gen type min avg max std min avg max
0 scen [0.46924696] [0.57235399] [0.71521253] [0.08066848] 1 1 1
0 mlco [0.38137794] [0.48082553] [0.71521253] [0.07585265] 1 1 1
0 cs [0.16177757] [0.34403937] [0.71521253] [0.12035314] 2 2 2
1 scen [0.43314687] [0.57673013] [0.71471573] [0.11435636] 1 1 1
1 mlco [0.40595225] [0.52934783] [0.71471573] [0.08558928] 1 1 1
1 cs [0.13533558] [0.36546781] [0.71471573] [0.1391729] 2 2 2
2 scen [0.30494452] [0.58984317] [0.85199279] [0.2268949] 1 1 1
2 mlco [0.40595225] [0.63791735] [0.85199279] [0.19981765] 1 1 1
2 cs [0.12116625] [0.38101467] [0.85199279] [0.21774756] 2 2 2
3 scen [0.2766141] [0.58976771] [0.88595519] [0.26009821] 1 1 1
3 mlco [0.40595225] [0.74843653] [0.88595519] [0.19400767] 1 1 1
3 cs [0.1117949] [0.41284364] [0.88595519] [0.26290041] 2 2 2
4 scen [0.15738951] [0.54682627] [0.89401924] [0.30649028] 1 1 1
4 mlco [0.530488] [0.81879614] [0.89401924] [0.14417003] 1 1 1
4 cs [0.10193422] [0.45527543] [0.89401924] [0.29756964] 2 2 2
5 scen [0.23289145] [0.56400812] [0.90218582] [0.27346462] 1 1 1
5 mlco [0.44280526] [0.80275949] [0.90218582] [0.15851868] 1 1 1
5 cs [0.09072473] [0.43045445] [0.90218582] [0.28350782] 2 2 2
6 scen [0.20088963] [0.52298671] [0.92131241] [0.26125691] 1 1 1
6 mlco [0.68506597] [0.88621243] [0.92131241] [0.07906191] 1 1 1
6 cs [0.06035487] [0.43318322] [0.92131241] [0.2761722] 2 2 2
7 scen [0.17652737] [0.48733082] [0.92889245] [0.26094251] 1 1 1
7 mlco [0.59738418] [0.88078391] [0.92889245] [0.11131516] 1 1 1
7 cs [0.05034304] [0.40315363] [0.92889245] [0.27132531] 2 2 2
8 scen [0.17652737] [0.55815155] [0.93928777] [0.24418593] 1 1 1
8 mlco [0.56185613] [0.90453632] [0.93928777] [0.09201446] 1 1 1
8 cs [0.04045196] [0.46675558] [0.93928777] [0.26574865] 2 2 2
9 scen [0.16641506] [0.55297915] [0.94121292] [0.23495566] 1 1 1
9 mlco [0.530488] [0.9034545] [0.94121292] [0.10004736] 1 1 1
9 cs [0.03303344] [0.46697758] [0.94121292] [0.26392307] 2 2 2
10 scen [0.14928594] [0.5248227] [0.93836818] [0.23606981] 1 1 1
10 mlco [0.47751463] [0.86317255] [0.93836818] [0.1523894] 1 1 1
10 cs [0.02764604] [0.43169139] [0.93836818] [0.25634494] 2 2 2
11 scen [0.13224397] [0.51000396] [0.93573428] [0.24458403] 1 1 1
11 mlco [0.38282932] [0.80449361] [0.93573428] [0.20165409] 1 1 1
11 cs [0.02376931] [0.39717601] [0.93573428] [0.25148415] 2 2 2
12 scen [0.12116625] [0.51151563] [0.95156251] [0.2672405] 1 1 1
12 mlco [0.34234666] [0.80925369] [0.95156251] [0.22711859] 1 1 1
12 cs [0.03056912] [0.40248187] [0.95156251] [0.26908261] 2 2 2
13 scen [0.11629279] [0.49319215] [0.94661997] [0.26527459] 1 1 1
13 mlco [0.33071164] [0.88510441] [0.94661997] [0.15029952] 1 1 1
13 cs [0.02889342] [0.42079539] [0.94661997] [0.27210703] 2 2 2
14 scen [0.03361135] [0.46565447] [0.95469873] [0.27991709] 1 1 1
14 mlco [0.86016258] [0.93258813] [0.95469873] [0.03462779] 1 1 1
14 cs [0.02638058] [0.43324067] [0.95469873] [0.27397941] 2 2 2
15 scen [0.03090772] [0.45330783] [0.961646] [0.28009647] 1 1 1
15 mlco [0.84624815] [0.94931718] [0.961646] [0.02775933] 1 1 1
15 cs [0.02416802] [0.42447737] [0.961646] [0.27590525] 2 2 2
16 scen [0.03041835] [0.44202793] [0.96544326] [0.27898215] 1 1 1
16 mlco [0.8135197] [0.94268297] [0.96544326] [0.0439307] 1 1 1
16 cs [0.02229784] [0.4051148] [0.96544326] [0.27124226] 2 2 2
17 scen [0.02918605] [0.43077332] [0.96891381] [0.27800259] 1 1 1
17 mlco [0.82471617] [0.95647675] [0.96891381] [0.03536462] 1 1 1
17 cs [0.02130849] [0.4031421] [0.96891381] [0.27122333] 2 2 2
18 scen [0.02791384] [0.42208136] [0.96914247] [0.2750266] 1 1 1
18 mlco [0.81252847] [0.94833346] [0.96914247] [0.04741444] 1 1 1
18 cs [0.01990977] [0.38908737] [0.96914247] [0.26756288] 2 2 2
19 scen [0.01990977] [0.40969892] [0.97063579] [0.27579959] 1 1 1
19 mlco [0.96302093] [0.9658489] [0.97063579] [0.00239575] 1 1 1
19 cs [0.01844416] [0.38952834] [0.97063579] [0.27261269] 2 2 2
20 scen [0.01838532] [0.40043245] [0.97001694] [0.27419114] 1 1 1
20 mlco [0.95941225] [0.96492808] [0.97001694] [0.00424176] 1 1 1
20 cs [0.01712846] [0.37880077] [0.97001694] [0.27037116] 2 2 2
21 scen [0.01970553] [0.39146921] [0.96875708] [0.27249585] 1 1 1
21 mlco [0.96109125] [0.96598857] [0.96875708] [0.00262888] 1 1 1
21 cs [0.01625861] [0.36581134] [0.96875708] [0.26969507] 2 2 2
22 scen [0.01668221] [0.38290105] [0.9663991] [0.26986622] 1 1 1
22 mlco [0.95520738] [0.9633274] [0.9663991] [0.00410066] 1 1 1
22 cs [0.01564081] [0.36409675] [0.9663991] [0.26878069] 2 2 2
23 scen [0.01603244] [0.37709224] [0.96441343] [0.26919123] 1 1 1
23 mlco [0.94488579] [0.96185982] [0.96441343] [0.00459627] 1 1 1
23 cs [0.01506825] [0.35985133] [0.96441343] [0.26741092] 2 2 2
24 scen [0.01534919] [0.3705945] [0.96224205] [0.26844458] 1 1 1
24 mlco [0.93416926] [0.95570324] [0.96224205] [0.00650845] 1 1 1
24 cs [0.01446315] [0.3550484] [0.96224205] [0.26601353] 2 2 2
25 scen [0.01621287] [0.41496057] [0.96745466] [0.31073007] 1 1 1
25 mlco [0.93239995] [0.95571099] [0.96745466] [0.0090585] 1 1 1
25 cs [0.01390478] [0.37022516] [0.96745466] [0.28657672] 2 2 2
26 scen [0.01559848] [0.46076677] [0.96505446] [0.30195726] 1 1 1
26 mlco [0.91847173] [0.94110909] [0.96505446] [0.01062504] 1 1 1
26 cs [0.01338791] [0.39515281] [0.96505446] [0.29141962] 2 2 2
27 scen [0.01502895] [0.46607991] [0.96364502] [0.3134495] 1 1 1
27 mlco [0.91038373] [0.93544697] [0.96364502] [0.0130212] 1 1 1
27 cs [0.0129081] [0.40653275] [0.96364502] [0.28478293] 2 2 2
28 scen [0.01453612] [0.42548635] [0.95009035] [0.28178472] 1 1 1
28 mlco [0.8836971] [0.91715813] [0.95009035] [0.01632614] 1 1 1
28 cs [0.01240782] [0.40046969] [0.95009035] [0.27089433] 2 2 2
29 scen [0.01502895] [0.42356233] [0.96497581] [0.28331238] 1 1 1
29 mlco [0.87028206] [0.90590296] [0.96497581] [0.02397106] 1 1 1
29 cs [0.00779322] [0.38619027] [0.96497581] [0.26567316] 2 2 2
30 scen [0.01435507] [0.42056631] [0.955613] [0.28266386] 1 1 1
30 mlco [0.85956423] [0.89185926] [0.955613] [0.0215557] 1 1 1
30 cs [0.00739327] [0.38530432] [0.955613] [0.26425755] 2 2 2
31 scen [0.01377186] [0.41683121] [0.94249672] [0.28115496] 1 1 1
31 mlco [0.8483434] [0.87636726] [0.94249672] [0.02006534] 1 1 1
31 cs [0.00704958] [0.38360126] [0.94249672] [0.26262278] 2 2 2
32 scen [0.01317367] [0.41457416] [0.93290233] [0.28069088] 1 1 1
32 mlco [0.84000581] [0.87197954] [0.93290233] [0.02465118] 1 1 1
32 cs [0.00671287] [0.37685207] [0.93290233] [0.26165921] 2 2 2
33 scen [0.01268086] [0.41208544] [0.92589688] [0.28025124] 1 1 1
33 mlco [0.83254712] [0.8590174] [0.92589688] [0.01995741] 1 1 1
33 cs [0.00974343] [0.37866166] [0.92589688] [0.26149271] 2 2 2
34 scen [0.01222358] [0.41071686] [0.91839148] [0.28002889] 1 1 1
34 mlco [0.82694999] [0.85673753] [0.91839148] [0.02560652] 1 1 1
34 cs [0.00936346] [0.37824051] [0.91839148] [0.26211579] 2 2 2
35 scen [0.01175002] [0.36726485] [0.93739564] [0.28504601] 1 1 1
35 mlco [0.84511051] [0.87280537] [0.93739564] [0.02673374] 1 1 1
35 cs [0.00898392] [0.35342759] [0.93739564] [0.28921985] 2 2 2
36 scen [0.01131178] [0.44445503] [0.93705445] [0.27757811] 1 1 1
36 mlco [0.84590272] [0.86761693] [0.93705445] [0.02047017] 1 1 1
36 cs [0.00863394] [0.3690468] [0.93705445] [0.28267034] 2 2 2
37 scen [0.01090506] [0.48966773] [0.95713211] [0.26605965] 1 1 1
37 mlco [0.86786326] [0.89020115] [0.95713211] [0.01977105] 1 1 1
37 cs [0.00831021] [0.39917885] [0.95713211] [0.28440274] 2 2 2
38 scen [0.01052657] [0.53641475] [0.97213255] [0.31017436] 1 1 1
38 mlco [0.88352542] [0.91266711] [0.97213255] [0.02629443] 1 1 1
38 cs [0.00798767] [0.4391918] [0.97213255] [0.29614276] 2 2 2
39 scen [0.01015554] [0.54196557] [0.97598348] [0.31348759] 1 1 1
39 mlco [0.89809796] [0.92584053] [0.97598348] [0.02319248] 1 1 1
39 cs [0.00768923] [0.44932433] [0.97598348] [0.30228067] 2 2 2
40 scen [0.00979311] [0.5210877] [0.97952034] [0.29560345] 1 1 1
40 mlco [0.91124676] [0.93731492] [0.97952034] [0.01983627] 1 1 1
40 cs [0.00739327] [0.4545293] [0.97952034] [0.30861847] 2 2 2
41 scen [0.32182935] [0.70650931] [0.98119688] [0.24984869] 1 1 1
41 mlco [0.91745232] [0.94275366] [0.98119688] [0.01862704] 1 1 1
41 cs [0.05497799] [0.59254341] [0.98119688] [0.24187595] 2 2 2
42 scen [0.56056234] [0.92314131] [0.98273099] [0.14225767] 1 1 1
42 mlco [0.96612301] [0.97612743] [0.98273099] [0.0056759] 1 1 1
42 cs [0.31613122] [0.86086235] [0.98273099] [0.17080027] 2 2 2
43 scen [0.60274611] [0.94161896] [0.98358693] [0.09776093] 1 1 1
43 mlco [0.98215118] [0.98280343] [0.98358693] [0.00049985] 1 1 1
43 cs [0.29272947] [0.89359795] [0.98358693] [0.15202824] 2 2 2
44 scen [0.67523131] [0.929415] [0.98421415] [0.08655687] 1 1 1
44 mlco [0.97898819] [0.98288698] [0.98421415] [0.00121634] 1 1 1
44 cs [0.26703202] [0.88442494] [0.98421415] [0.14822114] 2 2 2
45 scen [0.73190465] [0.89580505] [0.97732649] [0.07326781] 1 1 1
45 mlco [0.92930985] [0.95502902] [0.97732649] [0.01276733] 1 1 1
45 cs [0.24426581] [0.83610565] [0.97732649] [0.14488598] 2 2 2
46 scen [0.88953699] [0.92665198] [0.95081281] [0.01729799] 1 1 1
46 mlco [0.90306913] [0.92780927] [0.95081281] [0.01287089] 1 1 1
46 cs [0.75650055] [0.89092787] [0.95081281] [0.03467792] 2 2 2
47 scen [0.8489215] [0.92202749] [0.98156361] [0.03139957] 1 1 1
47 mlco [0.91527888] [0.94911984] [0.98156361] [0.02074708] 1 1 1
47 cs [0.73361299] [0.89356896] [0.98156361] [0.04614472] 2 2 2
48 scen [0.90944702] [0.93392487] [0.98367588] [0.02738131] 1 1 1
48 mlco [0.90847433] [0.94332172] [0.98367588] [0.02322059] 1 1 1
48 cs [0.80446653] [0.92019644] [0.98367588] [0.03031236] 2 2 2
49 scen [0.84367142] [0.92063665] [0.98192203] [0.04004019] 1 1 1
49 mlco [0.89443217] [0.9297737] [0.98192203] [0.02611619] 1 1 1
49 cs [0.75664295] [0.89043913] [0.98192203] [0.04008608] 2 2 2
50 scen [0.80831813] [0.93379075] [0.98389172] [0.0498924] 1 1 1
50 mlco [0.96887208] [0.97885027] [0.98389172] [0.00410537] 1 1 1
50 cs [0.66638312] [0.8915569] [0.98389172] [0.07396943] 2 2 2
51 scen [0.87638091] [0.94893083] [0.98476932] [0.04074105] 1 1 1
51 mlco [0.9674507] [0.97920832] [0.98476932] [0.00498718] 1 1 1
51 cs [0.79420446] [0.92050447] [0.98476932] [0.04451147] 2 2 2
52 scen [0.90736402] [0.96532593] [0.98515695] [0.03011228] 1 1 1
52 mlco [0.96929585] [0.97975968] [0.98515695] [0.00446666] 1 1 1
52 cs [0.79922313] [0.93087597] [0.98515695] [0.03655173] 2 2 2
53 scen [0.87997852] [0.96727084] [0.9846911] [0.03114015] 1 1 1
53 mlco [0.94524537] [0.95951525] [0.9846911] [0.01315056] 1 1 1
53 cs [0.86188678] [0.9230622] [0.9846911] [0.03281955] 2 2 2
54 scen [0.93834738] [0.98058358] [0.98492038] [0.01129657] 1 1 1
54 mlco [0.98182936] [0.98298111] [0.98492038] [0.00094062] 1 1 1
54 cs [0.89896064] [0.95702433] [0.98492038] [0.02717098] 2 2 2
55 scen [0.98218262] [0.98402211] [0.98572741] [0.00138155] 1 1 1
55 mlco [0.98226709] [0.98421973] [0.98572741] [0.00106559] 1 1 1
55 cs [0.92418537] [0.96931237] [0.98572741] [0.01579334] 2 2 2
56 scen [0.97809657] [0.98394311] [0.98621746] [0.00213096] 1 1 1
56 mlco [0.98223082] [0.98498014] [0.98621746] [0.00147118] 1 1 1
56 cs [0.92137372] [0.96852369] [0.98621746] [0.01639857] 2 2 2
57 scen [0.97690329] [0.98206847] [0.98627594] [0.00315076] 1 1 1
57 mlco [0.97952101] [0.98395806] [0.98627594] [0.00186953] 1 1 1
57 cs [0.90845137] [0.96520355] [0.98627594] [0.01958849] 2 2 2
58 scen [0.94580868] [0.9781251] [0.98663306] [0.01129777] 1 1 1
58 mlco [0.96334471] [0.98237071] [0.98663306] [0.00745654] 1 1 1
58 cs [0.87977412] [0.95670941] [0.98663306] [0.02749673] 2 2 2
59 scen [0.96217369] [0.98122138] [0.98659653] [0.0081745] 1 1 1
59 mlco [0.98284712] [0.98519438] [0.98659653] [0.00110629] 1 1 1
59 cs [0.90352915] [0.96596326] [0.98659653] [0.02392067] 2 2 2
60 scen [0.94409646] [0.97715306] [0.9870261] [0.01422771] 1 1 1
60 mlco [0.98325157] [0.98563345] [0.9870261] [0.00102225] 1 1 1
60 cs [0.87849452] [0.95643409] [0.9870261] [0.03089736] 2 2 2
61 scen [0.97373523] [0.98506449] [0.98691841] [0.00344786] 1 1 1
61 mlco [0.98321097] [0.98584673] [0.98691841] [0.00112832] 1 1 1
61 cs [0.90748741] [0.97488553] [0.98691841] [0.01705194] 2 2 2
62 scen [0.98353397] [0.9868024] [0.98737853] [0.00088353] 1 1 1
62 mlco [0.98341469] [0.98631086] [0.98737853] [0.00126832] 1 1 1
62 cs [0.92339324] [0.98066669] [0.98737853] [0.01209319] 2 2 2
63 scen [0.98703273] [0.98728473] [0.98746295] [0.00011977] 1 1 1
63 mlco [0.98338958] [0.98627746] [0.98746295] [0.00126276] 1 1 1
63 cs [0.95045826] [0.9820561] [0.98746295] [0.00925277] 2 2 2
64 scen [0.86461749] [0.97884926] [0.98763003] [0.03054958] 1 1 1
64 mlco [0.98401948] [0.98638783] [0.98763003] [0.0010183] 1 1 1
64 cs [0.84515426] [0.96839374] [0.98763003] [0.03943688] 2 2 2
65 scen [0.98701967] [0.98746309] [0.98772518] [0.00031817] 1 1 1
65 mlco [0.98601534] [0.98729724] [0.98772518] [0.00044895] 1 1 1
65 cs [0.93530033] [0.98167904] [0.98772518] [0.01110297] 2 2 2
66 scen [0.98743663] [0.98766766] [0.98784068] [0.00013637] 1 1 1
66 mlco [0.98055602] [0.98711572] [0.98784068] [0.00177348] 1 1 1
66 cs [0.9001527] [0.97981565] [0.98784068] [0.01479672] 2 2 2
67 scen [0.98699812] [0.98763333] [0.98797783] [0.00035054] 1 1 1
67 mlco [0.9826589] [0.98716804] [0.98797783] [0.0012767] 1 1 1
67 cs [0.91101395] [0.98092115] [0.98797783] [0.01217074] 2 2 2
68 scen [0.98665308] [0.98777061] [0.98813731] [0.00042437] 1 1 1
68 mlco [0.98440513] [0.98718121] [0.98813731] [0.00101988] 1 1 1
68 cs [0.92018544] [0.9812495] [0.98813731] [0.00988669] 2 2 2
69 scen [0.98581555] [0.98752391] [0.98826518] [0.00080613] 1 1 1
69 mlco [0.98460091] [0.98712425] [0.98826518] [0.00115631] 1 1 1
69 cs [0.92941668] [0.98091224] [0.98826518] [0.00816339] 2 2 2
70 scen [0.98617597] [0.98726948] [0.98841491] [0.00064702] 1 1 1
70 mlco [0.98402359] [0.98680327] [0.98841491] [0.00128052] 1 1 1
70 cs [0.9366998] [0.98068432] [0.98841491] [0.00760167] 2 2 2
71 scen [0.93517782] [0.98382393] [0.98834607] [0.01308369] 1 1 1
71 mlco [0.98445644] [0.98725717] [0.98834607] [0.00113783] 1 1 1
71 cs [0.83089601] [0.97343547] [0.98834607] [0.02218763] 2 2 2
72 scen [0.93612386] [0.98407703] [0.98834658] [0.01289798] 1 1 1
72 mlco [0.9848039] [0.98745609] [0.98834658] [0.0010581] 1 1 1
72 cs [0.83457901] [0.97401758] [0.98834658] [0.02180731] 2 2 2
73 scen [0.93702573] [0.98434671] [0.98852987] [0.01273106] 1 1 1
73 mlco [0.9851378] [0.98767007] [0.98852987] [0.00100561] 1 1 1
73 cs [0.83807833] [0.97457063] [0.98852987] [0.02145145] 2 2 2
74 scen [0.93800991] [0.9846292] [0.98879271] [0.0125464] 1 1 1
74 mlco [0.98643759] [0.98807993] [0.98879271] [0.00081002] 1 1 1
74 cs [0.84173498] [0.97545074] [0.98879271] [0.02064777] 2 2 2
75 scen [0.94094592] [0.98408758] [0.98875918] [0.01189487] 1 1 1
75 mlco [0.98638753] [0.98806045] [0.98875918] [0.00079365] 1 1 1
75 cs [0.84753287] [0.97540258] [0.98875918] [0.01910891] 2 2 2
76 scen [0.94537417] [0.98376174] [0.98907965] [0.01088863] 1 1 1
76 mlco [0.98605741] [0.98802285] [0.98907965] [0.0009653] 1 1 1
76 cs [0.85494943] [0.97559913] [0.98907965] [0.01719869] 2 2 2
77 scen [0.94957998] [0.98403821] [0.98932134] [0.01019574] 1 1 1
77 mlco [0.98574508] [0.98801161] [0.98932134] [0.00123042] 1 1 1
77 cs [0.86196318] [0.97579849] [0.98932134] [0.01607826] 2 2 2
78 scen [0.95755323] [0.98387837] [0.98927046] [0.0089259] 1 1 1
78 mlco [0.98633747] [0.98799752] [0.98927046] [0.00102692] 1 1 1
78 cs [0.87235363] [0.97548333] [0.98927046] [0.01607611] 2 2 2
79 scen [0.96004767] [0.98349328] [0.98883769] [0.00931442] 1 1 1
79 mlco [0.98617677] [0.98794074] [0.98883769] [0.0008576] 1 1 1
79 cs [0.94111301] [0.97471641] [0.98883769] [0.01337394] 2 2 2
80 scen [0.94904816] [0.98309871] [0.98912309] [0.01173559] 1 1 1
80 mlco [0.98623794] [0.98780759] [0.98912309] [0.00114776] 1 1 1
80 cs [0.93247553] [0.96997958] [0.98912309] [0.01615504] 2 2 2
81 scen [0.93847981] [0.98138475] [0.98945883] [0.01525175] 1 1 1
81 mlco [0.98498316] [0.98837793] [0.98945883] [0.0011882] 1 1 1
81 cs [0.92237918] [0.96451097] [0.98945883] [0.01992483] 2 2 2
82 scen [0.92916652] [0.97793951] [0.98956323] [0.01808314] 1 1 1
82 mlco [0.98563739] [0.98845639] [0.98956323] [0.00107689] 1 1 1
82 cs [0.91418524] [0.95968003] [0.98956323] [0.02359916] 2 2 2
83 scen [0.91666965] [0.97285614] [0.9894049] [0.02098305] 1 1 1
83 mlco [0.98712642] [0.98824384] [0.9894049] [0.00055267] 1 1 1
83 cs [0.90727779] [0.95631318] [0.9894049] [0.0263019] 2 2 2
84 scen [0.90773651] [0.96273151] [0.98960314] [0.02373129] 1 1 1
84 mlco [0.9864907] [0.98848519] [0.98960314] [0.0010512] 1 1 1
84 cs [0.89782034] [0.94192381] [0.98960314] [0.02909775] 2 2 2
85 scen [0.8932321] [0.95276725] [0.98824252] [0.02735553] 1 1 1
85 mlco [0.97654326] [0.98529283] [0.98824252] [0.00246842] 1 1 1
85 cs [0.8651981] [0.93036429] [0.98824252] [0.03063311] 2 2 2
86 scen [0.87445011] [0.93450444] [0.98627431] [0.02884172] 1 1 1
86 mlco [0.96233617] [0.97181079] [0.98627431] [0.00803766] 1 1 1
86 cs [0.82182059] [0.91318803] [0.98627431] [0.03160463] 2 2 2
87 scen [0.86005951] [0.91926287] [0.98932272] [0.03084041] 1 1 1
87 mlco [0.94785066] [0.95917457] [0.98932272] [0.01234839] 1 1 1
87 cs [0.80619396] [0.89959739] [0.98932272] [0.03161437] 2 2 2
88 scen [0.84084265] [0.90125595] [0.98823391] [0.03456355] 1 1 1
88 mlco [0.92370394] [0.95312883] [0.98823391] [0.02260342] 1 1 1
88 cs [0.75604367] [0.87929273] [0.98823391] [0.04037577] 2 2 2
89 scen [0.81901133] [0.88496628] [0.98582876] [0.03823113] 1 1 1
89 mlco [0.91833234] [0.94750706] [0.98582876] [0.02427776] 1 1 1
89 cs [0.72586688] [0.86107439] [0.98582876] [0.0467012] 2 2 2
90 scen [0.80898976] [0.86612344] [0.96775425] [0.03685094] 1 1 1
90 mlco [0.89016473] [0.91768637] [0.96775425] [0.023577] 1 1 1
90 cs [0.61863042] [0.82928811] [0.96775425] [0.05702532] 2 2 2
91 scen [0.78796498] [0.85114386] [0.94881913] [0.03639211] 1 1 1
91 mlco [0.78690065] [0.90337024] [0.94881913] [0.03874348] 1 1 1
91 cs [0.501038] [0.80552674] [0.94881913] [0.06858039] 2 2 2
92 scen [0.77687928] [0.8392726] [0.94069659] [0.03668664] 1 1 1
92 mlco [0.87050487] [0.90737991] [0.94069659] [0.02493801] 1 1 1
92 cs [0.6832077] [0.80759829] [0.94069659] [0.0518818] 2 2 2
93 scen [0.76478144] [0.82577164] [0.92371086] [0.03554349] 1 1 1
93 mlco [0.85662942] [0.89918475] [0.92371086] [0.02559893] 1 1 1
93 cs [0.67094827] [0.78694607] [0.92371086] [0.05630376] 2 2 2
94 scen [0.75048452] [0.80971562] [0.90374015] [0.03409316] 1 1 1
94 mlco [0.83075682] [0.87009616] [0.90374015] [0.02554737] 1 1 1
94 cs [0.58049207] [0.76281585] [0.90374015] [0.06038146] 2 2 2
95 scen [0.73916883] [0.7971197] [0.88801227] [0.03307734] 1 1 1
95 mlco [0.82728132] [0.85544188] [0.88801227] [0.0236312] 1 1 1
95 cs [0.6409912] [0.75192759] [0.88801227] [0.05799152] 2 2 2
96 scen [0.72938613] [0.78624319] [0.87447082] [0.03221561] 1 1 1
96 mlco [0.81608195] [0.84096377] [0.87447082] [0.02196094] 1 1 1
96 cs [0.64145381] [0.76244759] [0.87447082] [0.04505884] 2 2 2
97 scen [0.71985276] [0.77561598] [0.86132404] [0.03135716] 1 1 1
97 mlco [0.79180919] [0.83590342] [0.86132404] [0.02327517] 1 1 1
97 cs [0.56734278] [0.74088095] [0.86132404] [0.05318666] 2 2 2
98 scen [0.70713458] [0.76181957] [0.84393001] [0.03065522] 1 1 1
98 mlco [0.44618824] [0.79399865] [0.84393001] [0.09748883] 1 1 1
98 cs [0.05738829] [0.68124642] [0.84393001] [0.12712867] 2 2 2
99 scen [0.70368653] [0.75047418] [0.8336203] [0.0295743] 1 1 1
99 mlco [0.774086] [0.81158746] [0.8336203] [0.01997002] 1 1 1
99 cs [0.61467098] [0.72126893] [0.8336203] [0.04427711] 2 2 2
As shown, the max fitness increases consistently and then drops a bit towards the end. I will think about it more and write my thoughts on it in later comments. But in short, it seems the algorithm is going towards the higher fitness value. Nevertheless, I am not sure why the maximum fitness drops even a bit given our update archive strategy. I will try this with the alternative strategies that always keep the best individual and report the results here.
Before moving on to evaluations, I ran a test run for multiple hours and discovered inconsistent behavior in the progress of fitness values. In that run, the fitness values were decreasing as opposed to increasing.
To address understand the reason for the unexpected behavior of the search, I ran numerous benchmark tests with different population sizes and number of generations with multiple strategies to update the population archives (iCCEA vs. elitist or best+random approaches).
After our discussing the results, we reached the following conclusions:
The lower the populations' size is, the higher the chances of observing a falling or oscillatory trend in max or average fitness of the populations are. 5 turned out to be a number that would not work. Whereas, with a size of 15, undesirable behaviors were rarely observed.
By increasing the weight of the term related to the confidence interval in the fitness function, the behavior improved. A rough explanation for why such behavior exists can be contributed to the heavy reliance of the fitness function on the number of observations per region. If the number of samples per region are too low, there is a higher chance that their fitness would change drastically given new samples. Thus, there is a higher chance that the search algorithm might move towards regions that it perceives to have a high fitness value while it is moving towards a lower fitness landscape (which becomes apparent after the new samples in the future generations are observed).
Conclusion (1): our search algorithm would most likely not work with a population size of 5 in 10 generations of search. We would have to bring the population size as close to 15 as possible in the least.
Conclusion (2): due to the population size increase, and the cost of simulations, the iCCEA version of updating population archives will not be feasible given our resource constraints since it forces the simulation of every possible complete solution per generation. Instead, other conventional strategies of updating population archives are suggested to be used, such as elitist or best+random strategies.
Conclusion (3): again, due to the increased population size, parallelization has become necessary for running the evaluations.
I was debugging the code using the MTQ problem.
There are several redundant individuals in a population, mainly because mutation_rate was not 1 and we simply append the archived individuals to the next population. Since we already maintain the same individuals for the next generation using the population archive, the remaining seats should be filled with mutated individuals, meaning that we should use mutation_rate = 1.
So, I used the following parameter values:
# Search hyperparameters
scenario_population_size = 5 # Size of the scenario population
mlco_population_size = 5 # Size of the MLC output population
min_distance = 0.5 # Minimum distance between members of an archive
region_radius = 0.05 # The radius of the region for fitness evaluations
number_of_generations = 40
random_seed = None
max_num_evals = 51200
# Evolution hyperparameters
tournament_selection = 2
crossover_probability = 0.5
guassian_mutation_mean = 0
guassian_mutation_std = 0.1
guassian_mutation_probability = 1
integer_mutation_probability = 1
bitflip_mutation_probability = 1
After this, we either stuck at a certain fitness score or find a good solution. The fitness score does not significantly decrease even when the population size is 5.
For the stuck, I think I know the reason. It's simply because, for each individual in a population, the number of collaborations is too small (i.e., only 5 since there are at most 5 individuals in the collaboration population), and therefore it does not result in a good individual. Furthermore, since the two populations evolve together based on insufficient collaboration, any of the individuals does not evolve stably. I think this is an inherent limitation of CCEAs.
To conclude, using this commit 4bd1086600ebca47ab2aaf5a3eeba3c46f282932, we can at least overcome the problem of significantly decreasing fitness values. And it seems clear that the small population size does not work for a complex problem. How to improve the performance even for a relatively small population size is still an open question, but I guess the fitness function has no issue at least. We may just need to fine-tune hyperparameters.
The MTQ benchmark problem will act as the first step. To run the benchmark problem simply run the following command:
python3 run_iccea_mtq.py
To modify the search parameters you can change them in
benchmark/mtq/search_config.py
.To modify the boundary, you can change the value at the end of the
joint_fitness_mtq()
function located inbenchmark/mtq/problem.py
. Kindly note that the joint fitness function does not return the actual joint fitness value in the updated version of the ICCEA, rather it returns the value of thesafety_req_value
attribute of the complete solution.