SepShr / MLCSHE

This repo houses the ML-Component Systemic Hazard Envelope project, or MILSCHE (pronounced /'mɪlʃ/).
MIT License
3 stars 0 forks source link

Sanity Checking the Boundary Seeking ICCEA #36

Closed SepShr closed 2 years ago

SepShr commented 2 years ago

The MTQ benchmark problem will act as the first step. To run the benchmark problem simply run the following command: python3 run_iccea_mtq.py

To modify the search parameters you can change them in benchmark/mtq/search_config.py.

To modify the boundary, you can change the value at the end of the joint_fitness_mtq() function located in benchmark/mtq/problem.py. Kindly note that the joint fitness function does not return the actual joint fitness value in the updated version of the ICCEA, rather it returns the value of the safety_req_value attribute of the complete solution.

donghwan-shin commented 2 years ago

What I did

So, I found three different issues, addressed in the following commits:

4af1f3b33ed3875b7d504260e12c5fc10aaf85dd

This is to fix a bug in calculating the safety_req_value value of each complete solution. Previously, the value was zero for all complete solutions since the map() function you used did not properly save the results in the corresponding field.

698f59565f10d749a0d1b7b5aa8878c310c53727

This is to reduce the "radius" (i.e., max_dist) of a cs_region; since 0.5 was too large, the search does not make fine-grained movements and is easily stuck in some regions.

eebc72f9ff57df4a77e39e0e291849e5d2a6352f

This is to make the MTQ problem "solvable". Previously, use used h2= 150 but s_2 = 1.0/32.0, meaning that it is very challenging to find the peak of h2 on a very small surface (1/32). So, I simply swapped h1 and h2 to make the problem easier to solve.

New result and its interpretation

This is the full log: 20220609_160328_CCEA_MTQ.log

As you can see, we have reached the fitness score of 0.9735897570338863.

Using the final solution x, y = [0.5853844653731718], [0.894080760273642], the function compute_safety_req_value() returns -21.786309582260785, which is quite close to 0.

In fact, on different runs, I found x, y = [0.6182038195955862], [0.8721837981696042], resulting the return value of compute_safety_req_value() only 1.5513294434849598.

Since the positive return value of compute_safety_req_value() indicates "safe" and the negative one indicates "unsafe", the return value of zero indicates the "boundary". Therefore, we can say that the algorithm successfully found a complete solution (x, y) that lies near the boundary.

To conclude, the sanity check is passed, but it turned out that max_dist should be carefully tuned for a given problem

SepShr commented 2 years ago

Hi, I managed to get the new setup working on Pylot+Carla. As you mentioned, the size of the region for joint fitness evaluations is important. The general behavior of the algorithm seems to be reasonable and it follows the general fitness improvement behavior.

What I did

To make the algorithm work with Pylot, aside from mirroring the changes in MTQ to Pylot's problem files, the returned values from the simulation had to be modified in the following manner: previously the values varied between 0 and 1, where 1 corresponded to no deviation, whereas 0 meant violation of the safety limit. To ensure the sign changes in case of violation, -1 will be returned instead of 1, but the rest remains the same. For instance, it does not matter how close to violation the complete solution is, as long as it has not violated the safety limit, it would have a positive value. In case of violation, we do not measure how much violation occurs and the value of -1 is returned. The reason for this decision is due to nature of the safety violation. In rough terms, if a safety violation occurs, it is game over and measuring how much violation has happened is not reasonable. However, violations can be differentiated based on their severity, for instance, crashing into a person is much more severe than crashing into a lamp post. However, for a single safety requirement, all violations can be seen as equal. Nevertheless, to improve our search algorithm, we might want to use a threshold to count a value as a violation rather than only focusing on 0.

Here is a log of a test run: 20220613_171322_CCEA.log

The attached log is for the test run No. 5. In this test run, interestingly, a bug with the mutate_mlco() was discovered. After breeding, the following mlco was created: mlco=[[[0, 0, 0, 0.0, 50.0, 0.0, 50.0, 0.0, 50.0, 0.0, 50.0], [1, 0, 0, 0.0, 50.0, 0.0, 50.0, 0.0, 50.0, 0.101, 50.101]]. Such an mlco would lead to a ZeroDivisionError, since the start and end times are 0. Hopefully, commit e8b3ef43f54d829cfabc7f8eae76712ca6c879f9 addresses this issue at ln.201 of problem_utils.py. Related to this issue, I realized that the test_mutate_mlco() unit test is flakey, for which issue #40 is opened.

Conclusion

We can conclude the setup is reasonably working. However, due to the importance of the effort, your review of the updated code is much appreciated. To review the code, pull the latest commit and run python3 run_iccea.py. Kindly note that you can modify the search parameters in ./search_config.py. If your review is satisfactory, then the last step before focusing on evaluations would be to parallelize the simulations, please see issue #39 .

donghwan-shin commented 2 years ago

In case of violation, we do not measure how much violation occurs and the value of -1 is returned. The reason for this decision is due to nature of the safety violation. In rough terms, if a safety violation occurs, it is game over and measuring how much violation has happened is not reasonable.

Okay, sounds reasonable. Also, since our current fitness evaluation only counts for the number of safe and unsafe complete solutions, it is enough to distinguish between safe and unsafe complete solutions without considering the degree of safety violations.

We can conclude the setup is reasonably working. However, due to the importance of the effort, your review of the updated code is much appreciated.

Okay, I will review the commits now. Thanks.

donghwan-shin commented 2 years ago

I made some improvements in mutate_time() and test_mutate_time()` in this commit: 66fa7881c12eb2212f4e514e4a41be3eebf82fed.

I couldn't look at the other code, but I can do it tomorrow morning if you want. Otherwise, feel free to go ahead if you already feel it's good to try to run. :-)

SepShr commented 2 years ago

As you know, I conducted a test run of around 31 hours. The results are, well, weird. The populations had a size of 5 each with the min_distance being 0.5. The number of generations was 15 and the total number of evaluations was 427 simulations.

The detailed log of the run can be accessed here: 20220614_124948_CCEA.log.

The terminal output of the run is also available which provides additional information on duration of the run (I just realized that this can be simply added to the log :smile:. terminal_output.log

Finally, the statistical information on the fitness value for throughout the generations is available here: 20220614_124948_logbook.log. I have printed the logbook in the following.

                                         fitness                                       size       
                ------------------------------------------------------------    -------------------
gen     type    min             avg             max             std             min     avg     max
0       scen    [0.6234189]     [0.68703697]    [0.7341396]     [0.03716327]    7       7       7  
0       mlco    [0.67981879]    [0.70762463]    [0.7341396]     [0.01760144]    2       2       2  
0       cs      [0.44034454]    [0.63897178]    [0.7341396]     [0.0728814]     2       2       2  
1       scen    [0.72929305]    [0.74862936]    [0.7995379]     [0.02635198]    7       7       7  
1       mlco    [0.69340341]    [0.73918904]    [0.7995379]     [0.03502136]    2       2       2  
1       cs      [0.56928779]    [0.68733723]    [0.7995379]     [0.0486562]     2       2       2  
2       scen    [0.66852081]    [0.71785378]    [0.82932162]    [0.05820451]    7       7       7  
2       mlco    [0.62530197]    [0.76377781]    [0.82932162]    [0.08146579]    2       2       2  
2       cs      [0.55732867]    [0.66228839]    [0.82932162]    [0.07133759]    2       2       2  
3       scen    [0.69144882]    [0.73377064]    [0.83057685]    [0.04992332]    7       7       7  
3       mlco    [0.65040406]    [0.7869763]     [0.83057685]    [0.06846054]    2       2       2  
3       cs      [0.58519065]    [0.69083291]    [0.83057685]    [0.07632528]    2       2       2  
4       scen    [0.53940695]    [0.60849553]    [0.72236995]    [0.0690791]     7       7       7  
4       mlco    [0.5987755]     [0.64800697]    [0.72236995]    [0.04098705]    2       2       2  
4       cs      [0.50767005]    [0.56693144]    [0.72236995]    [0.05131364]    2       2       2  
5       scen    [0.47575206]    [0.55725396]    [0.69194258]    [0.07840432]    7       7       7  
5       mlco    [0.53404053]    [0.60616453]    [0.69194258]    [0.06510572]    2       2       2  
5       cs      [0.44280526]    [0.5130585]     [0.69194258]    [0.06298238]    2       2       2  
6       scen    [0.44901181]    [0.56741654]    [0.64454244]    [0.066772]      7       7       7  
6       mlco    [0.57303726]    [0.60352767]    [0.64454244]    [0.02468984]    2       2       2  
6       cs      [0.44901181]    [0.55564525]    [0.64454244]    [0.05145507]    2       2       2  
7       scen    [0.42800984]    [0.46625675]    [0.50842683]    [0.03011656]    7       7       7  
7       mlco    [0.42494099]    [0.47881817]    [0.50842683]    [0.02940178]    2       2       2  
7       cs      [0.38592126]    [0.44488609]    [0.50842683]    [0.03412655]    2       2       2  
8       scen    [0.35161408]    [0.42322096]    [0.52575649]    [0.06955965]    7       7       7  
8       mlco    [0.43690625]    [0.4806653]     [0.52575649]    [0.02818666]    2       2       2  
8       cs      [0.34561999]    [0.42621187]    [0.52575649]    [0.06030999]    2       2       2  
9       scen    [0.37986032]    [0.47973068]    [0.53569025]    [0.05711809]    7       7       7  
9       mlco    [0.47005378]    [0.50323696]    [0.53569025]    [0.02646158]    2       2       2  
9       cs      [0.37986032]    [0.48600296]    [0.53569025]    [0.04091374]    2       2       2  
10      scen    [0.53322187]    [0.5766821]     [0.60302637]    [0.02513307]    7       7       7  
10      mlco    [0.53813287]    [0.57034537]    [0.60302637]    [0.02241255]    2       2       2  
10      cs      [0.48848674]    [0.54284181]    [0.60302637]    [0.03220183]    2       2       2  
11      scen    [0.54075197]    [0.56935929]    [0.58995124]    [0.01665187]    7       7       7  
11      mlco    [0.54919638]    [0.5694217]     [0.58995124]    [0.01402076]    2       2       2  
11      cs      [0.50059416]    [0.55574353]    [0.58995124]    [0.02271427]    2       2       2  
12      scen    [0.51541146]    [0.52568993]    [0.54021618]    [0.00847303]    7       7       7  
12      mlco    [0.52316174]    [0.52916383]    [0.54021618]    [0.00581389]    2       2       2  
12      cs      [0.48564721]    [0.5145295]     [0.54021618]    [0.01349907]    2       2       2  
13      scen    [0.49855108]    [0.55390467]    [0.57645644]    [0.02810277]    7       7       7  
13      mlco    [0.55178689]    [0.56853928]    [0.57645644]    [0.00905047]    2       2       2  
13      cs      [0.46909499]    [0.54431]       [0.57645644]    [0.03087536]    2       2       2  
14      scen    [0.5244632]     [0.53967216]    [0.55956432]    [0.01236508]    7       7       7  
14      mlco    [0.54255052]    [0.55127563]    [0.55956432]    [0.00687808]    2       2       2  
14      cs      [0.43050504]    [0.5219403]     [0.55956432]    [0.03245033]    2       2       2  

The curious behavior in the fitness values does not match our expectation (it does not increase). However, in the logs, the cs with highest fitness is recorded as having the values of 0.83. There might be issues with the logbook that I will investigate and report the results on. However, the algorithm has gotten stuck on a local optima from the 4rd generation and did not improve.

I will add more to the discussion in later comments.

SepShr commented 2 years ago

Hi, I ran an MTQ for 100 generations. The log can be found here: 20220616_160610_CCEA_MTQ.log

The fitness stats per generation are as follows:

                                          fitness                                       size       
                ------------------------------------------------------------    -------------------
gen     type    min             avg             max             std             min     avg     max
0       scen    [0.46924696]    [0.57235399]    [0.71521253]    [0.08066848]    1       1       1  
0       mlco    [0.38137794]    [0.48082553]    [0.71521253]    [0.07585265]    1       1       1  
0       cs      [0.16177757]    [0.34403937]    [0.71521253]    [0.12035314]    2       2       2  
1       scen    [0.43314687]    [0.57673013]    [0.71471573]    [0.11435636]    1       1       1  
1       mlco    [0.40595225]    [0.52934783]    [0.71471573]    [0.08558928]    1       1       1  
1       cs      [0.13533558]    [0.36546781]    [0.71471573]    [0.1391729]     2       2       2  
2       scen    [0.30494452]    [0.58984317]    [0.85199279]    [0.2268949]     1       1       1  
2       mlco    [0.40595225]    [0.63791735]    [0.85199279]    [0.19981765]    1       1       1  
2       cs      [0.12116625]    [0.38101467]    [0.85199279]    [0.21774756]    2       2       2  
3       scen    [0.2766141]     [0.58976771]    [0.88595519]    [0.26009821]    1       1       1  
3       mlco    [0.40595225]    [0.74843653]    [0.88595519]    [0.19400767]    1       1       1  
3       cs      [0.1117949]     [0.41284364]    [0.88595519]    [0.26290041]    2       2       2  
4       scen    [0.15738951]    [0.54682627]    [0.89401924]    [0.30649028]    1       1       1  
4       mlco    [0.530488]      [0.81879614]    [0.89401924]    [0.14417003]    1       1       1  
4       cs      [0.10193422]    [0.45527543]    [0.89401924]    [0.29756964]    2       2       2  
5       scen    [0.23289145]    [0.56400812]    [0.90218582]    [0.27346462]    1       1       1  
5       mlco    [0.44280526]    [0.80275949]    [0.90218582]    [0.15851868]    1       1       1  
5       cs      [0.09072473]    [0.43045445]    [0.90218582]    [0.28350782]    2       2       2  
6       scen    [0.20088963]    [0.52298671]    [0.92131241]    [0.26125691]    1       1       1  
6       mlco    [0.68506597]    [0.88621243]    [0.92131241]    [0.07906191]    1       1       1  
6       cs      [0.06035487]    [0.43318322]    [0.92131241]    [0.2761722]     2       2       2  
7       scen    [0.17652737]    [0.48733082]    [0.92889245]    [0.26094251]    1       1       1  
7       mlco    [0.59738418]    [0.88078391]    [0.92889245]    [0.11131516]    1       1       1  
7       cs      [0.05034304]    [0.40315363]    [0.92889245]    [0.27132531]    2       2       2  
8       scen    [0.17652737]    [0.55815155]    [0.93928777]    [0.24418593]    1       1       1  
8       mlco    [0.56185613]    [0.90453632]    [0.93928777]    [0.09201446]    1       1       1  
8       cs      [0.04045196]    [0.46675558]    [0.93928777]    [0.26574865]    2       2       2  
9       scen    [0.16641506]    [0.55297915]    [0.94121292]    [0.23495566]    1       1       1  
9       mlco    [0.530488]      [0.9034545]     [0.94121292]    [0.10004736]    1       1       1  
9       cs      [0.03303344]    [0.46697758]    [0.94121292]    [0.26392307]    2       2       2  
10      scen    [0.14928594]    [0.5248227]     [0.93836818]    [0.23606981]    1       1       1  
10      mlco    [0.47751463]    [0.86317255]    [0.93836818]    [0.1523894]     1       1       1  
10      cs      [0.02764604]    [0.43169139]    [0.93836818]    [0.25634494]    2       2       2  
11      scen    [0.13224397]    [0.51000396]    [0.93573428]    [0.24458403]    1       1       1  
11      mlco    [0.38282932]    [0.80449361]    [0.93573428]    [0.20165409]    1       1       1  
11      cs      [0.02376931]    [0.39717601]    [0.93573428]    [0.25148415]    2       2       2  
12      scen    [0.12116625]    [0.51151563]    [0.95156251]    [0.2672405]     1       1       1  
12      mlco    [0.34234666]    [0.80925369]    [0.95156251]    [0.22711859]    1       1       1  
12      cs      [0.03056912]    [0.40248187]    [0.95156251]    [0.26908261]    2       2       2  
13      scen    [0.11629279]    [0.49319215]    [0.94661997]    [0.26527459]    1       1       1  
13      mlco    [0.33071164]    [0.88510441]    [0.94661997]    [0.15029952]    1       1       1  
13      cs      [0.02889342]    [0.42079539]    [0.94661997]    [0.27210703]    2       2       2  
14      scen    [0.03361135]    [0.46565447]    [0.95469873]    [0.27991709]    1       1       1  
14      mlco    [0.86016258]    [0.93258813]    [0.95469873]    [0.03462779]    1       1       1  
14      cs      [0.02638058]    [0.43324067]    [0.95469873]    [0.27397941]    2       2       2  
15      scen    [0.03090772]    [0.45330783]    [0.961646]      [0.28009647]    1       1       1  
15      mlco    [0.84624815]    [0.94931718]    [0.961646]      [0.02775933]    1       1       1  
15      cs      [0.02416802]    [0.42447737]    [0.961646]      [0.27590525]    2       2       2  
16      scen    [0.03041835]    [0.44202793]    [0.96544326]    [0.27898215]    1       1       1  
16      mlco    [0.8135197]     [0.94268297]    [0.96544326]    [0.0439307]     1       1       1  
16      cs      [0.02229784]    [0.4051148]     [0.96544326]    [0.27124226]    2       2       2  
17      scen    [0.02918605]    [0.43077332]    [0.96891381]    [0.27800259]    1       1       1  
17      mlco    [0.82471617]    [0.95647675]    [0.96891381]    [0.03536462]    1       1       1  
17      cs      [0.02130849]    [0.4031421]     [0.96891381]    [0.27122333]    2       2       2  
18      scen    [0.02791384]    [0.42208136]    [0.96914247]    [0.2750266]     1       1       1  
18      mlco    [0.81252847]    [0.94833346]    [0.96914247]    [0.04741444]    1       1       1  
18      cs      [0.01990977]    [0.38908737]    [0.96914247]    [0.26756288]    2       2       2  
19      scen    [0.01990977]    [0.40969892]    [0.97063579]    [0.27579959]    1       1       1  
19      mlco    [0.96302093]    [0.9658489]     [0.97063579]    [0.00239575]    1       1       1  
19      cs      [0.01844416]    [0.38952834]    [0.97063579]    [0.27261269]    2       2       2  
20      scen    [0.01838532]    [0.40043245]    [0.97001694]    [0.27419114]    1       1       1  
20      mlco    [0.95941225]    [0.96492808]    [0.97001694]    [0.00424176]    1       1       1  
20      cs      [0.01712846]    [0.37880077]    [0.97001694]    [0.27037116]    2       2       2  
21      scen    [0.01970553]    [0.39146921]    [0.96875708]    [0.27249585]    1       1       1  
21      mlco    [0.96109125]    [0.96598857]    [0.96875708]    [0.00262888]    1       1       1  
21      cs      [0.01625861]    [0.36581134]    [0.96875708]    [0.26969507]    2       2       2  
22      scen    [0.01668221]    [0.38290105]    [0.9663991]     [0.26986622]    1       1       1  
22      mlco    [0.95520738]    [0.9633274]     [0.9663991]     [0.00410066]    1       1       1  
22      cs      [0.01564081]    [0.36409675]    [0.9663991]     [0.26878069]    2       2       2  
23      scen    [0.01603244]    [0.37709224]    [0.96441343]    [0.26919123]    1       1       1  
23      mlco    [0.94488579]    [0.96185982]    [0.96441343]    [0.00459627]    1       1       1  
23      cs      [0.01506825]    [0.35985133]    [0.96441343]    [0.26741092]    2       2       2  
24      scen    [0.01534919]    [0.3705945]     [0.96224205]    [0.26844458]    1       1       1  
24      mlco    [0.93416926]    [0.95570324]    [0.96224205]    [0.00650845]    1       1       1  
24      cs      [0.01446315]    [0.3550484]     [0.96224205]    [0.26601353]    2       2       2  
25      scen    [0.01621287]    [0.41496057]    [0.96745466]    [0.31073007]    1       1       1  
25      mlco    [0.93239995]    [0.95571099]    [0.96745466]    [0.0090585]     1       1       1  
25      cs      [0.01390478]    [0.37022516]    [0.96745466]    [0.28657672]    2       2       2  
26      scen    [0.01559848]    [0.46076677]    [0.96505446]    [0.30195726]    1       1       1  
26      mlco    [0.91847173]    [0.94110909]    [0.96505446]    [0.01062504]    1       1       1  
26      cs      [0.01338791]    [0.39515281]    [0.96505446]    [0.29141962]    2       2       2  
27      scen    [0.01502895]    [0.46607991]    [0.96364502]    [0.3134495]     1       1       1  
27      mlco    [0.91038373]    [0.93544697]    [0.96364502]    [0.0130212]     1       1       1  
27      cs      [0.0129081]     [0.40653275]    [0.96364502]    [0.28478293]    2       2       2  
28      scen    [0.01453612]    [0.42548635]    [0.95009035]    [0.28178472]    1       1       1  
28      mlco    [0.8836971]     [0.91715813]    [0.95009035]    [0.01632614]    1       1       1  
28      cs      [0.01240782]    [0.40046969]    [0.95009035]    [0.27089433]    2       2       2  
29      scen    [0.01502895]    [0.42356233]    [0.96497581]    [0.28331238]    1       1       1  
29      mlco    [0.87028206]    [0.90590296]    [0.96497581]    [0.02397106]    1       1       1  
29      cs      [0.00779322]    [0.38619027]    [0.96497581]    [0.26567316]    2       2       2  
30      scen    [0.01435507]    [0.42056631]    [0.955613]      [0.28266386]    1       1       1  
30      mlco    [0.85956423]    [0.89185926]    [0.955613]      [0.0215557]     1       1       1  
30      cs      [0.00739327]    [0.38530432]    [0.955613]      [0.26425755]    2       2       2  
31      scen    [0.01377186]    [0.41683121]    [0.94249672]    [0.28115496]    1       1       1  
31      mlco    [0.8483434]     [0.87636726]    [0.94249672]    [0.02006534]    1       1       1  
31      cs      [0.00704958]    [0.38360126]    [0.94249672]    [0.26262278]    2       2       2  
32      scen    [0.01317367]    [0.41457416]    [0.93290233]    [0.28069088]    1       1       1  
32      mlco    [0.84000581]    [0.87197954]    [0.93290233]    [0.02465118]    1       1       1  
32      cs      [0.00671287]    [0.37685207]    [0.93290233]    [0.26165921]    2       2       2  
33      scen    [0.01268086]    [0.41208544]    [0.92589688]    [0.28025124]    1       1       1  
33      mlco    [0.83254712]    [0.8590174]     [0.92589688]    [0.01995741]    1       1       1  
33      cs      [0.00974343]    [0.37866166]    [0.92589688]    [0.26149271]    2       2       2  
34      scen    [0.01222358]    [0.41071686]    [0.91839148]    [0.28002889]    1       1       1  
34      mlco    [0.82694999]    [0.85673753]    [0.91839148]    [0.02560652]    1       1       1  
34      cs      [0.00936346]    [0.37824051]    [0.91839148]    [0.26211579]    2       2       2  
35      scen    [0.01175002]    [0.36726485]    [0.93739564]    [0.28504601]    1       1       1  
35      mlco    [0.84511051]    [0.87280537]    [0.93739564]    [0.02673374]    1       1       1  
35      cs      [0.00898392]    [0.35342759]    [0.93739564]    [0.28921985]    2       2       2  
36      scen    [0.01131178]    [0.44445503]    [0.93705445]    [0.27757811]    1       1       1  
36      mlco    [0.84590272]    [0.86761693]    [0.93705445]    [0.02047017]    1       1       1  
36      cs      [0.00863394]    [0.3690468]     [0.93705445]    [0.28267034]    2       2       2  
37      scen    [0.01090506]    [0.48966773]    [0.95713211]    [0.26605965]    1       1       1  
37      mlco    [0.86786326]    [0.89020115]    [0.95713211]    [0.01977105]    1       1       1  
37      cs      [0.00831021]    [0.39917885]    [0.95713211]    [0.28440274]    2       2       2  
38      scen    [0.01052657]    [0.53641475]    [0.97213255]    [0.31017436]    1       1       1  
38      mlco    [0.88352542]    [0.91266711]    [0.97213255]    [0.02629443]    1       1       1  
38      cs      [0.00798767]    [0.4391918]     [0.97213255]    [0.29614276]    2       2       2  
39      scen    [0.01015554]    [0.54196557]    [0.97598348]    [0.31348759]    1       1       1  
39      mlco    [0.89809796]    [0.92584053]    [0.97598348]    [0.02319248]    1       1       1  
39      cs      [0.00768923]    [0.44932433]    [0.97598348]    [0.30228067]    2       2       2  
40      scen    [0.00979311]    [0.5210877]     [0.97952034]    [0.29560345]    1       1       1  
40      mlco    [0.91124676]    [0.93731492]    [0.97952034]    [0.01983627]    1       1       1  
40      cs      [0.00739327]    [0.4545293]     [0.97952034]    [0.30861847]    2       2       2  
41      scen    [0.32182935]    [0.70650931]    [0.98119688]    [0.24984869]    1       1       1  
41      mlco    [0.91745232]    [0.94275366]    [0.98119688]    [0.01862704]    1       1       1  
41      cs      [0.05497799]    [0.59254341]    [0.98119688]    [0.24187595]    2       2       2  
42      scen    [0.56056234]    [0.92314131]    [0.98273099]    [0.14225767]    1       1       1  
42      mlco    [0.96612301]    [0.97612743]    [0.98273099]    [0.0056759]     1       1       1  
42      cs      [0.31613122]    [0.86086235]    [0.98273099]    [0.17080027]    2       2       2  
43      scen    [0.60274611]    [0.94161896]    [0.98358693]    [0.09776093]    1       1       1  
43      mlco    [0.98215118]    [0.98280343]    [0.98358693]    [0.00049985]    1       1       1  
43      cs      [0.29272947]    [0.89359795]    [0.98358693]    [0.15202824]    2       2       2  
44      scen    [0.67523131]    [0.929415]      [0.98421415]    [0.08655687]    1       1       1  
44      mlco    [0.97898819]    [0.98288698]    [0.98421415]    [0.00121634]    1       1       1  
44      cs      [0.26703202]    [0.88442494]    [0.98421415]    [0.14822114]    2       2       2  
45      scen    [0.73190465]    [0.89580505]    [0.97732649]    [0.07326781]    1       1       1  
45      mlco    [0.92930985]    [0.95502902]    [0.97732649]    [0.01276733]    1       1       1  
45      cs      [0.24426581]    [0.83610565]    [0.97732649]    [0.14488598]    2       2       2  
46      scen    [0.88953699]    [0.92665198]    [0.95081281]    [0.01729799]    1       1       1  
46      mlco    [0.90306913]    [0.92780927]    [0.95081281]    [0.01287089]    1       1       1  
46      cs      [0.75650055]    [0.89092787]    [0.95081281]    [0.03467792]    2       2       2  
47      scen    [0.8489215]     [0.92202749]    [0.98156361]    [0.03139957]    1       1       1  
47      mlco    [0.91527888]    [0.94911984]    [0.98156361]    [0.02074708]    1       1       1  
47      cs      [0.73361299]    [0.89356896]    [0.98156361]    [0.04614472]    2       2       2  
48      scen    [0.90944702]    [0.93392487]    [0.98367588]    [0.02738131]    1       1       1  
48      mlco    [0.90847433]    [0.94332172]    [0.98367588]    [0.02322059]    1       1       1  
48      cs      [0.80446653]    [0.92019644]    [0.98367588]    [0.03031236]    2       2       2  
49      scen    [0.84367142]    [0.92063665]    [0.98192203]    [0.04004019]    1       1       1  
49      mlco    [0.89443217]    [0.9297737]     [0.98192203]    [0.02611619]    1       1       1  
49      cs      [0.75664295]    [0.89043913]    [0.98192203]    [0.04008608]    2       2       2  
50      scen    [0.80831813]    [0.93379075]    [0.98389172]    [0.0498924]     1       1       1  
50      mlco    [0.96887208]    [0.97885027]    [0.98389172]    [0.00410537]    1       1       1  
50      cs      [0.66638312]    [0.8915569]     [0.98389172]    [0.07396943]    2       2       2  
51      scen    [0.87638091]    [0.94893083]    [0.98476932]    [0.04074105]    1       1       1  
51      mlco    [0.9674507]     [0.97920832]    [0.98476932]    [0.00498718]    1       1       1  
51      cs      [0.79420446]    [0.92050447]    [0.98476932]    [0.04451147]    2       2       2  
52      scen    [0.90736402]    [0.96532593]    [0.98515695]    [0.03011228]    1       1       1  
52      mlco    [0.96929585]    [0.97975968]    [0.98515695]    [0.00446666]    1       1       1  
52      cs      [0.79922313]    [0.93087597]    [0.98515695]    [0.03655173]    2       2       2  
53      scen    [0.87997852]    [0.96727084]    [0.9846911]     [0.03114015]    1       1       1  
53      mlco    [0.94524537]    [0.95951525]    [0.9846911]     [0.01315056]    1       1       1  
53      cs      [0.86188678]    [0.9230622]     [0.9846911]     [0.03281955]    2       2       2  
54      scen    [0.93834738]    [0.98058358]    [0.98492038]    [0.01129657]    1       1       1  
54      mlco    [0.98182936]    [0.98298111]    [0.98492038]    [0.00094062]    1       1       1  
54      cs      [0.89896064]    [0.95702433]    [0.98492038]    [0.02717098]    2       2       2  
55      scen    [0.98218262]    [0.98402211]    [0.98572741]    [0.00138155]    1       1       1  
55      mlco    [0.98226709]    [0.98421973]    [0.98572741]    [0.00106559]    1       1       1  
55      cs      [0.92418537]    [0.96931237]    [0.98572741]    [0.01579334]    2       2       2  
56      scen    [0.97809657]    [0.98394311]    [0.98621746]    [0.00213096]    1       1       1  
56      mlco    [0.98223082]    [0.98498014]    [0.98621746]    [0.00147118]    1       1       1  
56      cs      [0.92137372]    [0.96852369]    [0.98621746]    [0.01639857]    2       2       2  
57      scen    [0.97690329]    [0.98206847]    [0.98627594]    [0.00315076]    1       1       1  
57      mlco    [0.97952101]    [0.98395806]    [0.98627594]    [0.00186953]    1       1       1  
57      cs      [0.90845137]    [0.96520355]    [0.98627594]    [0.01958849]    2       2       2  
58      scen    [0.94580868]    [0.9781251]     [0.98663306]    [0.01129777]    1       1       1  
58      mlco    [0.96334471]    [0.98237071]    [0.98663306]    [0.00745654]    1       1       1  
58      cs      [0.87977412]    [0.95670941]    [0.98663306]    [0.02749673]    2       2       2  
59      scen    [0.96217369]    [0.98122138]    [0.98659653]    [0.0081745]     1       1       1  
59      mlco    [0.98284712]    [0.98519438]    [0.98659653]    [0.00110629]    1       1       1  
59      cs      [0.90352915]    [0.96596326]    [0.98659653]    [0.02392067]    2       2       2  
60      scen    [0.94409646]    [0.97715306]    [0.9870261]     [0.01422771]    1       1       1  
60      mlco    [0.98325157]    [0.98563345]    [0.9870261]     [0.00102225]    1       1       1  
60      cs      [0.87849452]    [0.95643409]    [0.9870261]     [0.03089736]    2       2       2  
61      scen    [0.97373523]    [0.98506449]    [0.98691841]    [0.00344786]    1       1       1  
61      mlco    [0.98321097]    [0.98584673]    [0.98691841]    [0.00112832]    1       1       1  
61      cs      [0.90748741]    [0.97488553]    [0.98691841]    [0.01705194]    2       2       2  
62      scen    [0.98353397]    [0.9868024]     [0.98737853]    [0.00088353]    1       1       1  
62      mlco    [0.98341469]    [0.98631086]    [0.98737853]    [0.00126832]    1       1       1  
62      cs      [0.92339324]    [0.98066669]    [0.98737853]    [0.01209319]    2       2       2  
63      scen    [0.98703273]    [0.98728473]    [0.98746295]    [0.00011977]    1       1       1  
63      mlco    [0.98338958]    [0.98627746]    [0.98746295]    [0.00126276]    1       1       1  
63      cs      [0.95045826]    [0.9820561]     [0.98746295]    [0.00925277]    2       2       2  
64      scen    [0.86461749]    [0.97884926]    [0.98763003]    [0.03054958]    1       1       1  
64      mlco    [0.98401948]    [0.98638783]    [0.98763003]    [0.0010183]     1       1       1  
64      cs      [0.84515426]    [0.96839374]    [0.98763003]    [0.03943688]    2       2       2  
65      scen    [0.98701967]    [0.98746309]    [0.98772518]    [0.00031817]    1       1       1  
65      mlco    [0.98601534]    [0.98729724]    [0.98772518]    [0.00044895]    1       1       1  
65      cs      [0.93530033]    [0.98167904]    [0.98772518]    [0.01110297]    2       2       2  
66      scen    [0.98743663]    [0.98766766]    [0.98784068]    [0.00013637]    1       1       1  
66      mlco    [0.98055602]    [0.98711572]    [0.98784068]    [0.00177348]    1       1       1  
66      cs      [0.9001527]     [0.97981565]    [0.98784068]    [0.01479672]    2       2       2  
67      scen    [0.98699812]    [0.98763333]    [0.98797783]    [0.00035054]    1       1       1  
67      mlco    [0.9826589]     [0.98716804]    [0.98797783]    [0.0012767]     1       1       1  
67      cs      [0.91101395]    [0.98092115]    [0.98797783]    [0.01217074]    2       2       2  
68      scen    [0.98665308]    [0.98777061]    [0.98813731]    [0.00042437]    1       1       1  
68      mlco    [0.98440513]    [0.98718121]    [0.98813731]    [0.00101988]    1       1       1  
68      cs      [0.92018544]    [0.9812495]     [0.98813731]    [0.00988669]    2       2       2  
69      scen    [0.98581555]    [0.98752391]    [0.98826518]    [0.00080613]    1       1       1  
69      mlco    [0.98460091]    [0.98712425]    [0.98826518]    [0.00115631]    1       1       1  
69      cs      [0.92941668]    [0.98091224]    [0.98826518]    [0.00816339]    2       2       2  
70      scen    [0.98617597]    [0.98726948]    [0.98841491]    [0.00064702]    1       1       1  
70      mlco    [0.98402359]    [0.98680327]    [0.98841491]    [0.00128052]    1       1       1  
70      cs      [0.9366998]     [0.98068432]    [0.98841491]    [0.00760167]    2       2       2  
71      scen    [0.93517782]    [0.98382393]    [0.98834607]    [0.01308369]    1       1       1  
71      mlco    [0.98445644]    [0.98725717]    [0.98834607]    [0.00113783]    1       1       1  
71      cs      [0.83089601]    [0.97343547]    [0.98834607]    [0.02218763]    2       2       2  
72      scen    [0.93612386]    [0.98407703]    [0.98834658]    [0.01289798]    1       1       1  
72      mlco    [0.9848039]     [0.98745609]    [0.98834658]    [0.0010581]     1       1       1  
72      cs      [0.83457901]    [0.97401758]    [0.98834658]    [0.02180731]    2       2       2  
73      scen    [0.93702573]    [0.98434671]    [0.98852987]    [0.01273106]    1       1       1  
73      mlco    [0.9851378]     [0.98767007]    [0.98852987]    [0.00100561]    1       1       1  
73      cs      [0.83807833]    [0.97457063]    [0.98852987]    [0.02145145]    2       2       2  
74      scen    [0.93800991]    [0.9846292]     [0.98879271]    [0.0125464]     1       1       1  
74      mlco    [0.98643759]    [0.98807993]    [0.98879271]    [0.00081002]    1       1       1  
74      cs      [0.84173498]    [0.97545074]    [0.98879271]    [0.02064777]    2       2       2  
75      scen    [0.94094592]    [0.98408758]    [0.98875918]    [0.01189487]    1       1       1  
75      mlco    [0.98638753]    [0.98806045]    [0.98875918]    [0.00079365]    1       1       1  
75      cs      [0.84753287]    [0.97540258]    [0.98875918]    [0.01910891]    2       2       2  
76      scen    [0.94537417]    [0.98376174]    [0.98907965]    [0.01088863]    1       1       1  
76      mlco    [0.98605741]    [0.98802285]    [0.98907965]    [0.0009653]     1       1       1  
76      cs      [0.85494943]    [0.97559913]    [0.98907965]    [0.01719869]    2       2       2  
77      scen    [0.94957998]    [0.98403821]    [0.98932134]    [0.01019574]    1       1       1  
77      mlco    [0.98574508]    [0.98801161]    [0.98932134]    [0.00123042]    1       1       1  
77      cs      [0.86196318]    [0.97579849]    [0.98932134]    [0.01607826]    2       2       2  
78      scen    [0.95755323]    [0.98387837]    [0.98927046]    [0.0089259]     1       1       1  
78      mlco    [0.98633747]    [0.98799752]    [0.98927046]    [0.00102692]    1       1       1  
78      cs      [0.87235363]    [0.97548333]    [0.98927046]    [0.01607611]    2       2       2  
79      scen    [0.96004767]    [0.98349328]    [0.98883769]    [0.00931442]    1       1       1  
79      mlco    [0.98617677]    [0.98794074]    [0.98883769]    [0.0008576]     1       1       1  
79      cs      [0.94111301]    [0.97471641]    [0.98883769]    [0.01337394]    2       2       2  
80      scen    [0.94904816]    [0.98309871]    [0.98912309]    [0.01173559]    1       1       1  
80      mlco    [0.98623794]    [0.98780759]    [0.98912309]    [0.00114776]    1       1       1  
80      cs      [0.93247553]    [0.96997958]    [0.98912309]    [0.01615504]    2       2       2  
81      scen    [0.93847981]    [0.98138475]    [0.98945883]    [0.01525175]    1       1       1  
81      mlco    [0.98498316]    [0.98837793]    [0.98945883]    [0.0011882]     1       1       1  
81      cs      [0.92237918]    [0.96451097]    [0.98945883]    [0.01992483]    2       2       2  
82      scen    [0.92916652]    [0.97793951]    [0.98956323]    [0.01808314]    1       1       1  
82      mlco    [0.98563739]    [0.98845639]    [0.98956323]    [0.00107689]    1       1       1  
82      cs      [0.91418524]    [0.95968003]    [0.98956323]    [0.02359916]    2       2       2  
83      scen    [0.91666965]    [0.97285614]    [0.9894049]     [0.02098305]    1       1       1  
83      mlco    [0.98712642]    [0.98824384]    [0.9894049]     [0.00055267]    1       1       1  
83      cs      [0.90727779]    [0.95631318]    [0.9894049]     [0.0263019]     2       2       2  
84      scen    [0.90773651]    [0.96273151]    [0.98960314]    [0.02373129]    1       1       1  
84      mlco    [0.9864907]     [0.98848519]    [0.98960314]    [0.0010512]     1       1       1  
84      cs      [0.89782034]    [0.94192381]    [0.98960314]    [0.02909775]    2       2       2  
85      scen    [0.8932321]     [0.95276725]    [0.98824252]    [0.02735553]    1       1       1  
85      mlco    [0.97654326]    [0.98529283]    [0.98824252]    [0.00246842]    1       1       1  
85      cs      [0.8651981]     [0.93036429]    [0.98824252]    [0.03063311]    2       2       2  
86      scen    [0.87445011]    [0.93450444]    [0.98627431]    [0.02884172]    1       1       1  
86      mlco    [0.96233617]    [0.97181079]    [0.98627431]    [0.00803766]    1       1       1  
86      cs      [0.82182059]    [0.91318803]    [0.98627431]    [0.03160463]    2       2       2  
87      scen    [0.86005951]    [0.91926287]    [0.98932272]    [0.03084041]    1       1       1  
87      mlco    [0.94785066]    [0.95917457]    [0.98932272]    [0.01234839]    1       1       1  
87      cs      [0.80619396]    [0.89959739]    [0.98932272]    [0.03161437]    2       2       2  
88      scen    [0.84084265]    [0.90125595]    [0.98823391]    [0.03456355]    1       1       1  
88      mlco    [0.92370394]    [0.95312883]    [0.98823391]    [0.02260342]    1       1       1  
88      cs      [0.75604367]    [0.87929273]    [0.98823391]    [0.04037577]    2       2       2  
89      scen    [0.81901133]    [0.88496628]    [0.98582876]    [0.03823113]    1       1       1  
89      mlco    [0.91833234]    [0.94750706]    [0.98582876]    [0.02427776]    1       1       1  
89      cs      [0.72586688]    [0.86107439]    [0.98582876]    [0.0467012]     2       2       2  
90      scen    [0.80898976]    [0.86612344]    [0.96775425]    [0.03685094]    1       1       1  
90      mlco    [0.89016473]    [0.91768637]    [0.96775425]    [0.023577]      1       1       1  
90      cs      [0.61863042]    [0.82928811]    [0.96775425]    [0.05702532]    2       2       2  
91      scen    [0.78796498]    [0.85114386]    [0.94881913]    [0.03639211]    1       1       1  
91      mlco    [0.78690065]    [0.90337024]    [0.94881913]    [0.03874348]    1       1       1  
91      cs      [0.501038]      [0.80552674]    [0.94881913]    [0.06858039]    2       2       2  
92      scen    [0.77687928]    [0.8392726]     [0.94069659]    [0.03668664]    1       1       1  
92      mlco    [0.87050487]    [0.90737991]    [0.94069659]    [0.02493801]    1       1       1  
92      cs      [0.6832077]     [0.80759829]    [0.94069659]    [0.0518818]     2       2       2  
93      scen    [0.76478144]    [0.82577164]    [0.92371086]    [0.03554349]    1       1       1  
93      mlco    [0.85662942]    [0.89918475]    [0.92371086]    [0.02559893]    1       1       1  
93      cs      [0.67094827]    [0.78694607]    [0.92371086]    [0.05630376]    2       2       2  
94      scen    [0.75048452]    [0.80971562]    [0.90374015]    [0.03409316]    1       1       1  
94      mlco    [0.83075682]    [0.87009616]    [0.90374015]    [0.02554737]    1       1       1  
94      cs      [0.58049207]    [0.76281585]    [0.90374015]    [0.06038146]    2       2       2  
95      scen    [0.73916883]    [0.7971197]     [0.88801227]    [0.03307734]    1       1       1  
95      mlco    [0.82728132]    [0.85544188]    [0.88801227]    [0.0236312]     1       1       1  
95      cs      [0.6409912]     [0.75192759]    [0.88801227]    [0.05799152]    2       2       2  
96      scen    [0.72938613]    [0.78624319]    [0.87447082]    [0.03221561]    1       1       1  
96      mlco    [0.81608195]    [0.84096377]    [0.87447082]    [0.02196094]    1       1       1  
96      cs      [0.64145381]    [0.76244759]    [0.87447082]    [0.04505884]    2       2       2  
97      scen    [0.71985276]    [0.77561598]    [0.86132404]    [0.03135716]    1       1       1  
97      mlco    [0.79180919]    [0.83590342]    [0.86132404]    [0.02327517]    1       1       1  
97      cs      [0.56734278]    [0.74088095]    [0.86132404]    [0.05318666]    2       2       2  
98      scen    [0.70713458]    [0.76181957]    [0.84393001]    [0.03065522]    1       1       1  
98      mlco    [0.44618824]    [0.79399865]    [0.84393001]    [0.09748883]    1       1       1  
98      cs      [0.05738829]    [0.68124642]    [0.84393001]    [0.12712867]    2       2       2  
99      scen    [0.70368653]    [0.75047418]    [0.8336203]     [0.0295743]     1       1       1  
99      mlco    [0.774086]      [0.81158746]    [0.8336203]     [0.01997002]    1       1       1  
99      cs      [0.61467098]    [0.72126893]    [0.8336203]     [0.04427711]    2       2       2  

As shown, the max fitness increases consistently and then drops a bit towards the end. I will think about it more and write my thoughts on it in later comments. But in short, it seems the algorithm is going towards the higher fitness value. Nevertheless, I am not sure why the maximum fitness drops even a bit given our update archive strategy. I will try this with the alternative strategies that always keep the best individual and report the results here.

SepShr commented 2 years ago

Before moving on to evaluations, I ran a test run for multiple hours and discovered inconsistent behavior in the progress of fitness values. In that run, the fitness values were decreasing as opposed to increasing.

To address understand the reason for the unexpected behavior of the search, I ran numerous benchmark tests with different population sizes and number of generations with multiple strategies to update the population archives (iCCEA vs. elitist or best+random approaches).

After our discussing the results, we reached the following conclusions:

donghwan-shin commented 2 years ago

I was debugging the code using the MTQ problem.

Observation: redundant individuals in a population due to lack of mutation

There are several redundant individuals in a population, mainly because mutation_rate was not 1 and we simply append the archived individuals to the next population. Since we already maintain the same individuals for the next generation using the population archive, the remaining seats should be filled with mutated individuals, meaning that we should use mutation_rate = 1.

So, I used the following parameter values:

# Search hyperparameters
scenario_population_size = 5  # Size of the scenario population
mlco_population_size = 5  # Size of the MLC output population
min_distance = 0.5  # Minimum distance between members of an archive
region_radius = 0.05  # The radius of the region for fitness evaluations
number_of_generations = 40
random_seed = None
max_num_evals = 51200

# Evolution hyperparameters
tournament_selection = 2
crossover_probability = 0.5
guassian_mutation_mean = 0
guassian_mutation_std = 0.1
guassian_mutation_probability = 1
integer_mutation_probability = 1
bitflip_mutation_probability = 1

After this, we either stuck at a certain fitness score or find a good solution. The fitness score does not significantly decrease even when the population size is 5.

For the stuck, I think I know the reason. It's simply because, for each individual in a population, the number of collaborations is too small (i.e., only 5 since there are at most 5 individuals in the collaboration population), and therefore it does not result in a good individual. Furthermore, since the two populations evolve together based on insufficient collaboration, any of the individuals does not evolve stably. I think this is an inherent limitation of CCEAs.

Intermediate conclusion

To conclude, using this commit 4bd1086600ebca47ab2aaf5a3eeba3c46f282932, we can at least overcome the problem of significantly decreasing fitness values. And it seems clear that the small population size does not work for a complex problem. How to improve the performance even for a relatively small population size is still an open question, but I guess the fitness function has no issue at least. We may just need to fine-tune hyperparameters.