SimCMinMax / AutoSimC

Python script to create multiple profiles for Simcraft to find Best-in-Slot and best enchants/gems/talents combination.
GNU General Public License v3.0
54 stars 17 forks source link

Grab Filter percentile option. #21

Open scamille opened 6 years ago

scamille commented 6 years ago

Suggestion:

Nappy - Today at 2:19 AM
i have a suggestion for autosimc/simc: I would like to specify for example percentile=75, to not optimze for mean dps but slightly favor higher variance solutions?(edited)
is this somehow possible already?

This is not yet possible, but should be relatively simple to add.

nyri commented 6 years ago

I will try to clarify on this using terms of an genetic algorithm:

So basically we have a set of profiles. For each profile we determine the fitness using simc with a predefined target error. After we did this we have a selection process where we select all profiles with a fitness > max fitness - target error * 2 for the next stage.

My suggestion was NOT to change the selection process to always choose the top n% of profiles as an alternative to the target error method. My idea was basically to change the fitness function to not use the mean dps output of simc, but for example mean + x * standard variance or said percentile based dps value.

My motivation behind that is that real life experience shows that players tend to favor options which might be a little bit worse on mean dps but better in for example in 25% of the pulls for parses. The opposite might be true for progression, where consistency is key.

scamille commented 6 years ago

Thanks for clarifying.

As I wrote on Disc, the problem here is that I don't have any statistical measurement for the variance of the dps_percentile. The dps_error value only holds for the mean, based on central limit theorem.

This could be added with top_n grabbing, if that is still something you are interested in.

I personally don't want to encourage top_n grab method, exactly because it is statistically invalid to just rate profiles by their dps.