dlee138
commented
9 years ago Penalty functions can use different mathematical algorithms, such as a constant penalty functions and distance based penalty functions. In general, penalty functions can be defined by the following: penalized objective function = unpenalized objective function + penalty.
In a constant penalty function, a constant penalty is applied to an unfeasible solution, there is no consideration given to how “off” the solution is. An example of a constant penalized objective function would be : fp(x)=f(x)+ ∑Ci δi (from i=1,2…m), where δi=1 if constraint at i is violated, 0 if not and Ci is a constant penalty value for the particular constraint at i. The penalized objective function is fp(x), the “original” unpenalized objective function is f(x) and the penalty term is ∑Ci δi.
Distance-based penalty functions simply modifies this equation by replacing the binary δi term with a distance based term (di), where di= δi*gi(x) for i = 1, 2, …q and |hi(x)|, for i=q+1,…m. For i=1...q, this function behaves similarly to the constant penalty function. However, for i>q, the penalty will be applied for any distance between the solution and constraint value.
Both of these penalty function examples are static penalty functions, but there are dynamic penalty functions as well, where “length of search” (t), which can be defined as number of solutions searched, is factored into the equation, making the penalized objective function a function of both x and t.
We mentioned penalizing in qualitative terms and cross validation as a way to get around having to define a penalty function. What are some examples of a mathematically defined penalty function?