PRIMA bonds does not seem to work

AquaPore commented 8 months ago

Dear Prima,

A great thanks for developing so powerfull optimisation.

The optimiser BOBYQA

X, info = PRIMA.bobyqa(X -> optIndivSoil.OF_HYDROLAB(hydro, iZ, K_KΨobs, N_KΨobs, N_θΨobs, Of_Sample, optim, option, optionₘ, param, X, θ_θΨobs, Ψ_KΨobs, Ψ_θΨobs), Initial; xl=Lower, xu=Upper, iprint=false)

I get the following warning

Warning: BOBYQA: Invalid RHOBEG; BOBYQA requires 0 < RHOBEG <= MINVAL(XU-XL)/2; it is set to MINVAL(XU-XL)/4. Warning: BOBYQA: Invalid RHOEND; it should be a positive number and RHOEND <= RHOBEG; it is set to 1.000000000000000E-006.

The optimiser lincoa

The X propososed parameters are unfortunately not between Lowerand Upper, did i set the model properly?

Thanks, Joseph

zaikunzhang commented 8 months ago

Thank you @AquaPore .

Could you show how you called the solvers, and what results you got exactly?

AquaPore commented 8 months ago

Thanks for case 1: I did not get any results for case 2 I checked that the optimised parameters were not between the lower and upper limit.

zaikunzhang commented 8 months ago

Thank you @AquaPore . Could you show us exactly how you called the solvers, including the values of the input and output?

zaikunzhang commented 8 months ago

Hi @AquaPore ,

I get the following warning

Warning: BOBYQA: Invalid RHOBEG; BOBYQA requires 0 < RHOBEG <= MINVAL(XU-XL)/2; it is set to MINVAL(XU-XL)/4. Warning: BOBYQA: Invalid RHOEND; it should be a positive number and RHOEND <= RHOBEG; it is set to 1.000000000000000E-006. for case 1: I did not get any results

These warnings are not errors, and they should not stop the solvers from generating a result.
But they do suggest that rhobeg was not set properly for some reason.
Could you tell us your values of lower, upper, rhobeg and rhoend?
In addition, it will be also helpful to set iprint = 3 for debugging.

for case 2 I checked that the optimised parameters were not between the lower and upper limit.
Again, what are the values of lower, upper, rhobeg and rhoend?
What is the result you got?
Since LINCOA is an infeasible method, the result may slightly violate the constraints (they would be satisfied asymptotically). See https://github.com/libprima/prima/issues/42

Thank you. I hope I could help, but I would need the above-mentioned information.

AquaPore commented 8 months ago

Dear Zaikunzhang,

A great thanks for making PRIMA open source. I run sucessfully with BlackBoxOptim and I am looking forward to compare it with PRIMA. In my case I will not use LINCOA or COBYLA as it does not respect the bounds and causes NaN.

Here are the values of lower and upper lower = [0.432, 0.0, 0.00726561011740471, 1.000001, 2.3978952727983707] upper =[0.528, 0.10099999999999999, 0.008408001452570776, 5.0, 13.122365377402328] Initial = (Lower + Upper) .* 0.5 X, info = PRIMA.bobyqa(X -> optIndivSoil.OF_HYDROLAB(hydro, iZ, K_KΨobs, N_KΨobs, N_θΨobs, Of_Sample, optim, option, optionₘ, param, X, θ_θΨobs, Ψ_KΨobs, Ψ_θΨobs), Initial; xl=Lower, xu=Upper, iprint=2)

As I can see in your documentation PRIMA there is nowhere mention that we need rhobeg and rhoend values so I did not put them in the keyword.

The results are

New RHO =  1.000000000000000E-004   Delta =  2.376737615547252E-004
Number of function values = 61   Least value of F =  6.334467234840234E-002
The corresponding X is:
 4.831833410636296E-001   3.800000000000000E-002   1.383715366455852E-002   2.312022386730834E+000
 6.133813413934306E+000

The warning is important to remove because it slows down the optimisation

Many thanks for your help, Joseph

zaikunzhang commented 8 months ago

Dear Zaikunzhang,

A great thanks for making PRIMA open source. I run sucessfully with BlackBoxOptim and I am looking forward to compare it with PRIMA. In my case I will not use LINCOA or COBYLA as it does not respect the bounds and causes NaN.

Here are the values of lower and upper lower = [0.432, 0.0, 0.00726561011740471, 1.000001, 2.3978952727983707] upper =[0.528, 0.10099999999999999, 0.008408001452570776, 5.0, 13.122365377402328] Initial = (Lower + Upper) .* 0.5 X, info = PRIMA.bobyqa(X -> optIndivSoil.OF_HYDROLAB(hydro, iZ, K_KΨobs, N_KΨobs, N_θΨobs, Of_Sample, optim, option, optionₘ, param, X, θ_θΨobs, Ψ_KΨobs, Ψ_θΨobs), Initial; xl=Lower, xu=Upper, iprint=2)

As I can see in your documentation PRIMA there is nowhere mention that we need rhobeg and rhoend values so I did not put them in the keyword.

See te table of the Keywords.

The results are

New RHO =  1.000000000000000E-004   Delta =  2.376737615547252E-004
Number of function values = 61   Least value of F =  6.334467234840234E-002
The corresponding X is:
 4.831833410636296E-001   3.800000000000000E-002   1.383715366455852E-002   2.312022386730834E+000
 6.133813413934306E+000

The warning is important to remove because it slows down the optimisation

Could you try one (and only one) of the following? This should remove the warning.

Set

scale to [0.1 0.1 0.001 1.0 1.0] rhobeg to 0.25 rhoend to 1.0e-6

OR:

Set

rhobeg to 2.5e-4 rhoend to 1.0e-8

AquaPore commented 8 months ago

Thanks for your help:

Unfortunately option 1 did not work X, info = PRIMA.bobyqa(X -> optIndivSoil.OF_HYDROLAB(hydro, iZ, K_KΨobs, N_KΨobs, N_θΨobs, Of_Sample, optim, option, optionₘ, param, X, θ_θΨobs, Ψ_KΨobs, Ψ_θΨobs), Initial; xl=Lower, xu=Upper, rhobeg=2.5e-4, rhoend=1.0e-8, scale=[0.1, 0.1, 0.001, 1.0, 1.0, 1.0])

Option 2 did not work X, info = PRIMA.bobyqa(X -> optIndivSoil.OF_HYDROLAB(hydro, iZ, K_KΨobs, N_KΨobs, N_θΨobs, Of_Sample, optim, option, optionₘ, param, X, θ_θΨobs, Ψ_KΨobs, Ψ_θΨobs), Initial; xl=Lower, xu=Upper, rhobeg=0.25, rhoend=1.0e-6, scale=[0.1, 0.1, 0.001, 1.0, 1.0, 1.0])

Option 3 did not work X, info = PRIMA.bobyqa(X -> optIndivSoil.OF_HYDROLAB(hydro, iZ, K_KΨobs, N_KΨobs, N_θΨobs, Of_Sample, optim, option, optionₘ, param, X, θ_θΨobs, Ψ_KΨobs, Ψ_θΨobs), Initial; xl=Lower, xu=Upper, rhobeg=2.5e-4, rhoend=1.0e-8)

The warning Warning: BOBYQA: Invalid RHOBEG; BOBYQA requires 0 < RHOBEG <= MINVAL(XU-XL)/2; it is set to MINVAL(XU-XL)/4.

Thanks, Joseph

zaikunzhang commented 8 months ago

Hi Joseph,

scale to [0.1 0.1 0.001 1.0 1.0]

X, info = PRIMA.bobyqa(X -> optIndivSoil.OF_HYDROLAB(hydro, iZ, K_KΨobs, N_KΨobs, N_θΨobs, Of_Sample, optim, option, optionₘ, param, X, θ_θΨobs, Ψ_KΨobs, Ψ_θΨobs), Initial; xl=Lower, xu=Upper, rhobeg=2.5e-4, rhoend=1.0e-8, scale=[0.1, 0.1, 0.001, 1.0, 1.0, 1.0])

First of all, the scale should be [0.1 0.1 0.001 1.0 1.0] rather than [0.1, 0.1, 0.001, 1.0, 1.0, 1.0].

If things still do not work after setting scale as above, could you try the following?

Set

rhobeg = 0.25 minimum of (upper - lower); `rhoend = 1.0e-4rhobeg`

but do not set any value to scale.

Thank you.

BTW, my name is Zaikun ZHANG. People call me Zaikun (preferred), Prof. Zhang (only my students), or simply @zaikunzhang , or just "Hi", but not Zaikunzhang; I appreciate your efforts.

AquaPore commented 8 months ago

Thanks Zaikun ZHANG,

I am optimizing 6 parameters so the scale should be a vector so I proposescale=[0.1, 0.1, 0.001, 1.0, 1.0, 1.0] which works as it needs to be a Vector.

Thanks the following now works but not sure it is the most efficient for simple curve fitting:

`elseif 🎏_Model == :Prima Lower, Upper = optimizeOptim.SEARCHRANGE_OPTIM(optionₘ, optim) Initial = (Lower + Upper) . 0.5 rhobeg=2.5e-8 rhobeg = max(0.25 minimum(Upper .- Lower), rhobeg) rhoend=1.0e-4 * rhobeg

X, info = PRIMA.bobyqa(X -> optIndivSoil.OF_HYDROLAB(hydro, iZ, K_KΨobs, N_KΨobs, N_θΨobs, Of_Sample, optim, option, optionₘ, param, X, θ_θΨobs, Ψ_KΨobs, Ψ_θΨobs), Initial; xl=Lower, xu=Upper, rhobeg=rhobeg, rhoend=rhoend)`

The following code works, which parameter controles the accuracy of the simulation?

A great thanks, Joseph

zaikunzhang commented 8 months ago

Thank you Joseph for the explanation.

Thanks Zaikun ZHANG,

It is a pleasure to help.

I am optimizing 6 parameters so the scale should be a vector so I proposescale=[0.1, 0.1, 0.001, 1.0, 1.0, 1.0] which works as it needs to be a Vector.

However, your lower and upper mentioned above have only five components, not six. Thus I was guessing that you had five decision variables. Did I miss something?

Note that I am not a Julia user. I have absolutely no knowledge of the syntax of this language.

Thanks the following now works but not sure it is the most efficient for simple curve fitting:

`elseif 🎏_Model == :Prima Lower, Upper = optimizeOptim.SEARCHRANGE_OPTIM(optionₘ, optim) Initial = (Lower + Upper) . 0.5 rhobeg=2.5e-8 rhobeg = max(0.25 minimum(Upper .- Lower), rhobeg) rhoend=1.0e-4 * rhobeg

X, info = PRIMA.bobyqa(X -> optIndivSoil.OF_HYDROLAB(hydro, iZ, K_KΨobs, N_KΨobs, N_θΨobs, Of_Sample, optim, option, optionₘ, param, X, θ_θΨobs, Ψ_KΨobs, Ψ_θΨobs), Initial; xl=Lower, xu=Upper, rhobeg=rhobeg, rhoend=rhoend)`

I would not be surprised if it does not work quite well. According to the bounds lower and upper, your problem is not well-scaled. That is why I proposed setting

scale = [0.1 0.1 0.001 1.0 1.0], rhobeg = 0.25, and rhoend = 1.0e-6 .

N.B.: If you take this setting, you should set all the three parameters together as above.

The following code works, which parameter controles the accuracy of the simulation?

It is rhoend, which should reflect the accuracy required in the final solution.

Thanks.

Zaikun

emmt commented 8 months ago

Hello Zaikun and Joseph,

However, your lower and upper mentioned above have only five components, not six. Thus I was guessing that you had five decision variables. Did I miss something?

The Julia interface of PRIM forbid (see https://github.com/libprima/PRIMA.jl/blob/bda4ba7ae787089330e7a48db3df7943d5de11ad/src/PRIMA.jl#L389C1-L395C1) having different sizes for the variables, any of the bounds, or the scaling factors. So I do not see how this could be possible. @AquaPore can you provide us with a complete example code so that we can fix this issue please?

AquaPore commented 8 months ago

Sorry I optimised with 6 parameters.

` Lower, Upper = optimizeOptim.SEARCHRANGE_OPTIM(optionₘ, optim) Initial = (Lower + Upper) .* 0.5

            rhobeg=2.5e-8
            rhobeg = max(0.25 * minimum(Upper .- Lower), rhobeg)
            rhoend=1.0e-6 * rhobeg

            X, info = PRIMA.bobyqa(X -> optIndivSoil.OF_HYDROLAB(hydro, iZ, K_KΨobs, N_KΨobs, N_θΨobs, Of_Sample, optim, option, optionₘ, param, X, θ_θΨobs, Ψ_KΨobs, Ψ_θΨobs), Initial; xl=Lower, xu=Upper, rhobeg=rhobeg, rhoend=rhoend)

`

It works with no warning but BlackBoxOptim outperforms the accuracy. I am performing some compex optimisation of soils so it is challanging to show my problem. in a few lines

zaikunzhang commented 8 months ago

Sorry I optimised with 6 parameters.

` Lower, Upper = optimizeOptim.SEARCHRANGE_OPTIM(optionₘ, optim) Initial = (Lower + Upper) .* 0.5

Thank you, Joseph.

Could you by any chance show us the actual values of lower and upper, so that we could suggest the correct scale?

Your previously mentioned lower and upper only have five components, so they seem incorrect.

In addition, if you are concerned about the accuracy, you could try setting maxfun = 100000 (a large number) and rhoend = 1.0e-16 (a tiny number).

Thanks.

emmt commented 8 months ago

Hello,

It works with no warning but BlackBoxOptim outperforms the accuracy. I am performing some compex optimisation of soils so it is challanging to show my problem. in a few lines

Can you be more specific about the precision?

As pointed by Zaikun, the rhoend parameter can be tuned to set the precision of the solution. To be more precise, the precision of i-th variable is roughly rhoend*scale[i] with rhoend and scale two keywords of the algorithm. By default, rhobeg=1e-4 and scale=1. Also pointed by Zaikun, you have to realize that the maximum number of function calls (keyword maxfun, 100 times the number of variables by default) may have to be increased when better accuracy is required.

libprima / PRIMA.jl

PRIMA bonds does not seem to work #22