Unconverged Problem w/ NOMAD 4.4 MATLAB Interface Using "DISCO_MADS"

[Arrowstar]

I am attempting to use the new "DISCO_MADS_OPTIMIZATION" and "DISCO_MADS_HID_CONST" functionality in NOMAD 4.4 with the MATLAB interface. I have modified the "example_block_eval" run_nomad.m code slightly to try this out, but the problem does not converge. Here is my slightly modified run_nomad.m script:

% CRESCENT 5

% Parallel pool must be started separately (for multiple runs) or in the
% script (single run)
%parpool(2);

x0 = [0 0 0 0 0 ];
lb = [-6 -6 -6 -6 -6 ];
ub = [6 6 6 6 6];

params = struct('display_degree','2','max_bb_eval','100','direction_type','ortho 2n','bb_output_type','OBJ PB PB','bb_max_block_size','1', 'disco_mads_optimization','true', 'disco_mads_hid_const','true', 'QUAD_MODEL_SEARCH','false'); 

% Start optimization

% Without callback
[x,fval,hinf,exit_status,nfeval] = nomadOpt(@fun,x0,lb,ub,params);

% With callback
% [x,fval,hinf,exit_status,nfeval] = nomadOpt(@fun,x0,lb,ub,params,@cbFun);

Notice that the max block size has been set to 1 and I have turned on the two options I mentioned above. When doing do, I get the following unconverged output.

>> run_nomad
Warning: DiscoMads is used to reveal hidden constraints, so the detection radius and limit rate will not be considered. Detection radius is forced to 0.

------------------------------------------------------------------
 This is NOMAD v4.4.0
 Authors: C. Audet, S. Le Digabel, V. Rochon Montplaisir and C. Tribes
 MEX Interface C. Tribes 2023 

 Problem Properties:
 # Decision Variables:                  5
 # Number of Objectives:                1
 # Number of Nonlinear Constraints:     3
------------------------------------------------------------------
Insufficient outputs provided by the black box function. Exiting NOMAD...

X0 evaluation failed for X0 = ( 0 0 0 0 0 )
A termination criterion is reached: No termination (all). Problem with starting point evaluation (Algo) No more points to evaluate

Best feasible solution:     Undefined.

Best infeasible solution:   Undefined.

Blackbox evaluations: 1
NO solution obtained

The solution correctly solves in the example as shipped with NOMAD:

>> run_nomad

------------------------------------------------------------------
 This is NOMAD v4.4.0
 Authors: C. Audet, S. Le Digabel, V. Rochon Montplaisir and C. Tribes
 MEX Interface C. Tribes 2023 

 Problem Properties:
 # Decision Variables:                  5
 # Number of Objectives:                1
 # Number of Nonlinear Constraints:     2
------------------------------------------------------------------
1   0       
2   1       
5  -1       
10  -1       
16  -1       
51  -3       
53  -4       
A termination criterion is reached: Maximum number of blackbox evaluations (Eval Global) No more points to evaluate 100

Best feasible solution:     #13876 ( 1 1 1 1 -4 )   Evaluation OK    f =  -4                         h =   0                     

Best infeasible solution:   #29571 ( 1.09 1 1 1.03 -4 ) Evaluation OK    f =  -4                         h =   0.00008099999999999999

Blackbox evaluations:         100
Total model evaluations:      27410
Cache hits:                   18
Total number of evaluations:  118
Feasible solution obtained

Any idea what might be going on here? It's hard for me to tell if I'm using the software incorrectly, if this is just the way the new DISCO_MADS algorithm works, or if there's a bug. Please advise. Thank you!

[Arrowstar]

I will add that changing the bb_output_type string to include -R terms did not alter the MATLAB output:

'bb_output_type','OBJ-R PB-R PB-R'

>> run_nomad
Warning: DiscoMads is used to reveal hidden constraints, so the detection radius and limit rate will not be considered. Detection radius is forced to 0.

------------------------------------------------------------------
 This is NOMAD v4.4.0
 Authors: C. Audet, S. Le Digabel, V. Rochon Montplaisir and C. Tribes
 MEX Interface C. Tribes 2023 

 Problem Properties:
 # Decision Variables:                  5
 # Number of Objectives:                1
 # Number of Nonlinear Constraints:     3
------------------------------------------------------------------
Insufficient outputs provided by the black box function. Exiting NOMAD...

X0 evaluation failed for X0 = ( 0 0 0 0 0 )
A termination criterion is reached: No termination (all). Problem with starting point evaluation (Algo) No more points to evaluate

Best feasible solution:     Undefined.

Best infeasible solution:   Undefined.

Blackbox evaluations: 1
NO solution obtained

[Arrowstar]

The more I look at this, the more I get the sense that I'm not doing something correctly. Unfortunately, the documentation for the new DISCO_MADS functionality does not explain what the new "-R" outputs need to be or how the functionality should be used. In addition to my original request above, I'd like to also ask that the documentation gets an update to better explain what's going on. I've taken a look at the examples and, despite understanding what is going on in that code, I still don't understand what I could be doing wrong. Thank you!

[ctribes]

Thanks for reporting this bug.

The same problem on the batch version works. So it is related to the Matlab interface.

The pb is that DiscoMads adds an additional constraint (we can see that Nomad start reports # Number of Nonlinear Constraints: 3 but there are 2 constraints BB_OUTPUT_TYPE OBJ PB PB). I will modify the Matlab interface to detect this and work around the test that triggers "Insufficient outputs provided by the black box function".

About the "PB-R". In the user guide we briefly describes the additional mechanisms provided by DiscoMads to reveal hidden constraints or discontinuities. By default not all outputs are scanned for discontinuities. The "-R" added to "PB" is used to specify which constraint is considered. I will clarify this in the user guide. Please note that $NOMAD_HOME/bin/nomad -h DISCO or nomadOpt('-h DISCO') in Matlab can be used to have more information about DiscoMads parameters.

[Arrowstar]

Great, thank you! I look forward to using the fixed version. :)

[ctribes]

I have pushed the modifs into master branch (#158). Test pass on the crescent problem.

[Arrowstar]

I have pushed the modifs into master branch (#158). Test pass on the crescent problem.

I see the example now succeeding too.

[Arrowstar]

Thanks for reporting this bug.

The same problem on the batch version works. So it is related to the Matlab interface.

The pb is that DiscoMads adds an additional constraint (we can see that Nomad start reports # Number of Nonlinear Constraints: 3 but there are 2 constraints BB_OUTPUT_TYPE OBJ PB PB). I will modify the Matlab interface to detect this and work around the test that triggers "Insufficient outputs provided by the black box function".

About the "PB-R". In the user guide we briefly describes the additional mechanisms provided by DiscoMads to reveal hidden constraints or discontinuities. By default not all outputs are scanned for discontinuities. The "-R" added to "PB" is used to specify which constraint is considered. I will clarify this in the user guide. Please note that $NOMAD_HOME/bin/nomad -h DISCO or nomadOpt('-h DISCO') in Matlab can be used to have more information about DiscoMads parameters.

In the documentation updates, could you also please describe what a "hidden constraint" is? Thank you!

[ctribes]

Documentation has been updated in Master branch